U.S. Department of Registries for Evaluating Patient Outcomes: A User’s Guide Health and Human Services

Agency for Healthcare Research and Quality 540 Gaither Road Rockville, MD 20850

Volume 1

Registries for Evaluating Patient Outcomes: A User’s Guide

Third Edition

AHRQ Pub. No. 13(14)-EHC111 April 2014 ISBN: 978-1-58763-432-1 The Effective Health Care Program of the Agency for Healthcare Research and Quality (AHRQ) conducts and supports research focused on the outcomes, effectiveness, comparative clinical effectiveness, and appropriateness of pharmaceuticals, devices, and health care services. More information on the Effective Health Care Program can be found at www.effectivehealthcare.ahrq.gov. This report was produced under contract to AHRQ by the Outcome DEcIDE Center (Developing Evidence to Inform Decisions about Effectiveness) under Contract No. 290-2005-00351 TO7. The AHRQ Task Order Officer for this project was Elise Berliner, Ph.D. The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ or the U.S. Department of Health and Human Services. Therefore, no statement in this report should be construed as an official position of AHRQ or the U.S. Department of Health and Human Services.

Copyright Information: ©2014 United States Government, as represented by the Secretary of the Department of Health and Human Services, by assignment. All rights reserved. The Agency for Healthcare Research and Quality (AHRQ) permits members of the public to reproduce, redistribute, publicly display, and incorporate this work into other materials provided that it must be reproduced without any changes to the work or portions thereof, except as permitted as fair use under the U.S. Copyright Act. This work contains certain tables and figures noted herein that are subject to copyright owned by third parties. These tables and figures may not be reproduced, redistributed, or incorporated into other materials independent of this work without permission of the third-party copyright owner(s). This work may not reproduced, reprinted, or redistributed for a fee, nor may the work be sold for profit or incorporated into a profit-making venture without the express written permission of AHRQ. This work is subject to the restrictions of Section 1140 of the Social Security Act, 42 U.S.C. § 1320b-10. When parts of this work are used or quoted, the following citation should be used:

Citation: Gliklich R, Dreyer N, Leavy M, eds. Registries for Evaluating Patient Outcomes: A User’s Guide. Third edition. Two volumes. (Prepared by the Outcome DEcIDE Center [Outcome Sciences, Inc., a Quintiles company] under Contract No. 290 2005 00351 TO7.) AHRQ Publication No. 13(14)-EHC111. Rockville, MD: Agency for Healthcare Research and Quality. April 2014. http://www.effectivehealthcare.ahrq.gov/ registries-guide-3.cfm. Registries for Evaluating Patient Outcomes: A User’s Guide Third Edition

Volume 1

Prepared for: Agency for Healthcare Research and Quality U.S. Department of Health and Human Services 540 Gaither Road Rockville, MD 20850 www.ahrq.gov

Contract No. 290-2005-00351 TO7

Prepared by: Outcome Sciences, Inc., A Quintiles Company Cambridge, MA

Senior Editors Richard E. Gliklich, M.D. Nancy A. Dreyer, M.P.H., Ph.D.

Editor Michelle B. Leavy, M.P.H.

AHRQ Publication No. 13(14)-EHC111 April 2014 Acknowledgements

The editors would like to acknowledge the efforts of the following individuals who contributed to the third edition of this document: Daniel Campion and Laura Khurana of Quintiles Outcome and Elise Berliner, Scott R. Smith, Sandy Cummings, Chris Heidenrich, and Frances Eisel of the Agency for Healthcare Research and Quality. We would especially like to thank Marion Torchia, who served as the managing editor for AHRQ for this User’s Guide.

ii Preface

This project was performed under a contract from the Agency for Healthcare Research and Quality (AHRQ) in collaboration with the Centers for Medicare & Medicaid Services (CMS) through the Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) Network of AHRQ’s Effective Health Care (EHC) Program. The purpose of the project was to update and expand Registries for Evaluating Patient Outcomes: A User’s Guide. The User’s Guide was first published in 2007 as a reference for establishing, maintaining, and evaluating the success of registries created to collect data about patient outcomes. The second edition, which provided updates to the existing topics and addressed four new topics, was published in 2010. The purpose of this revised and expanded third edition is to incorporate information on new methodological and technological advances into the existing chapters and to add 11 new chapters to address emerging topics in registry science. Both the 2007 and 2010 versions and this third edition were created with support from a large group of stakeholders. Following award of the initial project on September 29, 2005, we created a draft outline for the document, which was posted for public comment on AHRQ’s Effective Health Care Web site (http://www.effectivehealthcare.ahrq.gov) from January through March 2006. During that same period, we worked with AHRQ to create a process for selecting contributors and reviewers. We broadly solicited recommendations from a of stakeholders, including government agencies, industry groups, medical professional societies, and other experts in the field; conducted a review of the pertinent literature; and contacted the initial list of contributors to confirm their interest and area of expertise and to seek further recommendations. Through that process and in collaboration with AHRQ and CMS, we arrived at a set of contributors and reviewers based on subject/content expertise, practical experience, and interest and availability, with balanced representation from key stakeholder groups for nearly all chapters. In addition, a request for submission of real-world case examples that could be used in the user’s guide to illustrate issues and challenges in implementing registries was posted on the Effective Health Care Web site. The primary selection criteria for these examples concerned their utility in illustrating a practical challenge and its resolution. An initial meeting of contributors was convened in February 2006. A second meeting including contributors and chapter reviewers was held in June 2006, following creation of an initial draft document and focused review by the reviewers. The collaborative efforts of contributors, reviewers, and editors resulted in a draft document that was posted for public comment on the Effective Health Care Web site in October and November 2006. In all, 39 contributors and 35 individual reviewers participated in the creation of the first document, which was released in April 2007 and has been published online and in print. In August 2008, the user’s guide update project was awarded. The project involved revising the existing chapters and case examples, creating new content to address four topics, and soliciting new case examples. From September to November 2008, we worked with AHRQ to select contributors and content reviewers for the new user’s guide. We followed a process similar to that used in the creation of the original user’s guide to arrive at a set of contributors and reviewers with subject matter expertise and a broad range of perspectives. The contributors drafted white papers on four topics: use of registries in product safety assessment, when to stop a registry, interfacing registries and electronic health records, and linking registry data. The white papers were reviewed and discussed at a meeting in April 2009. The papers were then posted for public comment in August and September 2009. After the papers were revised in response to public comments, the final papers were included in the expanded user’s guide.

iii During the same timeframe, we contacted the authors and reviewers of the 2007 version of the user’s guide. We asked authors and reviewers to update the existing chapters to address any new methodological, technological, or legal topics. The revised chapters were circulated for review and discussed at a meeting in July 2009. We also posted a new call for case examples on the Effective Health Care Web site in June 2009. The primary selection criteria for the new examples concerned their utility in illustrating issues and challenges related to the new topics addressed in the white papers. In addition, we contacted authors of the original case examples to obtain updated information on the registries. In all, 55 contributors and 49 individual reviewers participated in the creation of the second edition, which was published in September 2010. The project to create the third edition of the user’s guide was awarded in September 2010. The project involved revising the existing chapters and case examples, creating new content to address 11 topics, and soliciting new case examples. From October to December 2010, we followed a process similar to that used in the creation of the second edition to select contributors and reviewers with subject matter expertise and a broad range of perspectives. Beginning in January 2011, contributors drafted white papers on 11 new topics: registry transitions, analyzing linked datasets, patient identity management, informed consent for registries, protection of registry data, public-private partnerships, using patient- reported outcome measures in registries, rare disease registries, pregnancy registries, quality improvement registries, and medical device registries. The white papers were reviewed and discussed at a series of meetings held between July and October 2011. The papers were then posted for public comment in the spring and summer of 2012. After the papers were revised in response to public comments, the final papers were included in the expanded user’s guide. During the same timeframe, we contacted the authors and reviewers of the 2010 version of the user’s guide. We asked them to update the existing chapters to address any new methodological, technological, or legal topics. The revised chapters were circulated for review and discussed at a meeting in July 2012. We also posted a new call for case examples on the Effective Health Care Web site in the spring of 2012. The primary selection criteria for the new examples concerned their utility in illustrating issues and challenges related to the new topics addressed in the white papers. In addition, we contacted authors of the original case examples to obtain updated information. For all three editions, the contributors and reviewers participated as individuals and not necessarily as representatives of their organizations. We are grateful to all those who contributed to these documents, and who reviewed them and shared their comments. To begin the discussion of registries, we would like to clarify some distinctions between registries and clinical trials. Although this subject is discussed further in Chapter 1, we offer here the following distinctions from a high-level perspective. A is an in which an active intervention intended to change a human subject’s outcome is implemented, generally through a randomization procedure that takes decisionmaking away from the practitioner. The research protocol describes inclusion and exclusion criteria that are used to select the patients who will participate as human subjects, focusing the experiment on a homogeneous group. Human subjects and clinical researchers agree to adhere to a strict schedule of visits and to conduct protocol-specific tests and measurements. In contrast, registries use an design that does not specify treatments or require any therapies intended to change patient outcomes (except insofar as specific treatments or therapies may be inclusion criteria). IInclusion and exclusion criteria are kept to a minimum in an effort to study a broad range of patients in order to make the results more generalizable. Patients are typically observed as they present for care, and the data collected generally reflect whatever tests and measurements a provider customarily uses.

iv Patient registries represent a useful tool for a number of purposes. Their ideal use and their role in evidence development, design, operations, and evaluation resemble but differ from clinical trials in a number of substantive ways, and therefore they should not be evaluated with the same constructs. This user’s guide presents what the contributors and reviewers consider good registry practices. Many registries today may not meet even the basic practices described. On the whole, registry science is in an active state of development. This third edition of the user’s guide is an important step in developing the field. This book is divided into two volumes and six sections. Volume 1 includes the first three sections: Creating Registries; Legal and Ethical Considerations for Registries; and Operating Registries. Volume 2 includes: Technical, Legal, and Analytic Considerations for Combining Registry Data with Other Data Sources; Special Applications in Registries; and Evaluating Registries. The first three sections provide basic information on key areas of registry development and operations, highlighting the spectrum of practices in each of these areas and their potential strengths and weaknesses. Section I, “Creating Registries,” contains six chapters. “Patient Registries” defines and characterizes types of registries, their purposes, and uses, and describes their place within the scope of this document. “Planning a Registry” focuses on the recommended steps in planning a registry, from determining if a registry is the right option to describing goals and objectives. “Registry Design” examines the specifics of designing a registry once the goals and objectives are known. “Data Elements for Registries” provides a scientific and practical approach to selecting data elements. “Use of Patient-Reported Outcomes in Registries” discusses the role that patient-reported outcome measures play in registries and addresses factors in selecting and using these types of measures. “Data Sources for Registries” describes how existing data sources (administrative, pharmacy, other registries, etc.) may be used to enhance the value of patient registries. Section II, “Legal and Ethical Considerations for Registries,” contains three chapters. “Principles of Registry Ethics, Data Ownership, and Privacy” reviews several key legal and ethical issues that should be considered in creating or operating a registry. “Informed Consent for Registries” discusses how the requirements of informed consent for patient registries differ from those for clinical trials and offers suggestions for creating informed consent documents that address the unique aspects of registries. “Protecting Data: Confidentiality and Legal Concerns of Providers, Manufacturers, and Health Plans” reviews the legal protections available for data about providers, manufacturers, and health plans contained in registries. Section III, “Operating Registries,” provides a practical guide to the day-to-day operational issues and decisions for producing and interpreting high-quality registries. “Recruiting and Retaining Participants in the Registry” describes strategies for recruiting and retaining providers and patients. “ and Quality Assurance” reviews key areas of data collection, cleaning, storing, and quality assurance for registries. “Adverse Event Detection, Processing, and Reporting” examines relevant practical and regulatory issues. “Analysis, Interpretation, and Reporting of Registry Data To Evaluate Outcomes” addresses key considerations in analyzing and interpreting registry data. “Modifying and Stopping Registries” discusses the process of modifying an existing registry as well as considerations for determining when to end a registry. Section IV, “Technical, Legal, and Analytic Considerations for Combining Registry Data with Other Data Sources,” reviews several issues related to the emerging trend of linking or integrating registry data with other data sources, such as electronic health records, administrative databases, or other registries.

v “Interfacing Registries With Electronic Health Records” describes the current state of electronic health record (EHR) integration technology and maps out potential options for developing interfaces between registries and EHRs. “Linking Registry Data With Other Data Sources To Support New Studies” discusses the technical and legal issues surrounding the linkage of registry data with other data sources. “Managing Patient Identity Across Data Sources” reviews the options and strategies for linking patient information stored in multiple databases without the use of full personal identifiers. “Analysis of Linked Registry Data Sets” addresses issues that must be considered when analyzing combined or linked registry data, as well as issues related to using registry data to support secondary research studies. Section V, “Special Applications in Patient Registries,” highlights several specific types of patient registries that face unique challenges. “Use of Registries in Product Safety Assessment” describes the utility and challenges of designing a registry to assess safety. “Rare Disease Registries” discusses the increasing interest in using registries to study rare diseases and the related challenges in design, recruitment, retention, and analysis. “Pregnancy Registries” reviews the value of registries for understanding the effects of medication used during pregnancy and the challenges related to design, recruitment, analysis, and dissemination of results. “Quality Improvement Registries” examines the ability of registries to support efforts to improve quality of care through the use of specialized tools and reports. “Registries for Medical Devices” addresses the unique aspects of medical devices that must be considered in the development and analysis of a device-based registry. “Public-Private Partnerships” provides a review of public-private partnership models for supporting registries as well as a discussion of major considerations for planning and operating a registry using this type of model. Interspersed throughout the first five sections of the user’s guide are case examples. As discussed above, the choice of examples was limited to those submitted for consideration during the 2006, 2009, and 2012 public submission periods. Their purpose is solely to illustrate specific points in the text using real-world examples, regardless of whether the source of the example is within the scope of the user’s guide as described in Chapter 1. Inclusion of a case example is not intended as an endorsement of the quality of the particular registry, nor do the case examples necessarily present registries that meet all the criteria described in Chapter 25 as essential elements of good practice. Rather, case examples are introduced to provide the reader with a richer description of the issue or question being addressed in the text. In some cases, we have no independent information on the registry other than what has been provided by the contributor. Section VI is “Evaluating Registries.” This final chapter on “Assessing Quality” summarizes key points from the earlier chapters in a manner that can be used to review the structure, data, or interpretations of patient registries. It describes good registry practice in terms of “essential elements” and “further indicators of quality.” This information might be used by a person developing a registry, or by a reviewer or user of registry data or interpretations derived from registries.

Richard E. Gliklich Nancy A. Dreyer Senior Editors Michelle B. Leavy Editor

vi Contents Overview

Volume 1 Executive Summary...... 1 Section I. Creating Registries...... 11 Chapter 1. Patient Registries...... 13 Chapter 2. Planning a Registry...... 29 Chapter 3. Registry Design...... 47 Chapter 4. Data Elements for Registries...... 73 Chapter 5. Use of Patient-Reported Outcomes in Registries...... 93 Chapter 6. Data Sources for Registries...... 127

Section II. Legal and Ethical Considerations for Registries...... 143 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy...... 145 Chapter 8. Informed Consent for Registries...... 187 Chapter 9. Protecting Data: Confidentiality and Legal Concerns of Providers, Manufacturers, and Health Plans...... 213

Section III. Operating Registries...... 233 Chapter 10. Recruiting and Retaining Participants in the Registry...... 235 Chapter 11. Data Collection and Quality Assurance...... 251 Chapter 12. Adverse Event Detection, Processing, and Reporting...... 277 Chapter 13. Analysis, Interpretation, and Reporting of Registry Data To Evaluate Outcomes...... 291 Chapter 14. Modifying and Stopping Registries...... 315

Volume 2 Section IV. Technical, Legal, and Analytic Considerations for Combining Registry Data With Other Data Sources...... 1 Chapter 15. Interfacing Registries With Electronic Health Records...... 3 Chapter 16. Linking Registry Data With Other Data Sources To Support New Studies...... 23 Chapter 17. Managing Patient Identity Across Data Sources...... 47 Chapter 18. Analysis of Linked Registry Data Sets...... 65

Section V. Special Applications in Patient Registries...... 89 Chapter 19. Use of Registries in Product Safety Assessment...... 91 Chapter 20. Rare Disease Registries...... 113 Chapter 21. Pregnancy Registries...... 135 Chapter 22. Quality Improvement Registries...... 171 Chapter 23. Registries for Medical Devices...... 199 Chapter 24. Public-Private Partnerships...... 215

vii Section VI. Evaluating Registries...... 239 Chapter 25. Assessing Quality...... 241

Contributors...... 253 Reviewers...... 255 Case Example Contributors...... 259 Contributor and Reviewer Affiliations...... 263 Appendixes...... 281 Appendix A. An Illustration of Size Calculations...... 283 Appendix B. Copyright Law...... 289 Appendix C. Relevant Entities in Health Information Technology Standards...... 291 Appendix D. Linking Clinical Registry Data With Insurance Claims Files...... 295

viii Contents

Contents—Volume 1

Executive Summary...... 1 Section I. Creating Registries...... 11 Chapter 1. Patient Registries...... 13 1. Introduction...... 13 2. Current Uses for Patient Registries...... 14 3. Taxonomy for Patient Registries...... 19 4. Patient Registries and Policy Purposes...... 21 5. Global Registries...... 24 6. Summary...... 24 References for Chapter 1...... 24 Chapter 2. Planning a Registry...... 29 1. Introduction...... 29 2. Steps in Planning a Registry...... 29 3. Summary...... 41 Case Example for Chapter 2...... 41 Case Example 1. Creating a Registry To Fulfill Multiple Purposes and Using a Publications Committee To Review Data Requests...... 41 References for Chapter 2...... 43 Chapter 3. Registry Design...... 47 1. Introduction...... 47 2. Research Questions Appropriate for Registries...... 47 3. Translating Clinical Questions Into Measurable Exposures and Outcomes...... 49 4. Finding the Necessary Data...... 50 5. Resources and ...... 51 6. Study Designs for Registries...... 52 7. Choosing Patients for Study...... 54 8. ...... 57 9. Registry Size and Duration...... 59 10. Internal and External Validity...... 61 11. Summary...... 64 Case Examples for Chapter 3...... 65 Case Example 2. Designing a Registry for a Health Technology Assessment...... 65 Case Example 3. Developing Prospective Nested Studies in Existing Registries...... 66 Case Example 4. Designing a Registry To Address Unique Patient Enrollment Challenges...... 68 References for Chapter 3...... 69

ix Chapter 4. Data Elements for Registries...... 73 1. Introduction...... 73 2. Identifying Domains...... 73 3. Selecting Data Elements...... 74 4. Registry Data Map...... 83 5. Pilot Testing...... 83 6. Summary...... 85 Case Examples for Chapter 4...... 85 Case Example 5. Selecting Data Elements for a Registry...... 85 Case Example 6. Understanding the Needs and Goals of Registry Participants...... 87 Case Example 7. Using Standardized Data Elements in a Registry...... 89 References for Chapter 4...... 90 Chapter 5. Use of Patient-Reported Outcomes in Registries...... 93 1. Introduction...... 93 2. The Role of PROs in Registries...... 96 3. What Methods Are Available To Collect PROs and Which Is Best?...... 99 4. Which PRO Measure(s) Should Be Selected?...... 103 5. Example of PRO Use in a Registry...... 112 Case Examples for Chapter 5...... 115 Case Example 8. Developing and Validating a Patient-Administered ...... 115 Case Example 9. Using Validated Measures To Collect Patient-Reported Outcomes...... 116 Case Example 10. Challenges in the Collection of PROs in a Longitudinal Registry...... 118 Case Example 11. Collecting PRO Data in a Sensitive Patient Population...... 119 References for Chapter 5...... 120 Chapter 6. Data Sources for Registries...... 127 1. Introduction...... 127 2. Types of Data...... 127 3. Data Sources...... 129 4. Other Considerations for Secondary Data Sources...... 138 5. Summary...... 139 Case Example for Chapter 6...... 140 Case Example 12. Using Claims Data Along With Patient-Reported Data To Identify Patients...... 140 References for Chapter 6...... 141

x Contents

Section II. Legal and Ethical Considerations for Registries...... 143 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy...... 145 1. Introduction...... 145 2. Ethical Concerns Relating to Health Information Registries...... 146 3. Applicable Regulations...... 154 4. Registry Transparency, Oversight, and Data Ownership...... 169 5. Conclusions...... 172 6. Summary of Privacy Rule and Common Rule Requirements...... 174 Case Example for Chapter 7...... 179 Case Example 13. Obtaining a Waiver of Informed Consent...... 179 References for Chapter 7...... 181 Chapter 8. Informed Consent for Registries...... 187 1. Introduction...... 187 2. Registries, Research, and Other Activities...... 187 3. Current Challenges for Registries...... 190 4. Regulatory Consent Requirements...... 193 5. A Proposed Framework for Registry Consents...... 199 6. Consent Guidance...... 202 Case Examples for Chapter 8...... 205 Case Example 14. Issues With Obtaining Informed Consent...... 205 Case Example 15. Operationalizing Informed Consent for Children...... 206 Case Example 16. Using a Patient-Centered Study Design To Collect Informed Consent, Maximize Recruitment and Retention, and Provide Meaningful Clinical Data...... 208 References for Chapter 8...... 209 Chapter 9. Protecting Data: Confidentiality and Legal Concerns of Providers, Manufacturers, and Health Plans...... 213 1. Background...... 213 2. Relevant Laws and Regulations: Variety of Sources, but Limited Protection...... 214 3. Summary...... 224 Case Examples for Chapter 9...... 224 Case Example 17. Handling Discovery Requests for Registry Data...... 224 Case Example 18. Meeting the Confidentiality and Quality Improvement Needs of Providers Through a Patient Safety Organization...... 226 Case Example 19. Protections Available to Registry Data From Institutional Review Boards and Academic Institutions...... 228 References for Chapter 9...... 229

xi Section III. Operating Registries...... 233 Chapter 10. Recruiting and Retaining Participants in the Registry...... 235 1. Introduction...... 235 2. Recruitment...... 235 3. Retention...... 241 4. Pitfalls in Recruitment and Retention...... 243 5. International Considerations...... 243 Case Examples for Chapter 10...... 244 Case Example 20. Building Value as a To Recruit Hospitals...... 244 Case Example 21. Using Registry Tools To Recruit Sites...... 245 Case Example 22. Using a Scientific Advisory Board To Support Investigator Research Projects...... 247 Case Example 23. Identifying and Addressing Recruitment and Retention Barriers in an Ongoing Registry...... 249 References for Chapter 10...... 250 Chapter 11. Data Collection and Quality Assurance...... 251 1. Introduction...... 251 2. Data Collection...... 251 3. Quality Assurance...... 263 4. Resource Considerations...... 269 Case Examples for Chapter 11...... 271 Case Example 24. Developing a Performance-Linked Access System...... 271 Case Example 25. Using Audits To Monitor Data Quality...... 273 References for Chapter 11...... 274 Chapter 12. Adverse Event Detection, Processing, and Reporting...... 277 1. Introduction...... 277 2. Identifying and Reporting Adverse Drug Events...... 277 3. Collecting AE Data in a Registry...... 280 4. AE Reporting by the Registry...... 281 5. Coding...... 283 6. Adverse Event Management...... 283 7. Adverse Event Required Reporting for Registry Sponsors...... 285 8. Special Case: Risk Evaluation and Mitigation Strategies...... 286 9. Reporting Breaches of Confidentiality or Other Risks...... 287 References for Chapter 12...... 287

xii Contents

Chapter 13. Analysis, Interpretation, and Reporting of Registry Data To Evaluate Outcomes...... 291 1. Introduction...... 291 2. Hypotheses and Purposes of the Registry...... 291 3. Patient Population...... 292 4. Data Quality Issues...... 295 5. ...... 298 6. Summary of Analytic Considerations...... 306 7. Interpretation of Registry Data...... 306 Case Examples for Chapter 13...... 308 Case Example 26. Using Registry Data To Evaluate Outcomes by Practice...... 308 Case Example 27. Using Registry Data To Study Patterns of Use and Outcomes...... 309 References for Chapter 13...... 311 Chapter 14. Modifying and Stopping Registries...... 315 1. Introduction...... 315 2. Registry Transitions...... 315 3. Planning for the End of a Patient Registry...... 329 Case Examples for Chapter 14...... 335 Case Example 28. Determining When To Stop an Open-Ended Registry...... 335 Case Example 29. Challenges in Transitions and Changes in Data Collection...... 336 Case Example 30. Transitioning From Startup to Ongoing Registry Funding With Public and Private Partners...... 338 Case Example 31. Modifying a Registry Due to Changes in Standards of Care...... 340 References for Chapter 14...... 341 Tables Table 3–1. Considerations for Study Design...... 47 Table 3–2. Overview of Registry Purposes...... 48 Table 3–3. Examples of Research Questions and Key Exposures and Outcomes...... 50 Table 4–1. Standard Terminologies...... 76 Table 4–2. Examples of Possible Baseline Data Elements...... 79 Table 4–3. Examples of Possible Additional Enrollee, Provider, and Environmental Data Elements...... 79 Table 5–1. Definitions of Commonly Encountered Terms Within PRO-Related Literature...... 95 Table 5–2. Example Guidelines for PRO Incorporation Into Product-Labeling Claims in Oncology ...... 95 Table 6–1. Key Data Sources—Strengths and Limitations...... 131 Table 7–1. Summary of Privacy Rule and Common Rule Requirements...... 175 Table 10–1. Hospital Recruitment...... 237 Table 10–2. Physician Recruitment...... 238

xiii Table 10–3. Patient Recruitment...... 240 Table 11–1. Registry Functionalities...... 263 Table 11–2. Data Activities Performed During Registry Coordination...... 270 Table 12–1. Overview of Serious Adverse Event Reporting Requirements for Marketed Products...... 285 Table 13–1. Hypothetical Simple Sensitivity Analysis...... 305 Table 14–1. Considerations in Selecting a Registry Vendor...... 321 Table 14–2. Impact of Definition Changes on Data Linkage...... 324 Table 14–3. Possible Consequences of a Change in Registry Focus...... 327 Table 14–4. Checklist of Key Considerations for a Registry Transition...... 328 Figures Figure 1–1. Deciding When To Develop a Registry: The “Value of Information” Exercise...... 23 Figure 5–1. Psychometric Properties and Logistical Considerations Exist Along a Spectrum...... 106 Figure 5–2. Simplified Concept Map for Cystic Fibrosis...... 114 Figure 12–1. Best Practices for Adverse Event Reporting to FDA by Registries of Postmarket Products...... 279 Figure 13–1. Patient Populations...... 294 Figure 13–2. The Flow of Participants Into an Analysis...... 300 Figure 14–1. Potential Impact of a Change in Outcome...... 326

xiv Executive Summary

Defining Patient Registries The plan for registry governance and oversight should clearly address such issues as overall This User’s Guide is intended to support the direction and operations, scientific content, ethics, design, implementation, analysis, interpretation, safety, data access, publications, and change and quality evaluation of registries created to management. It is also helpful to plan for the increase understanding of patient outcomes. For entire lifespan of a registry, including how and the purposes of this guide, a patient registry is an when the registry will end and any plans for organized system that uses observational study transition at that time. methods to collect uniform data (clinical and other) to evaluate specified outcomes for a Registry Design population defined by a particular disease, condition, or exposure, and that serves one or A patient registry should be designed with respect more predetermined scientific, clinical, or policy to its major purpose, with the understanding that purposes. A registry database is a file (or files) different levels of rigor may be required for derived from the registry. Although registries can registries designed to address focused analytical serve many purposes, this guide focuses on questions to support decisionmaking, in contrast to registries created for one or more of the following registries intended primarily for descriptive purposes: to describe the natural history of purposes. The key points to consider in designing disease, to determine clinical effectiveness or a registry include formulating a research question; cost-effectiveness of health care products and choosing a study design; translating questions of services, to measure or monitor safety and harm, clinical interest into measurable exposures and and/or to measure quality of care. outcomes; choosing patients for study, including Registries are classified according to how their deciding whether a comparison group is needed; populations are defined. For example, product determining where data can be found; and registries include patients who have been exposed deciding how many patients need to be studied and to biopharmaceutical products or medical devices. for how long. Once these key design issues have Health services registries consist of patients who been settled, the registry design should be have had a common procedure, clinical encounter, reviewed to evaluate potential sources of bias or hospitalization. Disease or condition registries (systematic error); these should be addressed to are defined by patients having the same diagnosis, the extent that is practical and achievable. The such as cystic fibrosis or heart failure. information value of a registry is enhanced by its ability to provide an assessment of the potential for bias and to quantify how this bias could affect the Planning a Registry study results. There are several key steps in planning a patient The specific research questions of interest will registry, including articulating its purpose, guide the registry’s design, including the choice of determining whether it is an appropriate means of exposures and outcomes to be studied and the addressing the research question, identifying definition of the target population (the population stakeholders, defining the scope and target to which the findings are meant to apply). The population, assessing feasibility, and securing registry population should be designed to funding. The registry team and advisors should be approximate the characteristics of the target selected based on their expertise and experience. population as much as possible. The number of study subjects to be recruited and the length of observation (followup) should be planned in

1 Registries for Evaulating Patient Outcomes

accordance with the overall purpose of the registry. chart abstraction. Overall, the choice of data The desired study size (in terms of subjects or elements should be guided by parsimony, validity, person-years of observation) is determined by and a focus on achieving the registry’s purpose. specifying the magnitude of an expected, clinically meaningful effect or the desired precision of effect Use of Patient-Reported estimates. Study size determinants are also affected by practicality, cost, and whether the Outcomes in Registries registry is intended to support regulatory Patient-reported outcomes (PROs) are reports of decisionmaking. Depending on the purpose of the health status taken directly from patients without registry, internal, external, or historical comparison interpretation by clinicians. PROs can provide groups strengthen the understanding of whether useful information for registries designed for many the observed effects are indeed real and in fact purposes, including natural history of disease, different from what would have occurred under quality improvement, effectiveness, and other circumstances. comparative effectiveness. Using PROs raises such Registry study designs often restrict eligibility for questions as when and how often to collect the entry to individuals with certain characteristics data, which method or combination of methods (e.g., age) to ensure that the registry will have should be used (e.g., paper-based, electronic), and subgroups with sufficient numbers of patients for which instrument(s) should be used. Many analysis. Or the registry may use some form of validated instruments and measures are available, sampling—random selection, systematic sampling, such as general assessment scales (e.g., health- or a haphazard, nonrandom approach—to achieve related quality of life), disease-specific scales, this end. symptom-specific scales, evaluations of functioning across a variety of domains (e.g., Data Elements physical, social, emotional), and scales assessing satisfaction with care received. When selecting The selection of data elements requires balancing instruments or measures, it is important to define such factors as their importance for the integrity of (1) the population of interest, (2) the outcomes of the registry and for the analysis of primary interest, (3) the intended users of the registry, and outcomes, their reliability, their contribution to the (4) the purpose of the registry. Defining these overall burden for respondents, and the factors will help determine which PROs are useful incremental costs associated with their collection. and appropriate for the study. The instrument’s Selection begins with identifying relevant validity, reliability, and ability to detect change domains. Specific data elements are then selected should also be considered. Once PROs have been with consideration for established clinical data selected, the registry should focus on consistency, standards, common data definitions, and whether across patients and across sites, with respect to patient identifiers will be used. It is important to how the instruments are administered and how determine which elements are absolutely necessary data are entered into the registry. and which are desirable but not essential. In choosing measurement scales for the assessment Data Sources of patient-reported outcomes, it is preferable to use scales that have been appropriately validated, when A single registry may integrate data from various such tools exist. Once data elements have been sources. The form, structure, availability, and selected, a data map should be created, and the timeliness of the required data are important data collection tools should be pilot tested. Testing considerations. Data sources can be classified as allows assessment of respondent burden, the primary or secondary. Primary data are collected accuracy and completeness of questions, and by the registry for its direct purposes. Secondary potential areas of . Inter-rater data have been collected by a secondary source for agreement for data collection instruments can also purposes other than the registry, and may not be be assessed, especially in registries that rely on uniformly structured or validated with the same

2 Executive Summary rigor as the registry’s primary data. Sufficient Informed Consent for identifiers are necessary to guarantee an accurate match between data from secondary sources and Registries registry patients. Furthermore, it is advisable to The requirement of informed consent often raises obtain a solid understanding of the original different issues for patient registries versus clinical purpose of the secondary data, because the way trials. For example, registries may be used for those data were collected and verified or validated public health or quality improvement activities, will help shape or limit their use in a registry. which may not constitute “human subjects Common secondary sources of data linked to research.” Also, registries may integrate data from registries include medical records systems, multiple electronic sources (e.g., claims data, institutional or organizational databases, electronic health records) and may be linked to administrative health insurance claims data, death biobanks. Institutional review boards may approve and birth records, databases, and related waivers or alterations of informed consent (e.g., existing registry databases. electronic consent, oral consent) for some registries, depending on the purpose and risk to Ethics, Data Ownership, and participants. Established registries that undergo a Privacy change in scope (e.g., changes in data sharing policies, changes to the protocol, extension of the Critical ethical and legal considerations should followup period) may need to ask patients to guide the development and use of patient “re-consent.” When planning informed consent registries. The Common Rule is the uniform set of procedures, registry developers should consider regulations on the ethical conduct of human several factors, including documentation and subjects research, issued by the Federal agencies format, consent revisions and re-consent, the that fund such research. Institutions that conduct applicability of regulatory requirements, research agree to comply with the Common Rule withdrawal of participants from the study, and the for federally funded research, and may opt to apply physical and electronic security of patient data and that rule to all human subjects activities conducted biological specimens. In addition, registry within their facilities or by their employees and developers may need to consider the individual agents, regardless of the source of funding. The authorization requirements of the HIPAA Privacy Privacy Rule, promulgated under the Health Rule, where applicable. Insurance Portability and Accountability Act of 1996 (HIPAA), establishes Federal protections for Confidentiality and Legal the privacy of individually identifiable health information created and maintained by health Concerns for Providers, plans, health care clearinghouses, and most health Manufacturers, and Health care providers (collectively, “covered entities”). Plans The purpose of a registry, the type of entity that creates or maintains the registry, the types of As patient registries are increasingly recognized as entities that contribute data to the registry, and the a valuable data source, questions about privacy and extent to which registry data are individually the confidentiality of the data arise, particularly identifiable affect how the regulatory requirements when data are desired for litigation or other apply. Other important concerns include judicial or administrative proceedings. In addition transparency of activities, oversight, and data to patient data, registries often include private, ownership. This chapter of the User’s Guide confidential, and/or proprietary information about focuses solely on U.S. law. Health information is providers, manufacturers, and health plans. While also legally protected in European and some other significant attention has been paid to protecting the countries by distinctly different rules. privacy of identifiable patient information, there is no single comprehensive Federal law governing

3 Registries for Evaulating Patient Outcomes

protection of registry data about providers, Data Collection and Quality manufacturers, or health plans. Sources of protection for these data include the Patient Safety Assurance and Quality Improvement Act of 2005, the Health The integrated system for collecting, cleaning, and Human Services Certificate of Confidentiality, storing, monitoring, reviewing, and reporting on the Agency for Healthcare Research and Quality registry data determines the utility of those data Confidentiality Statute, the Privacy Act of 1974, for meeting the registry’s goals. A broad range of the Federal Rules of Evidence and Civil Procedure, data collection procedures and systems are the Freedom of Information Act, Quality available. Some are more suitable than others for Improvement Organizations, the Federal Trade particular purposes. Critical factors in the ultimate Secrets Act, and the Patient Protection and quality of the data include how data elements are Affordable Care Act. Additional protections are structured and defined, how personnel are trained, available at the State level through safe harbor and and how data problems (e.g., missing, out-of peer review laws. Registry developers should range, or logically inconsistent values) are consider this issue during the planning phase and handled. Registries may also be required to clearly articulate the policies and procedures that conform to guidelines or to the standards of the registry will follow in the case of a request for specific end users of the data (e.g., 21 Code of registry data (e.g., from litigation attorneys, Federal Regulations, Part 11). Quality assurance regulatory authorities, the press, or members of the aims to affirm that the data were, in fact, collected public). in accordance with established procedures and that they meet the requisite standards of quality to Patient and Provider accomplish the registry’s intended purposes and Recruitment and Management the intended use of the data. Requirements for quality assurance should be Recruitment and retention of patients as registry defined during the registry’s inception and participants, and of providers as registry sites, are creation. Because certain requirements may have essential to the success of a registry. Recruitment significant cost implications, a risk-based approach typically occurs at several levels, including to developing a quality assurance plan is facilities (hospitals, physicians’ practices, and recommended. It should be based on identifying pharmacies), providers, and patients. The the most important or likely sources of error or motivating factors for participation at each level potential lapses in procedures that may affect the and the factors necessary to achieve retention quality of the registry in the context of its intended differ according to the registry. Factors that purpose. motivate participation include the perceived relevance, importance, or scientific credibility of the registry, as well as a favorable balance of any Adverse Event Detection, incentives for participation versus the risks and Processing, and Reporting burdens thereof. Because patient and provider recruitment and retention can affect how well a The U.S. Food and Drug Administration defines an registry represents the target population, well- adverse event (AE) as any untoward medical planned strategies for enrollment and retention are occurrence in a patient administered a critical. Goals for recruitment, retention, and pharmaceutical product, whether or not related to followup should be explicitly laid out in the or considered to have a causal relationship with the registry planning phase, and deviations during the treatment. AEs are categorized according to the conduct of the registry should be continuously seriousness and, for drugs, the expectedness of the evaluated for their risk of introducing bias. event. Although AE reporting for all marketed products is dependent on the principle of “becoming aware,” collection of AE data falls into

4 Executive Summary

two categories: those events that are intentionally all, important covariates; and an understanding of solicited (meaning data that are part of the uniform how missing data were handled and reported. collection of information in the registry) and those Analysis of a registry should provide information that are unsolicited (meaning that the AE is on the characteristics of the patient population, the volunteered or noted in an unsolicited manner). exposures of interest, and the endpoints. The determination of whether the registry should Descriptive registry studies focus on describing use a case report form to collect AEs should be frequency and patterns of various elements in a based on the scientific importance of the patient population, whereas analytical studies information for evaluating the specified outcomes concentrate on associations between patients or of interest. Regardless of whether or not AEs treatment characteristics and health outcomes of constitute a primary objective of the registry, it is interest. A statistical analysis plan describes the important for any registry that has direct patient analytical plans and statistical techniques that will to develop a plan for detecting, be used to evaluate the primary and secondary processing, and reporting AEs. If the registry objectives specified in the study plan. receives sponsorship, in whole or in part, from a Interpretation of registry data should be provided regulated industry (drugs or devices), the sponsor so that the conclusions can be understood in the has mandated reporting requirements including appropriate context and any lessons from the stringent timelines, and the registry should registry can be applied to the target population and establish the process for detecting and reporting used to improve patient care and outcomes. AEs and should provide training to registry personnel on how to identify AEs and to whom Modifying and Stopping they should be reported. Sponsors of registries designed specifically to meet requirements for Registries surveillance of drug or device safety are Most, if not all, registries should undergo periodic encouraged to hold discussions with health critical evaluation by key stakeholders to ensure authorities about the most appropriate process for that the objectives are being met. When registry reporting serious AEs. objectives are no longer being met or when clinical or other changes affect the registry (e.g., changes Analysis, Interpretation, and in treatment practices, the introduction of a new Reporting of Registry Data therapy), the registry may need to be adapted, or the registry may stop collecting new data. Many Analysis and interpretation of registry data begin registries will undergo a modification or transition with answering a series of core questions: Who at some point in their lifecycle, and these changes was studied, and how were they chosen for study? will vary in scope and size. A major registry How were the data collected, edited, and verified, transition is a change in the registry’s purpose, and how were missing data handled? How were the stakeholders, and/or technology platform that has a analyses performed? Four populations are of substantive impact on the ongoing conduct of the interest in describing who was studied: the target registry. Considerations for the transition of a population, the accessible population, the intended registry are similar to those for starting a registry, population, and the population actually studied but transitions can also present some unique (the “actual population”). The representativeness challenges. It is important to select a leadership of the actual population to the target population is team that will carefully plan and implement the referred to as generalizability. transition and consider the impacts of the planned Analysis of registry outcomes first requires an changes (e.g., legal and ethical issues, technology, analysis of recruitment and retention, of the and data analysis). The transition team should also completeness of data collection, and of data be prepared to handle unplanned or exigent quality. Considerations include an evaluation of circumstances that may arise during the transition losses to followup; completeness for most, if not and modify the project plan accordingly. Open, ongoing communication between the project team,

5 Registries for Evaulating Patient Outcomes

stakeholders, participants, and other resources is Linking Registry Data With key to conducting a successful transition. Other Data Sources to Support A registry may stop collecting new data because it has fulfilled its original purpose, is unable to fulfill New Studies its purpose, is no longer relevant, or is unable to Registry data may be linked to other data sources maintain sufficient funding, staffing, or other (e.g., administrative data sources, other registries) support. If an open-ended registry is planned, to examine questions that cannot be addressed reasonable goals should be set for data quality, using the registry data alone. Two equally weighted study enrollment, and the amount of information and important sets of questions must be addressed needed to address specific endpoints of interest in the data linkage planning process: (1) What is a which will inform the decision if and when to end feasible technical approach to linking the data? the registry. (2) Is linkage legally feasible under the permissions, terms, and conditions that applied to Interfacing of Registries With the original compilations of each dataset? Many Electronic Health Records statistical techniques for linking records exist (e.g., deterministic matching, probabilistic Achieving interoperability between electronic matching); the choice of a technique should be health records (EHRs) and registries will be guided by the types of data available. Linkage increasingly important as adoption of EHRs and projects should include plans for managing the use of patient registries for many purposes common issues (e.g., records that exist in only one both grow significantly. Such interoperability database and variations in units of measure). In should be based on open standards that enable any addition, it is important to understand that linkage willing provider to interface with any applicable of de-identified data may result in accidental registry without requiring customization or re-identification. Risks of re-identification vary permission from the EHR vendor. Interoperability depending on the variables used, and should be for health information systems requires accurate managed with guidance from legal and statistical and consistent data exchange and use of the experts to minimize risk and ensure compliance information that has been exchanged. Syntactic with the HIPAA Privacy Rule, the Common Rule, interoperability (the ability to exchange data) and and other legal and regulatory requirements. semantic interoperability (the ability to understand the exchanged data) are the core constructs of Managing Patient Identity interoperability and must be present in order for EHRs and registries to share data successfully. Full Across Data Sources interoperability is unlikely to be achieved for some As new technologies emerge to manage electronic time. The successive development, testing, and health care data and create new opportunities for adoption of open standard building blocks (e.g., data linkage, patient identity management (PIM) the Healthcare Information Technology Standards strategies and standards grow increasingly Panel’s HITSP TP-50) is a pragmatic approach important. If shared patient identifiers exist toward incrementally advancing interoperability between two data sources, data can be linked using while providing real benefits today. Care must be a unique patient identifier (UPI), such as a medical taken to ensure that integration efforts comply record number. The concept of a universal UPI has with legal and regulatory requirements for the been the subject of debate for some time. Some protection of patient privacy and the security of view the UPI as a tool to reduce administrative individually identifiable health information. workload and facilitate the exchange of electronic data, while others raise serious concerns about the privacy and protection of patient-identifiable information. To date, these concerns have halted efforts to implement universal UPIs in the United

6 Executive Summary

States. As a result, common PIM practices in the purposes of safety, the size of the registry, the United States include algorithms and other enrolled population, and the duration of followup statistical methods to link and combine data when are all critical characteristics to ensure validity of no shared patient identifiers are present. However, the inferences made based on the data collected. with no standardized PIM practices in place, Consideration in the design phase must also be methods can vary widely, making it difficult to given to other recognized aspects of product use in ensure the accuracy and effectiveness of data the real world (e.g., the switching of therapies linkage techniques. during followup, the use of multiple products in combination or in sequence, dose effects, delayed Analysis of Linked Registry effects, and patient compliance). Data Sets Registries designed for safety assessment should also formulate a plan that ensures that appropriate Retrospective database studies are studies that use information will reach the right stakeholders data collected for a primary purpose other than (through reporting either to the manufacturer or research (e.g., administrative databases) or data directly to the regulator) in a timely manner. collected for specific research objectives but used Stakeholders include patients, clinicians, to support secondary studies focused on different providers, product manufacturers and authorization objectives. These studies have yielded substantial holders, and payers such as private, State, and information on the incidence, prevalence, and national insurers. Registries not designed outcomes of many diseases and can be used to specifically for safety assessment should, at a generate a rapid response to emerging research minimum, ensure that standard reporting questions. However, these studies require special mechanisms for AE information are described in considerations related to conduct and the registry’s standard operating procedures and interpretation because of the possibility of are made clear to investigators. producing biased or invalid results. Challenges faced by retrospective database studies include Rare Disease Registries inaccurate measurement of exposures, outcomes, and confounders and overweighting of results A rare disease registry can be a valuable tool for because of the large study population. To avoid increasing understanding of the disease and these pitfalls, it is important to clearly define the supporting the development of treatment protocols study objective, patient population, and potential and therapies. Typical goals of a rare disease confounders and modifiers. Researchers must also registry include generating knowledge around the understand the conditions under which the data natural history, evolution, risk, and outcomes of a were collected originally. specific disease; supporting research on genetic, molecular, and physiological basis of a disease; Use of Registries for Product establishing a patient base for evaluating drug, medical devices, and orphan products; and Safety Assessment facilitating connections among affected patients, Whether as part of a postmarketing requirement or families, and clinicians. out of a desire to supplement spontaneous Stakeholders often play an important role in rare reporting, prospective product and disease disease registries. Stakeholders may include registries are also increasingly being considered as patient advocacy groups, regulatory, funding, and resources for examining unresolved safety issues public health agencies, clinicians, scientists, and/or as tools for proactive risk assessment in the industry, payers, and individuals and families. postapproval setting. Registries can be valuable Because of their limited patient population, rare tools for evaluating product safety, although they disease registries face unique planning and design are only one of many approaches to safety challenges. For example, little information may be assessments. When designing a registry for the available on the disease to guide development of a

7 Registries for Evaulating Patient Outcomes

research plan, and diagnostic criteria may be Quality Improvement complex or evolving. Disease-specific patient- reported outcome measures may not be available. Registries Long-term (even lifelong) followup may be Quality improvement (QI) registries use systematic needed. Due to these challenges, rare disease data collection and other QI tools to improve the registries may need to adapt and change over time quality of care on the local, regional, or national as knowledge increases or treatments become level. In a QI registry, patients are either exposed available. Retention of patients and providers can to a particular health service (e.g., a procedure also be difficult over the duration of the registry, registry), or they have a disease/condition that is and registry developers should monitor followup tracked over time through multiple health care rates over time to identify potential issues. Clear providers and services. Most of the steps for policies should be developed for governance, data planning a QI registry are similar to the steps used access, and publications, particularly if multiple for other types of registries, with two major stakeholders are involved. differences. First, the identification of active, engaged participants, often called “champions,” is Pregnancy Registries critical for the early success of the registry. Second, the registry must collect actionable A pregnancy exposure registry is an observational information that can be used to modify behaviors, prospective cohort of women receiving a processes, or systems of care. Actionable biopharmaceutical product(s) of interest as part of information is typically presented to providers in their routine clinical care who are enrolled the form of process of care or quality measures. voluntarily during gestation, before outcomes can The selection of these measures requires balancing be known. Participants are followed until the end the goals of the registry with the desire to meet of pregnancy or longer to systematically collect other needs for providers. In the design phase, QI information on specific pregnancy outcomes and registries can use the process of care or quality evaluate their frequency relative to a scientifically measures to drive the selection of data elements. valid reference population(s). While pregnancy Because many data elements collected in QI registries are an efficient method for evaluating the registries are often collected for other purposes effects of medications used during pregnancy, they (e.g., claims, medical records), integration with present unique challenges related to patient other data sources may be important for recruitment and retention, the choice of reference encouraging participation. Motivations for or comparator groups, ways of mitigating bias, and participation often differ from other types of generalizability of registry results. Analysis and registries, and incentives for participation focus on interpretation of data from pregnancy registries QI (e.g., recognition programs, QI tools, and also requires careful consideration. Because benchmarking reports). Reporting information is specific birth defects are rare events, pregnancy also an important component of QI registries. registries usually do not have sufficient sample Registries may report blinded or unblinded data at size/power to evaluate increased risks for specific the individual patient, provider, or institution level. defects unless the relative risks are quite large. Lastly, QI registries must be able to adapt to new Most registries compare the overall proportion of evidence and improvements in care over time, and all major defects combined in the exposed group they may face questions from institutional review to the overall proportion in the reference group. boards less familiar with these types of registries.

8 Executive Summary

Registries for Medical Devices and academia, health agencies from several countries and industry, or professional associations Medical device registries are an increasingly and public payers. During the planning phase of a important tool for capturing patients’ experience PPP, it is important to define clear, transparent with medical devices throughout the device plans for governance, with documented roles for lifecycle. Registries help bridge the gap between each stakeholder. Formal policies for analyses, device performance in clinical trial settings and in publications, and data sharing are also critical, as routine practice. However, the unique features of are plans for managing conflicts of interest. During medical devices require special consideration the operational phase, PPPs should focus on when developing a registry. Regulations and consistent communication with stakeholders to approval guidelines for medical devices differ maintain their interest. PPP registries are more greatly from those for drugs. Compared with likely to succeed if they have clear, agreed-upon drugs, device technologies tend to see more rapid goals; explicit roles and responsibilities for each change over shorter time periods, and device stakeholder; strong leaders who are respected in registries must adapt to these changes. The current the field; consistent data collection and analysis lack of unique device identifiers is also plans; and the flexibility to adapt to changing challenging— although efforts are underway to conditions. create them. In many cases, multiple devices are used, and devices may be used in combination with Evaluating Registries a drug component, further complicating efforts to examine safety and effectiveness. In addition, Although registries can provide useful providers may have different levels of experience information, there are levels of rigor that enhance with the device, which may affect patient validity and make the information from some outcomes (especially with implantable devices). registries more useful for guiding decisions. The Medical device registries should attempt to term “quality” can be applied to registries to classify all parts of a device with as much describe the confidence that the design, conduct, identifying information as possible. Many and analysis of the registry can be shown to protect registries collect information on provider training against bias and errors in inference—that is, and experience as well. An emerging trend is the erroneous conclusions drawn from the registry. ability for medical devices to transmit data directly Although there are limitations to any assessment to an electronic health record or registry. This new of quality, a quality component analysis is used technology may reduce the burden of data entry both to evaluate high-level factors that may affect for registries and improve the timeliness of registry results and to differentiate between research data. quality (which pertains to the scientific process) and evidence quality (which pertains to the data/ Public-Private Partnerships findings emanating from the research process). Quality components are classified as either “basic A public-private partnership (PPP) refers to any elements of good practice,” which can be viewed partnership in which one entity is a public agency as a checklist that should be considered for all (e.g., a government entity) and the other entity is a patient registries, or as “potential enhancements to private organization. PPPs are increasingly used as good practice,” which may strengthen the value of a means to develop patient registries, in part the information in particular circumstances. The because of a growing interest from governments results of such an evaluation should be considered and payers in using registry data to make decisions in the context of the disease area(s), the type of about approval, coverage, and public health needs. registry, and the purpose of the registry, and should Many models for PPPs exist. For example, a PPP also take into account feasibility and affordability. may involve a partnership with Federal agencies

9

Section I

Creating Registries

Chapter 1. Patient Registries

1. Introduction a particular disease, a condition (e.g., a risk factor) that predisposes [them] to the occurrence of a The purpose of this document is to serve as a health-related event, or prior exposure to guide for the design and use of patient registries substances (or circumstances) known or suspected for scientific, clinical, and health policy purposes. to cause adverse health effects.” Properly designed and executed, patient registries Other terms also used to refer to patient registries can provide a real-world view of clinical practice, include clinical registries, clinical data registries, patient outcomes, safety, and comparative disease registries, and outcomes registries.4, 5 effectiveness. This user’s guide primarily focuses on practical design and operational issues, This user’s guide focuses on patient registries that evaluation principles, and best practices. Where are used for evaluating patient outcomes. It is not topics are well covered in other materials, intended to address several other types of or uses references and/or links are provided. The goal of for registries (although many of the principles may this document is to provide stakeholders in both be applicable), such as geographically based the public and private sectors with information population registries (not based on a disease, they can use to guide the design and condition, or exposure); registries created for implementation of patient registries, the analysis public health reporting without tracking outcomes and interpretation of data from patient registries, (e.g., vaccine registries); or listing registries that and the evaluation of the quality of a registry or are used solely to identify patients with particular one of its components. Where useful, case diseases in clinical practices but are not used for examples have been incorporated to illustrate evaluating outcomes. This user’s guide is also not particular points or challenges. intended to address the wide range of studies that use secondary analyses of data collected for other 1 The term registry is defined both as the act of purposes. recording or registering and as the record or entry itself. Therefore, “registries” can refer to both Many of these other types of registries are programs that collect and store data and the included in the Registry of Patient Registries records that are so created. (RoPR) effort. RoPR is a central listing of patient registries established in 2012 by the Agency for The term patient registry is generally used to Healthcare Research and Quality (AHRQ) in distinguish registries focused on health collaboration with the National Library of information from other record sets, but there is no Medicine.6 It is designed to improve transparency consistent definition in current use. E.M. Brooke, and reduce redundancy in registry-based research. in a 1974 publication of the World Health Inclusion of all types of patient registries is Organization, further delineated registries in health important to achieve RoPR’s goals, and the system information systems as “a file of documents therefore defines patient registries broadly. containing uniform information about individual persons, collected in a systematic and In contrast to RoPR, this user’s guide focuses on comprehensive way, in order to serve a the subset of patient registries used for evaluating predetermined purpose.” 2 patient outcomes, defined as follows: The National Committee on Vital and Health • A patient registry is an organized system that Statistics3 describes registries used for a broad uses observational study methods to collect range of purposes in public health and medicine as uniform data (clinical and other) to evaluate “an organized system for the collection, storage, specified outcomes for a population defined by retrieval, analysis, and dissemination of a particular disease, condition, or exposure, and information on individual persons who have either that serves one or more predetermined scientific, clinical, or policy purposes.

13 Section I. Creating Registries

• The patient registry database describes a file (or other outcomes or to assess pharmacy use or files) derived from the registry. resource utilization), as discussed in Chapter 6. Based on these definitions, the user’s guide focuses Data from patient registries are generally used for on patient registries in which the following are true studies that address the purpose for which the (although exceptions may apply): registry was created. In some respects, such as the • The data are collected in a naturalistic manner, collection of detailed clinical and longitudinal such that the management of patients is followup data, studies derived from the patient determined by the caregiver and patient registries described in this user’s guide resemble together and not by the registry protocol. traditional observational cohort studies. Beyond traditional cohort studies, however, some registry- • The registry is designed to fulfill specific based studies may be more flexible in that the purposes, and these purposes are defined before scope and focus of the data collection activity of collecting and analyzing the data. In other the registry may be adapted over time to address words, the data collection is purpose driven additional needs. For example, new studies, such rather than the purpose being data driven as cluster-randomized studies or case-control (meaning limited to or derived from what is studies, may be nested within an ongoing registry, already available in an existing ). and the database derived from the registry may be • The registry captures data elements with used to support secondary studies, such as studies specific and consistent data definitions. that link the registry database with other data sources to explore new questions. • The data are collected in a uniform manner for every patient. This consideration refers to both the types of data and the frequency of their 2. Current Uses for Patient collection. Registries

• The data collected include data derived from A patient registry can be a powerful tool to and reflective of the clinical status of the observe the course of disease; to understand patient (e.g., history, examination, laboratory variations in treatment and outcomes; to examine test, or patient-reported data). Registries factors that influence prognosis and quality of life; include the types of data that clinicians would to describe care patterns, including appropriateness use for the diagnosis and management of of care and disparities in the delivery of care; to patients. assess effectiveness; to monitor safety and harm; • At least one element of registry data collection and to measure quality of care. Through is active, meaning that some data are collected functionalities such as feedback of data, registries specifically for the purpose of the registry are also being used to study quality improvement.7 (usually collected from the patient or clinician) Different stakeholders perceive and may benefit rather than inferred from sources that are from the value of registries in different ways. For collected for another purpose (administrative, example, for a clinician, registries can collect data billing, pharmacy databases, etc.). This about disease presentation and outcomes on large definition does not exclude situations where numbers of patients rapidly, thereby producing a registry data collection is a specific, but not the real-world picture of disease, current treatment exclusive, reason data are being collected, such practices, and outcomes. For a physician as might be envisioned with future uses of organization, a registry might provide data that can electronic health records, as described in be used to assess the degree to which clinicians are Chapter 15. This definition also does not managing a disease in accordance with evidence- exclude the incorporation of other data sources. based guidelines, to focus attention on specific Registries can be enriched by linkage with aspects of a particular disease that might otherwise extant databases (e.g., to determine deaths and be overlooked, or to provide data for clinicians to

14 Chapter 1. Patient Registries

compare themselves with their peers.8 For patients 2.1 Evaluating Patient Outcomes and patient advocacy organizations, a registry may Studies from patient registries and randomized increase understanding of the natural history of a controlled trials (RCTs) have important and disease, contribute to the development of treatment complementary roles in evaluating patient guidelines, or facilitate research on treatment.9, 10 outcomes.15 Ideally, patient registries collect data From a payer’s perspective, registries can provide in a comprehensive manner (with few excluded detailed information from large numbers of patients) and therefore produce outcome results patients on how procedures, devices, or that may be generalizable to a wide range of pharmaceuticals are actually used and on their patients. They also evaluate care as it is actually effectiveness in different populations. This provided, because care is not assigned, determined, information may be useful for determining or even recommended by a protocol. As a result, coverage policies.11 For a drug or device the outcomes reported may be more representative manufacturer, a registry-based study might of what is achieved in real-world practice. Patient demonstrate the performance of a product in the registries also offer the ability to evaluate patient real world, meet a postmarketing commitment or outcomes when clinical trials are not practical requirement,12 develop hypotheses, or identify (e.g., very rare diseases), and they may be the only patient populations that will be useful for product option when clinical trials are not ethically development, clinical trials design, and patient acceptable. They are a powerful tool when RCTs recruitment. The U.S. Food and Drug are difficult to conduct, such as in surgery or when Administration (FDA) has noted that “through the very long-term outcomes are desired. creation of registries, a sponsor can evaluate safety signals identified from spontaneous case reports, RCTs are controlled designed to test literature reports, or other sources, and evaluate hypotheses that can ultimately be applied to the factors that affect the risk of adverse outcomes real-world care. Because RCTs are often such as dose, timing of exposure, or patient conducted under strict constraints, with detailed characteristics.”13 inclusion and exclusion criteria (and the need for subjects who are willing to be randomized), they The use of patient registries varies by priority are sometimes limited in their generalizability. If condition, with cancer and cardiovascular disease RCTs are not generalizable to the populations to having a large number of registries and areas such which the information will be applied, they may as developmental delays or dementia, far fewer. not be sufficiently informative for decisionmaking. Overall, the use of patient registries appears to be Conversely, patient registries that observe real- active and growing. For example, a review of world clinical practice may collect all of the ClinicalTrials.gov in the area of cancer reveals information needed to assess patient outcomes in a over 270 large (more than 2,000 patients) generalizable way, but interpreting this information observational studies that would meet the criteria correctly requires analytic methodology geared to for a patient registry. Of these studies, 4 have more address the potential sources of bias that challenge than 100,000 patients, and 27 have more than observational studies. Interpreting patient registry 10,000. In some cases, the drivers for these data also requires checks of internal validity and registries have been Federal stakeholders. For sometimes the use of external data sources to example, since 2005, the FDA Center for Devices validate key assumptions (such as comparing the and Radiological Health has called for some 160 key characteristics of registry participants with postapproval studies, many of which use new or external sources in order to demonstrate the existing registries to study the real-world comparability of registry participants with the effectiveness of specific devices in community ultimate reference population). Patient registries, practice.14 The establishment of RoPR provides a RCTs, other study designs, and other data sources new resource for tracking registry development should all be considered tools in the toolbox for and use by condition, purpose, type, and multiple evidence development, each with its own other factors.6 advantages and limitations.16

15 Section I. Creating Registries

2.2 Hierarchies of Evidence designed to support the systematic reviews conducted by the Evidence-based Practice Center One question that arises in a discussion of this type (EPC) program, is conceptually similar to the is where to place studies derived from patient GRADE system.23 When using the AHRQ registries within the hierarchies of evidence that approach, reviewers typically give evidence from are frequently used in developing guidelines or observational studies a low starting grade and decisionmaking. While the definition of patient evidence from RCTs a high starting grade. These registry used in this user’s guide is intentionally initial grades can then be raised or lowered broad, the parameters of quality described in depending on the strength of the five required Chapter 25 are intended to help the user evaluate evidence domains (study limitations, directness, and identify registries that are sufficiently rigorous consistency, precision, and reporting bias).24 For observational studies for use as evidence in example, the reviewers may find that observational decisionmaking. Many registries are, or include, studies are particularly relevant for some high-quality studies of cohorts designed to address systematic review questions. The report notes: a specific problem and hypothesis. Still, even the most rigorously conducted registries, like EPCs may act on the judgment that, for certain prospective observational studies, are traditionally outcomes such as harms, observational studies placed in a subordinate position to RCTs in some have less risk of bias than do RCTs or that the commonly used hierarchies, although equal to available RCTs have a substantial risk of bias. RCTs in others.17-19 Debate continues in the In such instances, the EPC may move up the evidence community regarding these traditional initial grade for strength of evidence based on methods of grading levels of evidence, their observational studies to moderate or move underlying assumptions, their shortcomings in down the initial rating based on RCTs to assessing certain types of evidence (e.g., benefit moderate.23 vs. harm), and their interscale consistency in Reviewers may also raise or lower evidence grades 16, 20, 21 evaluating the same evidence. based on a secondary set of domains (dose- The Grading of Recommendations Assessment, response association, existence of Development, and Evaluation (GRADE) Working that would diminish an observed effect, and Group has proposed a more robust approach that strength of association). These secondary domains addresses some of the decisionmaking issues supplement the required domains and are used described in this user’s guide. As noted by the when relevant to the systematic review question. GRADE collaborators: The report explains that the secondary domains “may increase strength of evidence and are [R]andomised trials are not always feasible especially relevant for observational studies where and, in some instances, observational studies one may begin with a lower overall strength of may provide better evidence, as is generally evidence grade based on study limitations.”23 the case for rare adverse effects. Moreover, the results of randomised trials may not always be As the methods for grading evidence for different applicable—for example, if the participants purposes continue to evolve, this user’s guide can are highly selected and motivated relative to serve as a guide to help such evaluators understand the population of interest. It is therefore study quality and identify well-designed registries. essential to consider study quality, the Beyond the evidence hierarchy debate, users of consistency of results across studies, and the evidence understand the value of registries for directness of the evidence, as well as the providing complementary information that can appropriateness of the study design.22 extend the results of clinical trials to populations not studied in those trials, for demonstrating the AHRQ has also developed a guidance system for real-world effects of treatments outside of the grading the strength of evidence that recommends research setting and potentially in large subsets of a careful assessment of the potential value of affected patients, and for providing long-term observational studies. The guidance, which is

16 Chapter 1. Patient Registries

followup when such data are not available from whether or not the patient achieves full recovery, is clinical trials. dependent not only on the product’s dissolving the clot but also on the timeliness of its delivery.30, 31 2.3 Defining Patient Outcomes The focus of this user’s guide is the use of 2.4 Purposes of Registries registries to evaluate patient outcomes. An As discussed throughout this user’s guide, outcome may be thought of as an end result of a registries should be designed and evaluated with particular health care practice or intervention. respect to their intended purpose(s). Registry According to AHRQ, end results include effects purposes can be broadly described in terms of that people experience and care about.25 The patient outcomes. While there are a number of National Cancer Institute further clarifies that potential purposes for registries, this handbook “final” endpoints are those that matter to primarily discusses four major purposes: decisionmakers: patients, providers, private payers, (1) describing the natural history of disease, government agencies, accrediting organizations, or (2) determining clinical and/or cost-effectiveness, society.26, 27 Examples of these outcomes include (3) assessing safety or harm, and (4) measuring or biomedical outcomes, such as survival and improving quality of care. Other purposes of disease-free survival, health-related quality of life, patient registries mentioned but not discussed in satisfaction with care, and economic burden.28 detail in this user’s guide are for public health Although final endpoints are ultimately what surveillance and disease control. An extensive matter, it is sometimes more practical when body of literature from the last half century of creating registries to collect intermediate outcomes experience with cancer and other disease (such as whether processes or guidelines were surveillance registries is available. followed) and clinical outcomes (such as whether a 2.4.1 Describing Natural History of Disease tumor regressed or recurred) that predict success in improving final endpoints. Registries may be established to evaluate the 29 natural history of a disease, meaning its In Crossing the Quality Chasm, the Institute of characteristics, management, and outcomes with Medicine (IOM) describes the six guiding aims of and/or without treatment. The natural history may health care as providing care that is safe, effective, be variable across different groups or geographic efficient, patient-centered, timely, and equitable. regions, and it often changes over time. In many (The last three aims focus on the delivery and cases, the natural histories of diseases are not well quality of care.) While these aims are not described. Furthermore, the natural histories of outcomes per se, they generally describe the diseases may change after the introduction of dimensions of results that matter to certain therapies. As an example, patients with rare decisionmakers in the use of a health care product diseases, such as the lysosomal storage diseases, or service: Is it safe? Does it produce greater who did not previously survive to their 20s, may benefit than harm? Is it clinically effective? Does it now be entering their fourth and fifth decades of produce the desired effect in real-world practice? life, and this uncharted natural history is being first Does the right patient receive the right therapy or described through a registry.32 The role of service at the right time? Is it cost effective or registries in rare diseases is explored in Chapter efficient? Does it produce the desired effect at a 20. reasonable cost relative to other potential expenditures? Is it patient oriented, timely, and 2.4.2 Determining Effectiveness equitable? Most of the patient outcomes that Registries may be developed to determine clinical registries evaluate reflect one or more of the IOM effectiveness or cost-effectiveness in real-world guiding aims. For example, a patient presenting clinical practice. Multiple studies have with an ischemic stroke to an emergency room has demonstrated disparities between the results of a finite window of opportunity to receive a clinical trials and results in actual clinical thrombolytic drug, and the patient outcome, practice.33, 34 Furthermore, efficacy in a clinical

17 Section I. Creating Registries

trial for a well-defined population may not be With improvement in methodologies for using generalizable to other populations or subgroups of observational research for comparative interest. As an example, many important heart effectiveness research (CER), including better failure trials have focused on a predominantly methods for managing bias and better white male population with a age of understanding of the limitations,41 there is both approximately 60 years, whereas actual heart increasing interest and investment in registries for failure patients are older, more diverse, and have a CER across a number of stakeholders. Reports higher mortality rate than the patients in these from the IOM and the Congressional Budget trials.35 Similarly, underrepresentation of older Office in 2007 cited the importance of patient patients has been reported in clinical trials of 15 registries in developing comparative effectiveness different types of cancer (e.g., studies with only 25 evidence.42, 43 The Federal Coordinating Council percent of patients age 65 years and over, while the for Comparative Effectiveness Research, in its expected rate is greater than 60 percent).36 Data Report to the President and the Congress (June 30, from registries have been used to fill these gaps for 2009), defined CER as “the conduct and synthesis decisionmakers. For example, the FDA used the of research comparing benefits and harms of American Academy of Ophthalmology’s different interventions and strategies to prevent, intraocular lens registry to expand the label for diagnose, treat and monitor health conditions in intraocular lenses to younger patients.37 Registries ‘real world’ settings.”44 The report specifically may also be particularly useful for tracking identifies patient registries as a core component of effectiveness outcomes for a longer period than is CER data infrastructure. typically feasible with clinical trials. For example, More recently, the newly formed Patient-Centered some growth hormone registries have tracked Outcomes Research Institute (PCORI) has children well into adulthood. identified registries as an important potential In addition to clinical effectiveness, registries can source of data to support patient-centered be used to assess cost-effectiveness. Registries can outcomes research (PCOR). PCOR “assesses the be designed to collect cost data and effectiveness benefits and harms of preventive, diagnostic, data for use in modeling cost-effectiveness.38 Cost- therapeutic, palliative, or health delivery system effectiveness is a means to describe the interventions to inform decisionmaking, comparative value of a health care product or highlighting comparisons and outcomes that service in terms of its ability to achieve a desired matter to people; is inclusive of an individual’s outcome for a given unit of resources.39 A cost- preferences, autonomy and needs, focusing on effectiveness analysis examines the incremental outcomes that people notice and care about such as benefit of a particular intervention and the costs survival, function, symptoms, and health related associated with achieving that benefit. Cost- quality of life; incorporates a wide variety of effectiveness studies compare costs with clinical settings and diversity of participants to address outcomes measured in units such as life individual differences and barriers to expectancy or disease-free periods. Cost-utility implementation and dissemination; and studies compare costs with outcomes adjusted for investigates (or may investigate) optimizing quality of life (utility), such as quality-adjusted life outcomes while addressing burden to individuals, years (QALYs). Utilities allow comparisons to be availability of services, technology, and personnel, made across conditions because the measurement and other stakeholder perspectives.”45 40 is not disease specific. It should be noted that for Similar to their function in CER, registries are both clinical effectiveness and cost-effectiveness, expected play an important role in this new area of differences between treatments are indirect and research in part because of their ability to provide must be inferred from data analysis, simulation information on ‘real-world’ settings and broad modeling, or some mixture. patient populations. PCORI included minimum standards for the use of registries for PCOR in the Methodology Report.46 While some registries are

18 Chapter 1. Patient Registries

designed explicitly to examine questions of differences between providers or patient comparative effectiveness or patient-centered populations based on performance measures that outcomes research, many others are designed for compare treatments provided or outcomes different objectives yet still collect data that are achieved with “gold standards” (e.g., evidence- useful for these analyses. Registries that were not based guidelines) or comparative benchmarks for explicitly designed for CER or PCOR may need to specific health outcomes (e.g., risk-adjusted be augmented or linked to other data sources—for survival or infection rates). Such programs may be example, to obtain long-term outcomes data in the used to identify disparities in access to care, case of an in-hospital registry using linkage to demonstrate opportunities for improvement, claims data to evaluate blood pressure establish differentials for payment by third parties, medications.47 or provide transparency through public reporting. 2.4.3 Measuring or Monitoring Safety and Harm There are multiple examples of such differences in treatment and outcomes of patients in a range of Registries may be created to assess safety versus disease areas.48-53 Quality improvement registries harm. Safety here refers to the concept of being are described further in Chapter 22. free from danger or hazard. One goal of registries in this context may be to quantify risk or to 2.4.5 Multiple Purposes attribute it properly. Broadly speaking, patient Many registries will be developed to serve more registries can serve as an active surveillance than one of these purposes. Registries developed system for the occurrence of unexpected or for one purpose may also be modified to serve harmful events for products and services. Such additional purposes as the research, practice, or events may range from patient complaints about policy environment changes. While registries often minor side effects to severe adverse events such as serve more than one purpose, their original or fatal drug reactions or patient falls in the hospital. primary purpose generally guides their design and, Patient registries offer multiple advantages for as a result, more care is needed in evaluating active surveillance. First, the current practice of results for secondary or additional purposes. spontaneous reporting of adverse events relies on a nonsystematic recognition of an adverse event by a 3. Taxonomy for Patient clinician and the clinician’s active effort to make a Registries report to manufacturers and health authorities. Second, these events are generally reported Even limited to the definitions described above, without a denominator (i.e., the exposed or treated the breadth of studies that might be included as population), and therefore an incidence rate is patient registries is large. Patients in a registry are difficult to determine. Because patient registries typically selected based on a particular disease, can provide systematic data on adverse events and condition (e.g., a risk factor), or exposure. This the incidence of these events, they are being used user’s guide uses these common selection criteria with increasing frequency in the areas of health to develop a taxonomy or classification based on care products and services. The role of registries in how the populations for registries are defined. monitoring product safety is discussed in more Three general categories with multiple detail in Chapter 19. subcategories and combinations account for the 2.4.4 Measuring Quality majority of registries that are developed for evaluating patient outcomes. These categories Registries may be created to measure quality of include observational studies in which the patient care. The IOM defines quality as “the degree to has had an exposure to a product or service, or has which health services for individuals and a particular disease or condition, or various populations increase the likelihood of desired combinations thereof. health outcomes and are consistent with current professional knowledge.” Quality-focused registries are being used increasingly to assess

19 Section I. Creating Registries

3.1 Product Registries include registries enrolling patients undergoing a procedure (e.g., carotid endarterectomy, In the case of a product registry, the patient is appendectomy, or primary coronary intervention) exposed to a health care product, such as a drug or or admitted to a hospital for a particular diagnosis a device. The exposure may be brief, as in a single (e.g., community-acquired pneumonia). In these dose of a pharmaceutical product, or extended, as registries, one purpose of the registry is to evaluate in an implanted device or chronic usage of a the health care service with respect to the medication. outcomes. Health care service registries are Device registries may include all, or a subset, of sometimes used to evaluate the processes and patients who receive the device. A registry for all outcomes of care for quality measurement patients who receive an implantable cardioverter purposes (e.g., Get With The Guidelines® of the defibrillator, a registry of patients with hip American Heart Association, National Surgical prostheses, or a registry of patients who wear Quality Improvement Program of the Department contact lenses are all examples of device registries. of Veterans Affairs and the American College of Biopharmaceutical product registries similarly Surgeons). have several archetypes, which may include all, or subsets, of patients who receive the 3.3 Disease or Condition Registries biopharmaceutical product. For example, the Disease or condition registries use the state of a British Society for Rheumatology established a particular disease or condition as the inclusion 54 national registry of patients on biologic therapy. criterion. In disease or condition registries, the Again, the duration of exposure may range from a patient may always have the disease (e.g., a rare single event to a lifetime of use. Eligibility for the disease such as cystic fibrosis or Pompe disease, or registry includes the requirement that the patient a chronic illness such as heart failure, diabetes, or received the product or class of products (e.g., end-stage renal disease) or may have the disease or COX-2 inhibitors). In some cases, public health condition for a more limited period of time (e.g., authorities mandate such registries to ensure safe infectious diseases, some cancers, obesity). These use of medications. Examples include registries for registries typically enroll the patient at the time of thalidomide, clozapine, and isotretinoin. a routine health care service, although patients also Pregnancy registries represent a separate class of can be enrolled through voluntary self- biopharmaceutical product registries that focus on identification processes that do not depend on possible exposures during pregnancy and the utilization of health care services (such as Internet neonatal consequences. The FDA has a specific recruiting of volunteers). In other disease guidance focused on pregnancy exposure registries, the patient has an underlying disease or registries.55 This guidance uses the term condition, such as atherosclerotic disease, but is “pregnancy exposure registry” to refer to “a enrolled only at the time of an acute event or prospective observational study that actively exacerbation, such as hospitalization for a collects information on medical product exposure myocardial infarction or ischemic stroke. during pregnancy and associated pregnancy outcomes.” Pregnancy registries are discussed in 3.4 Combinations more detail in Chapter 21. Complicating this classification approach is the reality that these categories can be overlapping in 3.2 Health Services Registries many registries. For example, a patient with In the context of evaluating patient outcomes, ischemic heart disease may have an acute another type of exposure that can be used to define myocardial infarction and undergo a primary registries is exposure to a health care service. coronary intervention with placement of a drug- Health care services that may be used to define eluting stent and postintervention management inclusion in a registry include individual clinical with clopidogrel. This patient could be enrolled in encounters, such as office visits or hospitalizations, an ischemic heart disease registry tracking all procedures, or full episodes of care. Examples patients with this disease over time, a myocardial

20 Chapter 1. Patient Registries

infarction registry that is collecting data on from clinical trials or other data sources and may patients who present to hospitals with acute provide information on the generalizability of the myocardial infarction (cross-sectional data data from clinical trials to populations not studied collection), a primary coronary intervention in those trials. registry that includes management with and The utility of registry data for decisionmaking is without devices, a coronary artery stent registry related to three factors: the stakeholders, the limited to ischemic heart disease patients, or a primary scientific question, and the context. The clopidogrel product registry that includes patients stakeholders are those associated with the disease undergoing primary coronary interventions. or procedure that may be affected from a patient, provider, payer, regulator, or other perspective. The 3.5 Duration of Observation primary scientific question for a registry may The duration of the observational period for a relate to effectiveness, safety, or practice patterns. registry is also a useful descriptor. Observation The context includes the scientific context (e.g., periods may be limited to a single episode of care previous randomized trials and modeling efforts (e.g., a hospital discharge registry for that help to more precisely define the primary diverticulitis), or they may extend for as long as scientific question), as well as the political, the lifetime of patients with a chronic disease (e.g., regulatory, funding, and other issues that provide cystic fibrosis or Pompe disease) or patients the practical parameters around which the registry receiving a novel therapy (e.g., gene therapy). The is developed. In identifying the value of period of observation or followup depends on the information from registries, it is essential to look outcomes of interest. at the data with specific reference to the purpose and focus of the registry. 3.6 From Registry Purpose to Design From a policy perspective, there are several As will be discussed extensively in this document, scenarios in which the decision to develop a the purpose of the registry defines the registry registry may arise. One possible scenario is as focus (e.g., product vs. disease) and therefore the follows. An item or service is considered for use. registry type. A registry created for the purpose of Stakeholders in the decision collaboratively define evaluating outcomes of patients receiving a “adequate data in support of the decision at hand.” particular coronary artery stent might be designed Here, “adequate data” refers to information of as a single product registry if, for example, the sufficient relevance and quality to permit an purpose is to systematically collect adverse event informed decision. An evidence development information on the first 10,000 patients receiving strategy is selected from one of many potential the product. However, the registry might strategies (RCT, practical clinical trial, registry, alternatively be designed as a health care service etc.) based on the quality of the evidence provided registry for primary coronary intervention if a by each design, as well as the burden of data purpose is to collect comparative effectiveness or collection and the cost that is imposed. This safety data on other treatments or products within tradeoff of the quality of evidence versus cost of the same registry. data collection for each possible design is termed the “value of information” exercise (Figure 1–1). 4. Patient Registries and Registries should be preferred in those Policy Purposes circumstances where they provide sufficiently high-quality information for decisionmaking at a In addition to the growth of patient registries for sufficiently low cost (relative to other “acceptable” scientific and clinical purposes, registries are designs). receiving increased attention for their potential One set of policy determinations that may be 56 role in policymaking or decisionmaking. As informed by a patient registry centers on the area stated earlier, registries may offer a view of of payment for items or services. For example, the real-world health care that is typically inaccessible Centers for Medicare & Medicaid Services (CMS)

21 Section I. Creating Registries

issued Guidance on National Coverage when current data do not address relevant Determinations With Data Collection as a outcomes for beneficiaries, off-label or Condition of Coverage in 2006. That original unanticipated uses, important patient subgroups, or guidance document (which has undergone operator experience or other qualifications. subsequent revisions, including an additional draft Registry-based studies may also be important guidance published in 201257) provided several when an existing treatment is being reconsidered. examples of how data collected in a registry might (An RCT may not be possible under such be used in the context of coverage determinations. circumstances.) Registry-based studies are also As described in the Guidance: being used increasingly in fulfillment of [T]he purpose of CED [Coverage with postmarketing commitments and requirements. Evidence Development] is to generate data on In many countries, policy determinations on the utilization and impact of the item or payment rely on cost-effectiveness and cost-utility service evaluated in the NCD [National data and therefore can be informed by registries as Coverage Determination], so that Medicare well as clinical trials.58 These data are used and can (a) document the appropriateness of use of reviewed in a variety of ways. In some countries, that item or service in Medicare beneficiaries there may be a threshold above which a payer is under current coverage; (b) consider future willing to pay for an improvement in patient changes in coverage for the item or service; outcomes.59 In these scenarios—particularly for (c) generate clinical information that will rare diseases, when it can be difficult to gather improve the evidence base on which providers clinical effectiveness data together with quality-of- base their recommendations to Medicare life data in a utility format—the establishment of beneficiaries regarding the item or service.56 disease-specific data registries has been The Guidance provided insight into when registry recommended to facilitate the process of technology assessment and improving patient data may be useful to policymakers. These 60 purposes range from demonstrating that a care. In fact, the use of new or existing registries particular item or service was provided to assess health technology or risk-sharing appropriately to patients meeting specific arrangements is growing in such countries as the characteristics, to collecting new information that United Kingdom, France, Germany, and Australia, and in conditions ranging from bariatric surgery to is not available from existing clinical trials. CED 61-66 based on registries may be especially relevant stroke care.

22 Chapter 1. Patient Registries

Figure 1–1. Deciding when to develop a registry: The “value of information” exercise

Item/service presented for consideration

Clinical/policy question(s) formulated

“Adequate data” defined

Current evidence reviewed

Data deemed adequate for decisionmaking List of alternative designs constructed

Decision made

“Value of information” exercise performed

Evidence development strategy implemented

23 Section I. Creating Registries

Consider the clinical question of carotid complexities (Chapter 11), ranging from available endarterectomy surgery for patients with a high technologies to languages; and specific degree of stenosis of the carotid artery. requirements for mandated pregnancy registries Randomized trials, using highly selected patients (Chapter 21). and surgeons, indicate a benefit of surgery over medical management in the prevention of stroke. 6. Summary However, that benefit may be exquisitely sensitive to the surgical complication rates; a relatively A patient registry is an organized system that uses small increase in the rate of surgical complications observational study methods to collect uniform is enough to make medical management the data (clinical and other) to evaluate specified preferred strategy instead. In addition, the studies outcomes for a population defined by a particular of surgical performance in a variety of hospitals disease, condition, or exposure and that serves may suggest substantial variation in surgical predetermined scientific, clinical, or policy mortality and morbidity for this procedure. In such purpose(s). Studies derived from well-designed a case, a registry to evaluate treatment outcomes, and well-performed patient registries can provide a adjusted by hospital and surgeon, might be real-world view of clinical practice, patient considered to support a policy decision as to when outcomes, safety, and clinical, comparative, and the procedure should be reimbursed (e.g., only cost-effectiveness, and can serve a number of when performed in medical centers resembling evidence development and decisionmaking those in the various randomized trials, or only by purposes. In the chapters that follow, this user’s surgeons or facilities with an acceptably low rate guide presents practical design and operational of complications).67 issues, evaluation principles, and good registry practices. 5. Global Registries References for Chapter 1 As many stakeholders have international interests in diseases, conditions, and health care products 1. Webster’s English Dictionary. http://www.m-w.com. and services, it is not surprising that interest in Accessed August 12, 2012. global patient registries is growing. While some of 2. Brooke EM. The current and future use of registers the specific legal and regulatory discussions in this in health information systems. Geneva: World user’s guide are intended for and limited to the Health Organization; 1974. Publication No. 8. United States, most of the concepts and specifics 3. National Committee on Vital and Health Statistics. are more broadly applicable to similar activities Frequently Asked Questions About Medical and worldwide. Chapter 7 (ethics, data ownership, and Public Health Registries. http://ncvhs.hhs. privacy), Chapter 9 (protection of registry data), gov/9701138b.htm. Accessed August 14, 2012. and Chapter 12 (adverse event detection, 4. Dokholyan RS, Muhlbaier LH, Falletta JM, et al. processing, and reporting) are perhaps the most Regulatory and ethical considerations for linking limited in their applicability outside the United clinical and administrative databases. Am Heart J. States. There may be additional considerations in 2009 Jun;157(6):971-82. PMID: 19464406. data element selection and patient-reported outcome measure selection (Chapters 4 and 5) 5. Hammill BG, Hernandez AF, Peterson ED, et al. Linking inpatient clinical registry data to stemming from differences ranging from medical Medicare claims data using indirect identifiers. training to use of local remedies; the types of data Am Heart J. 2009 Jun;157(6):995-1000. PMID: sources that are available outside the United States 19464409. Pubmed Central PMCID: 2732025. (Chapter 6); the requirements for informed consent (Chapter 8); the issues surrounding clinician and patient recruitment and retention in different health systems and cultures (Chapter 10); specific data collection and management options and

24 Chapter 1. Patient Registries

6. Developing a Registry of Patient Registries 15. Dreyer NA, Garner S. Registries for robust (RoPR). Project Abstract. Agency for Healthcare evidence. JAMA. 2009 Aug 19;302(7):790-1. Research and Quality. http://www. PMID: 19690313. effectivehealthcare.ahrq.gov/index.cfm/search-for- 16. Concato J, Shah N, Horwitz RI. Randomized, guides-reviews-and-reports/?productid=690&page controlled trials, observational studies, and the action=displayproduct. Accessed August 14, 2012. hierarchy of research designs. N Engl J Med. 2000 7. LaBresh KA, Gliklich R, Liljestrand J, et al. Using Jun 22;342(25):1887-92. PMID: 10861325. “get with the guidelines” to improve PMCID: 1557642. cardiovascular secondary prevention. Jt Comm 17. Guyatt GH, Sackett DL, Sinclair JC, et al. Users’ J Qual Saf. 2003 Oct;29(10):539-50. guides to the medical literature. IX. A method for PMID: 14567263. grading health care recommendations. Evidence- 8. Kennedy L, Craig AM. Global registries for Based Medicine Working Group. JAMA. 1995 measuring pharmacoeconomic and quality-of-life Dec 13;274(22):1800-4. PMID: 7500513. outcomes: focus on design and data collection, 18. Agency for Healthcare Research and Quality. analysis and interpretation. Pharmacoeconomics. Methods Reference Guide for Effectiveness and 2004;22(9):551-68. PMID: 15209525. Comparative Effectiveness Reviews. Version 1.0 9. Charrow J, Esplin JA, Gribble TJ, et al. Gaucher [Draft posted Oct. 2007]. Rockville, MD. http:// disease: recommendations on diagnosis, effectivehealthcare.ahrq.gov/repFiles/2007_10Dra evaluation, and monitoring. Arch Intern Med. ftMethodsGuide.pdf. Accessed August 14, 2012. 1998 Sep 14;158(16):1754-60. PMID: 9738604. 19. Vandenbroucke JP. Observational research, 10. Bushby K, Lynn S, Straub T, et al. Collaborating randomised trials, and two views of medical to bring new therapies to the patient--the TREAT- science. PLoS Med. 2008 Mar 11;5(3):e67. NMD model. Acta Myol. 2009 Jul;28(1):12-5. PMID: 18336067. PMCID: 2265762. PMID: 19772190. PMCID: 2859629. 20. Atkins D, Eccles M, Flottorp S, et al. Systems for 11. Dhruva SS, Phurrough SE, Salive ME, et al. grading the quality of evidence and the strength of CMS’s landmark decision on CT colonography-- recommendations I: critical appraisal of existing examining the relevant data. N Engl J Med. 2009 approaches The GRADE Working Group. BMC Jun 25;360(26):2699-701. PMID: 19474419. Health Serv Res. 2004 Dec 22;4(1):38. PMID: 15615589. PMCID: 545647. 12. Postmarketing studies and clinical trials – implementation of Section 505(o) of the Federal 21. Rawlins M. De Testimonio: on the evidence for Food, Drug and Cosmetic Act. FDA Guidance for decisions about the use of therapeutic Industry. Draft guidance. July 2009. http://www. interventions. Clin Med. 2008 Dec;8(6):579-88. fda.gov/downloads/Drugs/GuidanceCompliance PMID: 19149278. RegulatoryInformation/Guidances/UCM172001. 22. Atkins D, Best D, Briss PA, et al. Grading quality pdf. Accessed August 14, 2012. of evidence and strength of recommendations. 13. U.S. Food and Drug Administration. Guidance for BMJ. 2004 Jun 19;328(7454):1490. Industry. Good pharmacovigilance practices and PMID: 15205295. PMCID: 428525. pharmacoepidemiologic assessment. March 2005. 23. Grading the Strength of a Body of Evidence When http://www.fda.gov/downloads/ Assessing Health Care Interventions – AHRQ and RegulatoryInformation/Guidances/UCM126834. the Effective Health Care Program: An Update. pdf. Accessed August 14, 2012. Draft released for public comment. http://www. 14. U.S. Food and Drug Administration. FDA Post- effectivehealthcare.ahrq.gov/ehc/ Approval Studies. http://www.accessdata.fda.gov/ products/457/1163/ scripts/cdrh/cfdocs/cfPMA/pma_pas.cfm?utm_ GradingTheStrengthofEvidence_ campaign=Google2&utm_ DraftMethodsChapter_20120625.pdf. Accessed source=fdaSearch&utm_medium=website&utm_ October 12, 2012. term=post-approval%20studies&utm_content=1. Accessed August 14, 2012.

25 Section I. Creating Registries

24. Viswanathan M, Ansari MT, Berkman ND, Chang 33. Wennberg DE, Lucas FL, Birkmeyer JD, et al. S, Hartling L, McPheeters LM, Santaguida PL, Variation in carotid endarterectomy mortality in Shamliyan T, Singh K, Tsertsvadze A, Treadwell the Medicare population: trial hospitals, volume, JR. Assessing the Risk of Bias of Individual and patient characteristics. JAMA. 1998 Apr Studies in Systematic Reviews of Health Care 22-29;279(16):1278-81. PMID: 9565008. Interventions. Agency for Healthcare Research 34. MacIntyre K, Capewell S, Stewart S, et al. and Quality Methods Guide for Comparative Evidence of improving prognosis in heart failure: Effectiveness Reviews. AHRQ Publication No. trends in case fatality in 66 547 patients 12-EHC047-EF. Rockville, MD: Agency for hospitalized between 1986 and 1995. Circulation. Healthcare Research and Quality; March 2012. 2000 Sep 5;102(10):1126-31. PMID: 10973841. http://www.effectivehealthcare.ahrq.gov/ehc/ products/322/998/MethodsGuideforCERs_ 35. Konstam MA. Progress in heart failure Viswanathan_IndividualStudies.pdf. Accessed Management? Lessons from the real world. October 12, 2012. Circulation. 2000 Sep 5;102(10):1076-8. PMID: 10973832. 25. Clancy CM, Eisenberg JM. Outcomes research: measuring the end results of health care. Science. 36. Hutchins LF, Unger JM, Crowley JJ, et al. 1998 Oct 9;282(5387):245-6. PMID: 9841388. Underrepresentation of patients 65 years of age or older in cancer-treatment trials. N Engl J Med. 26. Lipscomb J, Snyder CF. The Outcomes of Cancer 1999 Dec 30;341(27):2061-7. PMID: 10615079. Outcomes Research: focusing on the National Cancer Institute’s quality-of-care initiative. Med 37. Opthalmic devices and clinical . In: Care. 2002 Jun;40(6 Suppl):III3-10. Brown SL, Bright RA, Tavris DR, eds. Medical PMID: 12064755. device epidemiology and surveillance: John Wiley & Sons, Ltd.; 2007:chapter 28. 27. National Cancer Institute. The Emerging Field of Outcomes Research. http://outcomes.cancer.gov/ 38. Lipscomb J, Yabroff KR, Brown ML, et al. Health aboutresearch/index.html. Accessed August 14, care costing: data, methods, current applications. 2012. Med Care. 2009 Jul;47(7 Suppl 1):S1-6. PMID: 19536004. 28. Lipscomb J, Donaldson MS, Hiatt RA. Cancer outcomes research and the arenas of application. 39. Eichler HG, Kong SX, Gerth WC, et al. Use of J Natl Cancer Inst Monogr. 2004 (33):1-7. cost-effectiveness analysis in health-care resource PMID: 15504917. allocation decision-making: how are cost- effectiveness thresholds expected to emerge? 29. Institute of Medicine, Committee on Quality of Value Health. 2004 Sep-Oct;7(5):518-28. Heallth Care in America. Crossing the Quality PMID: 15367247. Chasm: A New Health System for the 21st Century. Washington DC: National Academy 40. Palmer AJ. Health economics--what the Press, Institute of Medicine; 2001. nephrologist should know. Nephrol Dial Transplant. 2005 Jun;20(6):1038-41. 30. Schwamm LH, LaBresh KA, Pan WQ, et al. Get PMID: 15840678. With The Guidelines – Stroke produces sustainable improvements in hospital-based acute 41. Good ReseArch for Comparative Effectiveness. stroke care. Stroke. 2006;37(2):624. http://www.graceprinciples.org. Accessed August 14, 2012. 31. Schwamm LH, LaBresh KA, Pan WQ, et al. Get With The Guidelines – Stroke improves the rate of 42. Institute of Medicine. Learning what works best: “defect-free” acute stroke care. Stroke. the Nation’s need for evidence on comparative 2006;37(2):659. effectiveness in health care. Washington DC: National Academy Press; 2007. p. 1-80. 32. Barranger JA, O’Rourke E. Lessons learned from the development of enzyme therapy for Gaucher disease. J Inherit Metab Dis. 2001;24 Suppl 2:89-96; discussion 87-8. PMID: 11758684.

26 Chapter 1. Patient Registries

43. Congressional Budget Office. Research on the 51. Greene FL, Gilkerson S, Tedder P, et al. The role Comparative Effectiveness of Medical Treatments: of the hospital registry in achieving outcome Issues and Options for an Expanded Federal Role. benchmarks in cancer care. J Surg Oncol. 2009 2007. http://www.cbo.gov/sites/default/files/ Jun 15;99(8):497-9. PMID: 19466739. cbofiles/ftpdocs/88xx/doc8891/12-18- 52. Schweikert B, Hunger M, Meisinger C, et al. comparativeeffectiveness.pdf. Accessed August Quality of life several years after myocardial 14, 2012. infarction: comparing the MONICA/KORA 44. Federal Coordinating Council for Comparative registry to the general population. Eur Heart J. Effectiveness Research. Report to the President 2009 Feb;30(4):436-43. PMID: 19019995. and the Congress. June 30, 2009. U.S. 53. Lane K, Kempf A, Magno C, et al. Regional Department of Health and Human Services. http:// differences in the use of sentinel lymph node www.hhs.gov/recovery/programs/cer/cerannualrpt. biopsy for melanoma: a potential quality measure. pdf. Accessed August 14, 2012. Am Surg. 2008 Oct;74(10):981-4. 45. Patient-Centered Outcomes Research Institute. PMID: 18942627. “Patient-Centered Outcomes Research.”. http:// 54. Griffiths I, Silman A, Symmons D, et al. BSR www.pcori.org/what-we-do/pcor/. Accessed Biologics Registry. Rheumatology (Oxford). 2004 August 28, 2012. Dec;43(12):1463-4. PMID: 15328420. 46. Public comment draft report of the Patient- 55. U.S. Food and Drug Administration. Guidance for Centered Outcomes Research Institute (PCORI) Industry: Establishing Pregnancy Exposure Methodology Committee presented on July 23, Registries. http://www.fda.gov/downloads/Drugs/ 2012, and revised thereafter. http://pcori.org/ GuidanceComplianceRegulatoryInformation/ assets/MethodologyReport-Comment.pdf. Guidances/ucm071639.pdf. Accessed August 15, Accessed August 28, 2012. 2012. 47. Agency for Healthcare Research and Quality. 56. Centers for Medicare & Medicaid Services. Bridging Knowledge Gaps in the Comparative Guidance for the Public, Industry, and CMS Staff: Effectiveness of ACE Inhibitors and ARBs. Draft National Coverage Determinations with Data abstract. 2008. . http://www.effectivehealthcare. Collection as a Condition of Coverage: Coverage ahrq.gov/index.cfm/search-for-guides-reviews- with Evidence Development. July 12, 2006. and-reports/?productid=296&pageaction=displayp roduct. Accessed August 14, 2012. 57. Centers for Medicare & Medicaid Services. Draft Guidance for the Public, Industry, and CMS Staff: 48. Hodgson DC, Fuchs CS, Ayanian JZ. Impact of Coverage with Evidence Development in the patient and provider characteristics on the context of coverage decisions. http://www.cms. treatment and outcomes of colorectal cancer. gov/medicare-coverage-database/details/medicare- J Natl Cancer Inst. 2001 Apr 4;93(7):501-15. coverage-document-details.aspx?MCDId=23. PMID: 11287444. Accessed February 20, 2013. 49. Reeves MJ, Fonarow GC, Zhao X, et al. Quality of 58. Connock M, Burls A, Frew E, et al. The clinical care in women with ischemic stroke in the GWTG effectiveness and cost-effectiveness of enzyme program. Stroke. 2009 Apr;40(4):1127-33. replacement therapy for Gaucher’s disease: a PMID: 19211482. systematic review. Health Technol Assess. 2006 50. Fonarow GC, Abraham WT, Albert NM, et al. Jul;10(24):iii-iv, ix-136. PMID: 16796930. Influence of a performance-improvement initiative 59. Devlin N, Parkin D. Does NICE have a cost- on quality of care for patients hospitalized with effectiveness threshold and what other factors heart failure: results of the Organized Program to influence its decisions? A binary choice analysis. Initiate Lifesaving Treatment in Hospitalized Health Econ. 2004 May;13(5):437-52. Patients With Heart Failure (OPTIMIZE-HF). PMID: 15127424. Arch Intern Med. 2007 Jul 23;167(14):1493-502. PMID: 17646603.

27 Section I. Creating Registries

60. Connock M, Juarez-Garcia A, Frew E, et al. A 65. Haute Autorité de Santé. Heart surgery with or systematic review of the clinical effectiveness and without extracorporeal circulation: role of the cost-effectiveness of enzyme replacement second surgeon. 2008. http://www.has-sante.fr/ therapies for Fabry’s disease and portail/upload/docs/application/pdf/abstract_ mucopolysaccharidosis type 1. Health Technol heart_surgery.pdf. Accessed August 14, 2012. Assess. 2006 Jun;10(20):iii-iv, ix-113. 66. Haute Autorité de Santé. Interview du docteur PMID: 16729919. Jean-François Thébaut, cardiologue libéral et 61. Chalkidou K, Tunis S, Lopert R, et al. Président du Conseil National Professionnel de Comparative effectiveness research and evidence- Cardiologie. [French.]. http://www.has-sante.fr/ based health policy: experience from four portail/jcms/c_777504/interview-du-docteur-jean- countries. Milbank Q. 2009 Jun;87(2):339-67. francois-thebautcardiologue-liberal-et-president- PMID: 19523121. Pubmed Central PMCID: du-conseil-nationalprofessionnel-de- 2881450. cardiologie?portal=c_63456. Accessed August 14, 2012. 62. National Institute for Health and Clinical Excellence (UK). Final appraisal determination: 67. Matchar DB, Oddone EZ, McCrory DC, et al. alteplase for the treatment of acute ischaemic Influence of projected complication rates on stroke. 2007. http://www.nice.org.uk/nicemedia/ estimated appropriate use rates for carotid live/11617/33971/33971.pdf. Accessed August 14, endarterectomy. Appropriateness Project 2012. Investigators. Academic Medical Center Consortium. Health Serv Res. 1997 63. Graves SE, Davidson D, Ingerson L, et al. The Aug;32(3):325-42. PMID: 9240284. Australian Orthopaedic Association National Joint PMCID: 1070194. Replacement Registry. Med J Aust. 2004 Mar 1; 180(5 Suppl):S31-4. PMID: 14984361. 64. Owen A, Spinks J, Meehan A, et al. A new model to evaluate the long-term cost effectiveness of orphan and highly specialised drugs following listing on the Australian Pharmaceutical Benefits Scheme: the Bosentan Patient Registry. J Med Econ. 2008;11(2):235-43. PMID: 19450082.

28 Chapter 2. Planning a Registry

1. Introduction may stop collecting new data. Registries may undergo a transition or cease collecting new data There is tremendous variability in size, scope, and for many reasons. These considerations are fully resource requirements for registries. Registries discussed in Chapter 14. may be large or small in terms of numbers of The Guidelines for Good Pharmacoepidemiology patients or participating sites. They may target rare Practice from the International Society of or common conditions and exposures. They may Pharmacoepidemiology is a useful resource for require the collection of limited or extensive registry planners.1 The Updated Guidelines for amounts of data, operate for short or long periods Evaluating Public Health Surveillance Systems of time, and be funded generously or operate with may also be useful, especially the appendixes, limited financial support. In addition, the scope which provide various checklists.2 A Guide to the and focus of a registry may be adapted over time to Project Management Body of Knowledge reach broader or different populations, assimilate (PMBOK® Guide) and the GRACE principles for additional data, focus on or expand to different comparative effectiveness (www.graceprinciples. geographical regions, or address new research org) may also be useful resources to registry questions. While this degree of flexibility confers planners.3, 4 enormous potential, registries require good planning in order to be successful. 2. Steps in Planning a Registry When planning a registry, it is desirable to follow these initial steps: (1) articulate the purpose of the 2.1 Articulate the Registry’s Purpose registry; (2) determine if a registry is an appropriate means to achieve the purpose; One of the first steps in planning a registry is (3) identify key stakeholders; and (4) assess the articulating its purpose. Having a clearly defined feasibility of a registry. goal and/or purpose and supporting rationale makes it easier to evaluate whether a registry is the Once a decision is made to proceed, the next right approach for capturing the information of considerations in planning are to (5) build a interest.5, 6 In addition, a clearly defined purpose registry team; (6) establish a governance and helps clarify the need for certain data. Conversely, oversight plan; (7) define the scope and rigor having a clear sense of how the data may be used needed; (8) define the data set, patient outcomes, will help refine the stated purpose. Attempts to be and target population; (9) develop a study plan or all inclusive may add cost but not value, resulting protocol; and (10) develop a project plan. Of in overly burdensome data collection that can course, the planning for a registry is often not a reduce quality and erode compliance. linear process. Many of the steps described in this chapter occur in parallel. A registry may have a singular purpose or several purposes.7 In either case, the overall purpose Registry planners should also recognize the should be translated into specific objectives or importance of periodic critical evaluations of the questions to be addressed through the registry. registry by key stakeholders to ensure that the This process needs to take into account the objectives are being met. This is particularly interests of those collaborating in the registry and important for patient registries that collect data the key audiences to be reached.8 Clear objectives over many years. When registry objectives are no are essential to define the structure and process of longer being met or when clinical or other changes data collection and to ensure that the registry affect the registry (e.g., changes in treatment effectively addresses the important questions practices, the introduction of a new therapy), the through the appropriate outcomes analyses. registry may need to be adapted, or the registry Specific objectives also help the registry to avoid

29 Section I. Creating Registries

collecting large amounts of data of limited value. 2.2 Determine if a Registry Is an The time and resources needed to collect and Appropriate Means To Achieve the process data from a registry can be substantial.9 Purpose The identification of a core data set is essential. Two key questions to consider are whether a The benefits of any data element included in the registry (or other study) is needed to address the registry must outweigh the costs of including it. purpose and, if the answer is yes, whether Registry planners should establish specific prospective data collection through a registry is an objectives by considering what key questions the appropriate means of accomplishing the scientific registry needs to answer. Critical consideration objectives. Every registry developer should should be given to defining the key questions in consider the following questions early in the order to evaluate how best to proceed, as these planning process: questions will help to establish the type of registry • Do these data already exist? (e.g., whether single focus or comparative), the data elements to be captured, and the types of • If so, are they of sufficient quality to answer the analysis to be undertaken. Examples of key or research question? driving questions are listed below: • Are they accessible, or does an entirely new • What is the natural course of a disease, and data collection effort need to be initiated? how does geographic location affect the For example, could the necessary data be extracted course? from electronic medical records or administrative • Does a treatment lead to long-term benefits or health insurance claims data? In such cases, harm, including delayed complications? registries might avoid re-collecting data that have already been collected elsewhere and are • How is disease progression affected by accessible. Thought should be given to adapting available therapies? the registry (based on extant data) and/or linking • What are significant predictors of poor to other relevant data sources (including outcomes? “piggybacking” onto other registries). The • What is the safety profile of a specific therapy? Registry of Patient Registries (RoPR), developed by the Agency for Healthcare Research and • Is a specific product or therapy teratogenic? Quality, is a resource for finding patient • How do clinical practices vary, and what are the registries.10 When the required data have not been best predictors of treatment practices? sufficiently collected or are not accessible for the desired purpose, it is appropriate to consider • Are there disparities in the delivery and/or creating a new registry. outcomes of care? The next step is to consider whether the purpose • What characteristics or practices enhance would be well served by a registry. When making compliance and adherence? this decision, it is important to fully define the • Do quality improvement programs affect specific research question(s) of interest and to patient outcomes, and, if so, how? consider the state of current knowledge and gaps in evidence. Other factors that may influence this • What process and outcomes metrics should be decision include the breadth of the target incorporated to track quality of patient care? population of interest, the complexity of the • Should a particular procedure or product be a current treatment patterns, the length of an covered benefit in a particular population? observational period needed to achieve the • Was an intervention program or risk- objective, the scope and variety of treatments used, management activity successful? the approximate amount of funding available to address these objectives, and the urgency of • What are the resources used/economic decisions that will be made based on the resulting parameters of actual use in typical patients?

30 Chapter 2. Planning a Registry

evidence. Registries may be the most appropriate such as registries.13-15 (See Chapters 3, 13, and choice for some research questions. For example, 18.) These techniques may allow registries to be registries are particularly useful in situations where used to support investigations of comparative a comprehensive, flexible research design is safety and effectiveness. Following good registry needed,11, 12 or when the purpose is to discover practices, as described in this user’s guide, can how a product works in a wide variety of strengthen scientific rigor. (See Chapter 25.) subgroups. (See Chapter 3, Section 2 for a discussion of research questions appropriate for 2.3 Identify Key Stakeholders registries.) As a means of identifying potential stakeholders, it Other research questions, such as ones that might is important to consider to whom the research be used to petition a regulatory agency for a new questions matter. It is useful to identify these indication, will require different approaches, such stakeholders at an early stage of the registry as traditional randomized controlled trials. In some planning process, as they may have important cases, a hybrid approach, such as a registry that input into the type and scope of data to be incorporates data collected retrospectively as well collected, they may ultimately be users of the data, as prospectively, will be required. A research and/or they may have a key role in disseminating strategy, as opposed to a single study, may be the results of the registry. necessary to address some research questions. For One or more parties could be considered example, some research questions may require an stakeholders of the registry. These parties could be interventional approach to address concerns about as specific as a regulatory agency that will be efficacy combined with an observational approach monitoring postmarketing studies or as broad as to examine long-term outcomes and quality of life the general population, or simply those patients in a broad patient population. When making a with the conditions of interest. Often, a decision about study design, it is important to stakeholder’s input directly influences whether select the approach or combination of approaches development of a registry can proceed, and it can best able to answer the specific research questions, have a strong influence on how a registry is from both scientific and practical standpoints. A conducted. A regulatory agency looking for careful evaluation of the possibilities for data management of a therapeutic product with a collection and registry design, the degree of known toxicity profile may require a different certainty required, and the timeframe in which this registry design than a manufacturer with general certainty is expected can help in selecting an questions about how a product is being used. appropriate study design. Typically, there are primary and secondary Historically, there has been a lack of consensus stakeholders for any registry. A primary standards for conducting and reporting methods stakeholder is usually responsible for creating and and results for registries. Therefore, registries have funding the registry. The party that requires the been more variable in implementation and more data, such as a regulatory authority, may also be difficult to assess for quality than randomized considered a primary stakeholder. A secondary controlled trials. In recent years, advances in stakeholder is a party that would benefit from epidemiological and biostatistical methods have knowledge of the data or that would be impacted broadened the scope of questions that can be by the results but is not critical to establishing the addressed through observational studies such as registry. Treating clinicians and their patients could registries. Stratification, propensity score be considered secondary stakeholders. A partial matching, and risk adjustment are increasingly list of possible stakeholders, both primary and useful approaches for addressing confounding secondary, follows: issues and for creating comparably homogeneous subgroups for analysis within registry data sets, • Public health or regulatory authorities and advances in bias analysis are being used to • Product manufacturers help interpret results from observational studies • Health care service providers

31 Section I. Creating Registries

• Payer or commissioning authorities There are many potential funding sources for • Patients and/or advocacy groups registries. Funding sources are likely to want to share in planning and to provide input for the • Treating clinician groups many choices that need to be made in the • Academic institutions or consortia implementation plans. Funding sources may negotiate to receive access to deidentified data as a • Professional societies condition for their participation. Funding models Although interactions with potential stakeholders for registries may vary significantly, and there is will vary, the registry will be best supported by no preferred approach. Rather, the funding model defined interactions and communications with for a registry should be dictated by the needs of these parties. Defining these interactions during the registry. Potential sources of funding include: the planning stage will ensure that adequate dialog • Foundations: Nonprofit disease foundations occurs and appropriate input is received to support may be interested in a registry to track the the overall value of the registry. Interactions natural history of the disease of interest as well throughout the entire duration of the registry can as the impact of therapeutic interventions. also assure stakeholders that the registry is aligned Registries may be used to track practice with the purposes and goals that were set out patterns and outcomes for quality improvement during the planning stages and that the registry initiatives. Ongoing registries can sometimes complies with all required guidances, rules, and/or serve the additional purpose of assisting in regulations. recruitment for clinical trials.17 2.4 Assess Feasibility • Government: Federal agencies, such as the National Institutes of Health (NIH), Centers for A key element in determining the feasibility of Disease Control and Prevention (CDC), developing a new registry relates to funding. Centers for Medicare & Medicaid Services Registries that meet the attributes described in this (CMS), Agency for Healthcare Research and user’s guide will most likely require significant Quality (AHRQ), U.S. Food and Drug funding. The degree of expense incurred will be Administration (FDA), and State agencies, may determined by the scope of the registry, the rigor be interested in a registry to determine long- of data collection, and any audits that may be term outcomes of agents, devices, groups of required. The larger the number of sites, the drugs, or procedures. While the pharmaceutical number of patients, and the scope of data collected, industry or device manufacturers collect most and the greater the need for representation of a long-term data on drug and device safety, many wide variety of patient characteristics, the greater research questions arise that could potentially the expense will be. In addition, the method of be suitable for government funding, ranging data collection will contribute to expense. from clinical or comparative effectiveness to Historically, electronic data collection has been natural history of disease to the performance of more expensive to implement, but generally less health care providers based on accepted expensive to maintain, than forms that are faxed measures of quality of care. To determine if an and scanned or mailed;16 however, the cost agency might be interested in funding a difference for startup has been lessening. Funding registry, look for Requests for Proposals (RFPs) will be affected by whether other relevant data on its Web site. An RFP posting or direct sources and/or infrastructures exist that capture communication with the appropriate agency some of the information of interest; whether the staff may provide a great deal of specific registry adapts to new issues over time; and information as to how a submission will be whether multiple funding sources participate. judged and what criteria would be needed in Funding needs should also be examined in terms order for a proposal to be favorably ranked. of the projected life of the registry and/or its Even if an RFP is not posted, contacting the long-term sustainability.

32 Chapter 2. Planning a Registry

appropriate agency staff may uncover potential are receiving mechanical circulatory support interest in a registry to fill an unmet need. device therapy to treat advanced heart failure.19 • Health plan providers: Under certain While multiple sponsorship can decrease the circumstances, health plan providers may be costs for each funding source, their varied interested in funding a registry, since practical interests and needs almost always increase the clinical research is increasingly viewed as a complexity and overall cost of the registry. useful tool for providing evidence for health A public-private partnership is a service or coverage and health care decisions.18 business venture that is funded and operated • Patient groups: Patients may be able to through a partnership (contractual agreement) between a public agency (Federal, State, or local) contribute funding to focus on rare diseases or 20 patient subgroups of interest for more common and a private-sector entity or entities. While conditions. They may also contribute value some true public-private partnerships for registries in-kind. currently exist (e.g., State-level immunization registries, bioterrorism surveillance efforts),21-23 • Private funding: Private philanthropic there is great potential for growth in this approach. individuals or charitable foundations and trusts Both government and private sources have shown may have an interest in furthering research to increasing interest in registries for improved safety better understand the effects of a particular monitoring, for comparative effectiveness goals, intervention or sets of interventions on a and for streamlining the costs of the drug disease process. development process.24-29 Several legislative • Product manufacturers: Product manufacturers actions have stated or suggested the role of public- may be interested in studying the natural private partnerships for activities such as registry history of the disease for which they have (or development.30 There are many good reasons for are developing) a product; demonstrating the multiple stakeholders, including government effectiveness and/or safety of existing products agencies, providers, and industry, to work together in real-world use through Risk Evaluation and for certain purposes. Thus, it is anticipated that Mitigation Strategy (REMS) programs as part shared funding mechanisms are likely to become of postmarketing commitments or more common. Chapter 24 provides more detail on requirements, or through studies; or assisting the use of public-private partnerships to support providers in evaluating or improving quality of registries. care. 2.5 Build a Registry Team • Professional societies: Health care professional associations are increasingly participating in Several different kinds of knowledge, expertise, developing or partnering with registries for and skills are needed to plan and implement a scientific and quality measurement or registry. In a small registry run by a single improvement purposes. individual, consultants may be able to provide the critical levels of expertise needed to plan all • Professional society/pharmaceutical industry components of the registry. In a large registry, a “hybrids”: Situations may exist in which a variety of individuals may work together as a team product manufacturer funds a registry designed to contribute the necessary expertise. Depending and implemented by a professional society to on the size, scope, and purpose of the registry, few, gain insight into a set of research questions. some, or all of the individuals representing the • Multiple sponsors: Registries may meet the components of expertise described below may be goals of multiple stakeholders, and such included at the time of the planning process. stakeholders may have an interest in sharing the Whatever number of individuals is eventually funding. Registries for isotretinoin and assembled, it is important to build a group that can antiretrovirals in pregnancy are examples, as is work together as a collegial team to accomplish INTERMACS™, a registry for patients who the goals of the registry. Additionally, the team

33 Section I. Creating Registries

participants must understand the data sources. By • Data collection and database management: understanding the goals and data sources, the The decision to include various data elements registry team will enable the data to be used in the can be made in consultation with experts in this most appropriate context for the most appropriate field to place “critical fields” in a prominent interpretation. The different kinds of expertise and and logical position on the data form for both experience that are useful include the following: paper-based and electronic data collection • Project management: Project management will tools. (A final determination of what is usable be needed to coordinate the components of the and workable for data collection tools should registry; to manage timelines, milestones, be approved by all members of the team.) These deliverables, and budgets; and to ensure experts may also need to write specific communication with sites, stakeholders, programs so that the data received from the oversight committees, and funding sources. registry are grouped, stored, and identified. Ongoing oversight of the entire process will They may generate reports for individuals who require a team approach. (See Section 2.6, track registry participation, and they may “Establish a Governance and Oversight Plan.”) provide data downloads periodically to registry analysts. This team will also be responsible for • Subject matter: A registry must be designed so implementing and maintaining firewalls to that it contains the appropriate data to meet its protect the data according to accepted levels of goals as well as the needs of its stakeholders. security for similar collections of sensitive data. For example, experts in the treatment of the clinical disease to be studied who are also • Legal issues/patient privacy: It is critical that familiar with the potential toxicities of the either information that identifies individual treatment(s) to be studied are critical to the patients be excluded or applicable legal success of the registry. Clinical experts must be requirements for the inclusion of patient able to apply all of the latest published clinical, identifiable information be met (e.g., obtaining toxicity, and outcome data to components of informed consent or Health Insurance the registry and determine which elements are Portability and Accountability Act [HIPAA] necessary, desirable, or superfluous. Depending authorization, where required). The on the outcomes and registry purpose, it is complexities of this topic are dealt with in often useful to have patient representatives or detail in Chapters 7, 8, and 9. Legal and privacy advocates. expertise is needed to protect the patients and the owners of the database by ensuring that the • Registry science: Epidemiology and registry complies with all Federal and State expertise specific to the subtleties laws applicable to patient information. of patient registries and observational research is very important in the design, • Quality assurance: As discussed in Chapter 11, implementation, and analysis of registry data. Section 3, quality assurance of procedures and Epidemiologists can provide the study design data is another important component of registry and can work in collaboration with success. Expertise in quality assurance will biostatisticians to develop a mutual help in planning a good registry. The goals for understanding of the research objectives and quality assurance should be established for data needed. Health outcomes researchers and each registry, and the efforts made and the economics researchers can also lend valuable results achieved should be described. expertise to the registry team. These scientists 2.6 Establish a Governance and should work with the subject matter experts to Oversight Plan ensure that appropriate analytic methods are being used to address the clinical issues Governance refers to guidance and high-level relevant to achieving the goals of the registry. decisionmaking, including purpose, funding, execution, and dissemination of information. A goal of proper governance and oversight should be

34 Chapter 2. Planning a Registry

transparency to stakeholders in operations, decisions that determine the direction of the decisionmaking, and reporting of results. registry. These decisions are made with The composition and relative mix of stakeholders appropriate input from legal, scientific, and and experts relate largely to the purpose of the administrative experts. Depending on their registry. For example, if the purpose of the registry capabilities and the size and resources of the is to determine a comparative effectiveness or registry, the group serving the steering function reimbursement policy, those impacted by the may also assume some of the functions policy should not solely govern the registry. Broad described below. stakeholder involvement in governance boards is • Scientific: This function may include experts in most desirable when there are many stakeholders. areas ranging from database content, to general Depending on the size of the registry, governance clinical research, to epidemiology and may be assumed by various oversight committees biostatistics. This function may determine the made up of interested individuals who are part of overall direction of database inquiries and the design team (internal governance) or who recommend specific analyses to the executive remain external to the day-to-day operations of the or steering group. It is strongly desirable that registry (external governance). Differences in the the reports that emerge from a registry be nature of the study questions, the overall resources scientifically based analyses that are being consumed by the registry, the soundness of independent and transparent.31 To enhance the underlying data sources, and many other credibility and in the interest of full disclosure, factors will influence the degree of involvement the role of all stakeholders in the publication and role of oversight groups. In other words, the process should be specified and any potential purpose of the committee functions described conflicts of interest identified. below is to lay out the roles that need to be • Liaison: In large registries, a function may be assumed by the governance structure of many specified to focus on maintaining relationships registries, but these should be individualized for a with the funding source, health care providers, particular registry. It is also possible, if methods and patients who need access to registry are clear and transparent, that oversight information. The group serving this function requirements may be minimal. may develop monitoring and satisfaction tools Registries fulfill governance roles in a variety of to ensure that the day-to-day operations of the ways. Many of the roles, for example, could be registry remain healthy. assumed by a single committee (e.g., a steering • Adjudication: Adjudication is used to review committee) in some registries. Whatever model is and confirm cases (outcomes) that may be adopted, it must accommodate all of the working difficult to classify. Individuals performing this constituencies and provide a mechanism for these function are generally blinded to the exposure individuals to work together to achieve the goals of (product or process) under study so that the the registry. confirmation of outcomes is made without All aspects of governance should be codified in a knowledge of exposure. written format that can be reviewed, shared, and • External review: External review committees refined over time. In addition, governance is a and/or advisory boards can be useful for dynamic process, subject to change in policy as providing independent oversight throughout the evidence emerges that is likely to lead to course of the registry. The majority of registries improvements in the process. will not require a data safety monitoring board Governance and oversight functions that may be (DSMB), since a DSMB is commonly used in considered include: situations where data are randomized and • Executive or steering: This function assumes treatment status is blinded. However, there may responsibility for the major financial, be situations in which the registry is administrative, legal/ethical, and scientific responsible for the primary accumulation of safety data on a particular intervention; in such

35 Section I. Creating Registries

situations, an external committee or DSMB • National research interests, such as those driven would be useful for conducting periodic by NIH. reviews (e.g., annually). • Public health policy, such as CDC policy and • Data access, use, and publications: This immunization policy. function should address the process by which The scope is also affected by the degree of registry investigators access and perform uncertainty that is acceptable to the primary analyses of registry data for the purpose of stakeholders, with that uncertainty being submitting abstracts to scientific meetings and principally driven by the quantity, quality, and developing manuscripts for peer-reviewed detail of the data collection balanced against its journal submission. Authorship (including that considered importance and value. Therefore, it is of registry sponsors) in scientific publications critical to understand the potential questions that should satisfy the conditions of the Uniform may or may not be answerable because of the Requirements for Manuscripts Submitted to 32 quantity and quality of the data. It should also be Biomedical Journals. The rules governing noted that the broader the audience of authorship may be affected by the funding stakeholders, the broader the list of questions that source, as in the case of NIH or foundation may need to be included. This increased breadth funding, or by the biomedical journal. (See can result in an increase in the number of patients Case Examples 1 and 22.) Other investigators who need to be enrolled and/or data points that may request permission to access the data. For need to be collected in order to meet the objective example, a Ph.D. candidate at an institution of the registry with an acceptable level of might seek registry-wide aggregate data for the precision. purpose of evaluating a new scientific question. A process for reviewing and responding to such Some of the specific variables that can characterize requests from other investigators or entities the scope of a registry include: should be considered in some registries that • Size: This refers to the number and complexity may generate broad external interest if the of data points, the frequency of data collection, registry stakeholders and participants are and the enrollment of investigators and agreeable to such use. patients. A registry with a large number of complex data points may allow for detailed and 2.7 Consider the Scope and Rigor thoughtful analyses but may be so burdensome Needed as to discourage investigator and patient 2.7.1 Scope of Data enrollments. In turn, a small registry with few The scope of a registry may be viewed in terms of patients and data points may be easier to execute, but the data could lack depth and be size, setting, duration, geography, and financing. 33 The purpose and objectives of the registry should less meaningful. Size also determines the frame the scope, but other factors (aside from precision with which measures of risk or risk feasibility) may ultimately shape it. For example, difference can be calculated. the scope may be affected by: • Duration: The planning of a registry must • Regulatory requirements, such as those reflect the length of time that the registry is imposed by the FDA as a condition of product expected to collect the data in order to achieve marketing. its purpose and provide analysis of the data collected. Some registries are limited by • Reimbursement decisions, such as national commercial interests, such as when the product coverage decisions by CMS or “Prior under study is approaching the end of its patent Authorization” requirements used by health life. insurers in some situations.

36 Chapter 2. Planning a Registry

• Setting: This refers to the specific setting be addressed from the perspective of the intended through which the registry will recruit uses of the data in both the short term and long investigators and patients as well as collect data term. For industry-sponsored registries, if planning (e.g., hospital, doctor’s office, pharmacy, begins at an early stage, it may be possible to home). consider whether to align registry questions with • Geography: A locally run registry is very those from the clinical trial (where appropriate) so different in scope from a global registry, in that some data can carry over for more terms of setup, management, and analysis. A comprehensive longitudinal analyses. global registry poses challenges (e.g., language, 2.7.3 Scientific Rigor cultural, time zone, regulatory) that must be The content of the data to be collected should be taken into consideration in the planning driven by the scientific analyses that are planned process. for the registry, which, in turn, are determined by • Cost: The scope of a registry will determine the the specific objectives of the registry. A registry cost of creating, managing, and analyzing the designed primarily for monitoring safety will registry. Budgetary constraints must be contain different data elements from one designed carefully considered before moving from primarily for monitoring effectiveness. Similarly, conception to reality. Additionally, the value of the extent to which data need to be validated will the information is a factor in the financial depend on the purpose of the registry and the decisions. Certain choices in planning, such as complexity of the clinical information being building on existing infrastructure and/or sought. For some outcomes, clinical diagnosis may linking to data sources relevant to the purposes be sufficient; for others, supporting documents of the registry, may increase the net return. from hospitalizations, referrals, or biopsies may be • Richness of clinical data needed: In some needed; and for others, formal adjudication by a situations, the outcome may be relatively committee may be required. Generally, registries simple to characterize (e.g., death). In other that are undertaken for regulatory decisionmaking cases, the focus of interest may be a complex will require increased attention toward diagnostic set of symptoms and measurements (e.g., for confirmation (i.e., enhanced scientific rigor). Churg-Strauss Syndrome) or may require 2.8 Define the Core Data Set, Patient specialized diagnostic testing or tissue Outcomes, and Target Population sampling (e.g., sentinel node in melanoma). Some outcomes may require assessment by an 2.8.1 Core Data Set independent third party. Depending on the Elements of data to be included must have objectives of the registry, collection and storage potential value in the context of the current of biological samples may be considered. (See scientific and clinical climate and must be chosen Section 2.7.3 below.) The collection of by a team of experts, preferably with input from biosamples itself is a rapidly evolving field, and experts in biostatistics and epidemiology. Each registry developers should consult both data element should relate to the purpose and technical and legal sources regarding how to specific objectives of the registry. Ideally, each include biosamples in a registry. data element should address the central questions 2.7.2 When Data Need To Be Available for for which the registry was designed. It is useful to Analysis consider the generalizability of the information collected, as appropriate. For example, when Meaningful data on disease progression or other seeking information on cost-effectiveness, it may long-term patient outcomes may not be available be preferable to collect data on resource utilization through a registry for many years, whereas safety rather than actual costs of this utilization, since the data could be examined periodically over time. broader descriptor can be more easily generalized Therefore, the type of data on patient outcomes to other settings and cost structures. While a and when they will be available for analysis should certain number of “speculative” fields may be

37 Section I. Creating Registries

desired to generate and explore hypotheses, these Fourth, it is useful to consider what data are must be balanced against the risk of overburdening already available and/or collected and what sites with capturing superfluous data. A plan for additional data need to be collected. When quality assurance should be considered in tandem determining additional data elements, it is with developing the core data set. imperative to consider whether the information The core data set variables (“need to know”) define desired is consistent with general practice or the information set needed to address the critical whether it might be more intensive or exceeding questions for which the registry was created. At a usual practice. For some purposes, collecting minimum, when calculating the resource needs specific laboratory results or additional visits may and overall design of the registry, registry planners be necessary, but could change how the registry is must account for these fields. If additional noncore perceived by institutional review boards or ethics variables (“nice to know”) are included, such as committees. The distinction between more descriptive or exploratory variables, it is “interventional” and “observational” is important that such data elements align with the straightforward in terms of to goals of the registry and take into account the treatment, but some registries with requirements burden of data collection and entry at the site that exceed a threshold of usual practice—in level. A parsimonious use of “nice to know” Europe, for example—may be subject to additional variables is important for several reasons. requirements more typical of “interventional” research. In Chapter 1.7.1 of Volume 9A of the First, when data elements change, there is a Rules Governing Medicinal Products in the cascade effect on all dependent components of the European Union,34 it has been clarified that registry process and outputs. For example, the registries may “collect a battery of information addition of new data elements may require changes using standardized in a prospective to the data collection system, retraining of site fashion” and “questionnaires, by themselves, are personnel on data definitions and collection not considered interventional.” These rules also practices, adjustments to the registry protocol, and state that amendment submissions to institutional review boards. Such changes often require additional • “[T]he assignment of a patient to a particular financial resources. Ideally, the registry would both strategy is not decided in advance by a [trial] limit the total number of data elements and protocol but falls within the current practice…” include, at the outset, data elements that might • “[N]o additional diagnostic or monitoring change from “nice to know” to “need to know” procedures shall be applied to patients.” during the course of the registry. In practice, this is This last requirement can be challenging to a difficult balance to achieve, so most registries interpret since registries sometimes perform should plan adequate resources to be used for diagnostic tests that are consistent with general change management. practice but that may be performed more Second, a registry should avoid attempting to frequently than would be the case in general accomplish too many goals, or its burden will practice. The determination that a registry should outweigh its usefulness to the clinical sites and be considered “interventional” from a regulatory researchers. Examples exist, however, of registries perspective can add significant burden and cost to that serve multiple purposes successfully without the registry program, and, therefore, the tradeoffs overburdening clinicians. (See Case Example 1.) must be carefully considered in planning schedules Third, even “need-to-know” variables can for registry visits and the collection of data and/or sometimes be difficult to collect reliably (e.g., use specimens. of illegal substances) or without substantial burden Finally, it is important to consider patient privacy, (e.g., unusual laboratory tests). Even with a limited national and international rules concerning ethics, core data set, feasibility must still be considered. and regulatory requirements to assure that the (See Chapter 4) registry data requirements do not jeopardize

38 Chapter 2. Planning a Registry

patient privacy or put institutional/ethics reviews One of the goals for registry data may be to enable and approvals at risk. generalization of conclusions from clinical 2.8.2 Patient Outcomes research on narrowly defined populations to broader ones, and therefore the inclusion criteria The outcomes of greatest importance should be for most (although not all) registries are relatively identified early in the concept phase of the registry. broad. As an example, screening criteria for a Delineating these outcomes (e.g., primary or registry may allow inclusion of elderly patients, secondary endpoints) will force registry designers patients with multiple comorbidities, patients on to establish priorities. Prioritization of interests in multiple therapies, patients who switch treatments the planning phase will help focus the work of the during the period of observation, or patients who registry and will guide study size requirements. are using products “off label.” The definition of the (See Chapter 3.) Identifying the patient outcomes target population will depend on many factors of the greatest importance will also help to guide (e.g., scope and cost), but ultimately will be driven the selection of the data set. Avoiding the by the purpose of the registry. temptation to collect “nice to know” data that are likely of marginal value is of paramount As with defining patient outcomes, target importance, yet some registries do, in fact, need to population criteria and/or definitions should be collect large amounts of data to accomplish their consistent with established guidelines and purposes. Possessing adequate data in order to standards within the therapeutic area. Achieving properly address potential confounders during this goal increases the potential utility of the analyses is one reason that extensive data registry by leveraging other data sources (historical collection is sometimes required.35 or concurrent) with different information on the same target population and enhancing statistical Methods to ascertain the principal outcomes power if similar information is collected on the should be clearly established. The diagnostic target population. requirements, level of data detail, and level of data validation and/or adjudication should also be In establishing target population criteria, addressed. As noted below in the context of consideration should be given to the feasibility of identifying a target population, relying on access to that population. One should try to established guidelines and standards to aid in distinguish the ideal from the real. Some questions defining outcomes of interest has many benefits to consider in this regard are: and should be considered. • How common is the exposure or disease of The issues of ascertainment noted here are interest? important to consider because they will have a • Can eligible people be readily identified? bearing on some attributes by which registries may 36 • Are other sources competing for data on the be evaluated. These attributes include sensitivity same patients? (the extent to which the methods identify all outcomes of interest) and external validity • Is care centralized or dispersed (e.g., in a (generalizability to similar populations), among referral or tertiary care facility)? others. • How mobile is the target population? 2.8.3 Target Population Ultimately, methods to ascertain members of the The target population is the population to which target population should be carefully considered the findings of the registry are meant to apply. It (e.g., use of screening logs that identify all must be defined for two basic reasons. First, the potential patients and indicate whether they target population serves as the foundation for participate and, if not, why not), as should the use planning the registry. Second, it also represents a of sources outside the registry (e.g., patient major constituency that will be impacted by the groups). Greater accessibility to the target results of the registry. population will reap benefits in terms of enhanced representativeness and statistical power.

39 Section I. Creating Registries

Lastly, thought should be given to comparison • Quality management plan to describe the (control) groups either internal or external to the procedures to be used to test project concepts, registry. Again, much of this consideration will be ideas, and decisions in the process of building a driven by the purpose and specific objectives of registry. Having a quality management plan in the registry. For example, natural history registries place can help in detecting design errors early, do not need controls, but controls are especially formulating necessary changes to the scope, desirable for registries created to evaluate and ensuring that the final product meets comparative effectiveness or safety. stakeholders’ expectations. • Staffing management plan to determine what 2.9 Develop a Study Plan or Protocol skills will be needed and when to meet the The study plan documents the objectives of the project goals. (See Chapter 2, Section 2.5). registry and describes how those objectives will be achieved. At a minimum, the study plan should • Communication plan that includes who is include the registry objectives, the eligibility responsible for communicating information and criteria for participants, and the data collection to whom it should be communicated. procedures. Ideally, a full study protocol will be Considerations include different categories of developed to document the objectives, design, information, frequency of communications, and participant inclusion/exclusion criteria, outcomes methods of communication. The plan should of interest, data to be collected, data collection also provide steps to escalate issues that cannot procedures, governance procedures, and plans for be resolved on a lower staff level. complying with ethical obligations and protecting • Procurement plan for external components or patient privacy. equipment and/or outsourced software In addition to a study plan or protocol, registries development for the planned registry, if may have statistical analysis plans. Chapters 13 pertinent. Such a plan should describe how the and 25 discuss the importance of analysis plans. procurement process would be managed within the organization. Decisions to procure products 2.10 Develop a Project Plan or services may have a direct impact on other components of the project plan, including the Developing an overall project plan is critically staffing plan and timeline. important so that the registry team has a roadmap to guide their collective efforts. Depending on the • Risk management plan to identify and mitigate complexity of the registry project, the project plan risks. Many project risks are predictable events, may include some or all of the following elements: and therefore they can and should be assessed in the very early stages of registry planning. It • Scope management plan to control the scope of is important to prioritize project risks by their the project. It should provide the approach to potential impact on the specific objectives and making changes to the scope through a clearly to develop an adequate risk response plan for defined change-control system. the most significant risks. Some predictable • Detailed timeline and schedule management risks include— plan to ensure that the project and its –– Disagreement between stakeholders over the deliverables are completed on time. scope of specific tasks. • Cost management plan for keeping project –– Inaccurate cost estimates. costs within the budget. The cost management plan may provide estimates on cost of labor, –– Delays in the timeline. purchases and acquisitions, compliance with regulatory requirements, et cetera. This plan should be aligned with the change-control system so that all changes to the scope will be reflected in the cost component of the registry project.

40 Chapter 2. Planning a Registry

3. Summary guide the registry; these individuals should be selected based on their expertise and experience. In summary, planning a patient registry involves Governance and oversight for the registry should several key steps, including articulating its also be addressed during the planning phase. purpose, determining whether it is an appropriate While registries differ tremendously in size, scope, means of addressing the research question, and resource requirements, the basic elements of identifying stakeholders, defining the scope and planning described here are relevant for most, if target population, assessing feasibility, and not all registries, and can help to support the securing funding. A registry team and advisors launch and operation of a successful registry. must be assembled to develop, coordinate, and

Case Example for Chapter 2

Case Example 1. Creating a registry to fulfill are time sensitive and must be administered very multiple purposes and using a publications soon after hospital arrival in order to be most committee to review data requests effective. Description The National Registry of After the release of its first fibrinolytic therapy Myocardial Infarction (NRMI) product in 1987, the sponsor’s field collected, analyzed, and representatives learned from their discussions disseminated data on patients with emergency department physicians, experiencing acute myocardial cardiologists, and hospital staff that most infarction. Its goal was clinicians believed they were treating patients improvement of patient care at quickly, although there was no documentation or individual hospitals through the benchmarking to confirm this assumption or to hospital team’s evaluation of identify and correct delays. At that time, many data and assessment of care emergency departments did not have readily delivery systems. available diagnostic tools (such as angiography Sponsor Genentech, Inc. labs), and hospitals with AMI-specific decision pathways and treatment protocols were the Year Started 1990 exception rather than the rule. Year Ended 2006 In addition, since fibrinolytic therapy was being No. of Sites 451 hospitals in the final phase widely used for the first time, the sponsor wanted of NRMI (NRMI 5). Over 2,150 to gather safety information related to its use in hospitals participated in NRMI real-world situations and in a broader range of over 16 years. patients than those treated in the controlled environment of a clinical trial. No. of Patients 2,515,106 Proposed Solution Challenge The sponsor decided to create the registry to Over the past 20 years, there have been fulfill the multiple purposes of identifying significant changes in the treatment of acute treatment patterns, promoting time-to-treatment myocardial infarction (AMI) patients. Evidence and other quality improvements, and gathering from large clinical trials has led to the real-world safety data. The scope of the data introduction of new guidelines and therapies for collection necessary to meet these needs could treating AMI patients, including fibrinolytic have made such a registry impracticable, so the therapy and percutaneous coronary intervention. project team faced the sizable challenge of While these treatments can improve both balancing the data needs with the feasibility of morbidity and mortality for AMI patients, they the registry.

41 Section I. Creating Registries

Case Example 1. Creating a registry to fulfill process that would allow outside researchers to multiple purposes and using a publications access the registry data without overburdening committee to review data requests (continued) the registry team. Proposed Solution (continued) The registry team created a publication process to The sponsor formed a scientific advisory board determine when another group could use the data with members representing the various clinical for research. The team set high-level criteria for stakeholders (emergency department, cardiology, all data requests: the analysis had to be feasible nursing, research, etc.). The scientific advisory given the data in the registry, and the request board developed the data set for the registry, could not represent a duplication of another keeping a few guiding principles in mind. These research effort. principles emphasized maintaining balance The registry team involved its scientific advisory between the clinical research and the feasibility board, made up of cardiologists, emergency of the registry. The first principle was to department physicians, nurses, research scientists, determine whether the proposed data element was pharmacists, and reviewers with specialties in necessary by asking several key questions: How biostatistics and statistical programming, in will the data element be used in generating creating a publication review committee. The hospital feedback reports or research analyses? Is review committee evaluated all research the data element already collected? If not, should proposals to determine originality, interest to it be collected? If it should be collected, is it peers, feasibility, appropriateness, and priority. feasible to collect those data? The second The review committee limited its review of principle focused on using existing data standards research proposals to a set number of reviews per whenever possible. If a data standard did not year, and scheduled the reviews and deadlines exist, the team tried to collect the data in the around the abstract deadlines for the major simplest possible way. The third principle cardiology conferences. Research analyses had to emphasized data consistency and making the be intended to result in peer-reviewed registry user-friendly by continually refining data presentations and publications. Researchers were element definitions until they were as clear as asked to submit proposals that included well possible. defined questions and an analysis plan. If the In 1990, the sponsor launched the registry. proposal was accepted, the researchers discussed During the 16 years that the registry was any further details with the biostatisticians and conducted, it demonstrated that the advisory statistical programmers who performed the board’s efforts to create a feasible multipurpose analyses (and who were employed at an registry were successful. The registry collected independent clinical research organization). The data on the clinical presentation, treatment, and results were sent directly to the researchers. outcomes of over 2.5 million patients with AMI The scientific advisory board and review from more than 2,150 participating sites. committee remained involved in the process after The success of the registry presented a new a data request had been granted. All authors challenge for the registry team. The sponsor submitted their abstracts to the review committee received a large volume of requests to analyze the before sending them to conferences. The review registry data, often for research topics that fell committee offered constructive criticism to help outside of the standardized reports developed for the authors improve their abstracts. The review the registry. As a guiding principle, the registry committee also reviewed manuscripts before team was committed to making the data available journal submission to help identify any issues or for research projects, but it had limited resources. concerns that the authors should address. To support these requests, the team developed a

42 Chapter 2. Planning a Registry

Case Example 1. Creating a registry to fulfill groups interested in using the data. The registry multiple purposes and using a publications team may want to set up a publication process committee to review data requests (continued) during the registry design phase. Results For More Information This publication process enabled the wealth of Califf RM. The benefits of moving quality to a data collected in this registry to be used in over national level. Am Heart J. 2008;156(6):1019–22. 150 scientific abstracts and 100 peer-reviewed Rogers WJ, Frederick PD, Stoehr E, et al. NRMI articles, addressing each of the purposes of the Investigators; Trends in presenting characteristics registry as well as other research topics. By and hospital mortality among patients with ST involving the scientific advisory board and elevation and non-ST elevation myocardial providing independent biostatistical support, the infarction in the NRMI from 1990 to 2006. Am registry team developed an infrastructure that Heart J. 2008;156(6):1026–34. enhanced the credibility of the research uses of this observational database. Gibson CM, Pride YB, Frederick PD. et al. NRMI Investigators; Trends in reperfusion strategies, Key Point door-to-needle and door-to-balloon times, and Registries can be developed to fulfill more than in-hospital mortality among patients with ST- one purpose, but this added complexity requires segment elevation myocardial infarction enrolled careful planning to ensure that the final registry in the NRMI from 1990 to 2006. Am Heart J. data collection burden and procedures are 2008;156(6):1035–44. feasible. Making sure that the advisory board Peterson ED, Shah BR, Parsons L. et al. NRMI includes representatives with clinical and Investigators; Trends in quality of care for operational perspectives can help the board to patients with acute myocardial infarction in the maintain its focus on feasibility. As a registry NRMI from 1990 to 2006. Am Heart J. database gains large amounts of data, the registry 2008;156(6):1045–55. team will likely receive research proposals from

References for Chapter 2 5. Dreyer NA, Garner S. Registries for robust evidence. JAMA. 2009 Aug 19;302(7):790-1. 1. ISPE. Guidelines for good pharmacoepidemiology PMID: 19690313. practices (GPP). Pharmacoepidemiol Drug Saf. 6. Solomon DJ, Henry RC, Hogan JG, et al. 2008 Feb;17(2):200-8. PMID: 17868186. Evaluation and implementation of public health 2. German RR, Lee LM, Horan JM, et al. Updated registries. Public Health Rep. 1991 Mar- guidelines for evaluating public health Apr;106(2):142-50. PMID: 1902306. surveillance systems: recommendations from the PMCID: 1580226. Guidelines Working Group. MMWR Recomm 7. Glaser SL, Clarke CA, Gomez SL, et al. Cancer Rep. 2001 Jul 27;50(RR-13):1-35; quiz CE1-7. surveillance research: a vital subdiscipline of PMID: 18634202. cancer epidemiology. Cancer Causes Control. 3. Project Management Institute. A Guide to the 2005 Nov;16(9):1009-19. PMID: 16184466. Project Management Body of Knowledge 8. Kennedy L, Craig AM. Global registries for (PMBOK Guide). 4th ed. Newtown Square, PA: measuring pharmacoeconomic and quality-of-life Project Management Institute; 2008. outcomes: focus on design and data collection, 4. Dreyer NA, Schneeweiss S, McNeil BJ, et al. analysis and interpretation. Pharmacoeconomics. GRACE principles: recognizing high-quality 2004;22(9):551-68. PMID: 15209525. observational studies of comparative effectiveness. Am J Manag Care. 2010 Jun;16(6):467-71. PMID: 20560690.

43 Section I. Creating Registries

9. Bookman MA. Using tumor registry resources in 20. Wikipedia. Public-Private Partnerships. http:// analyzing concordance with guidelines and en.wikipedia.org/wiki/Public-private_partnership. outcomes. Oncology (Williston Park). 2000 Accessed August 14, 2012. Nov;14(11A):104-7. PMID: 11195403. 21. Riverside County Department of Public Health. 10. Agency for Healthcare Research and Quality. Vaxtrack Immunization Registry. http://www. Developing a Registry of Patient Registries naccho.org/topics/modelpractices/database/ (RoPR). http://www.effectivehealthcare.ahrq.gov/ practice.cfm?practiceID=139. Accessed August index.cfm/search-for-guides-reviews-and-reports/? 14, 2012. pageaction=displayproduct&productid=690. 22. California Immunization Registry (CAIR). http:// Accessed August 17, 2012. www.ca-siis.org. Accessed August 14, 2012. 11. Avorn J. In defense of pharmacoepidemiology-- 23. North Carolina Department of Health and Human embracing the yin and yang of drug research. Services. N.C Public-Private Partnership Creates N Engl J Med. 2007 Nov 29;357(22):2219-21. Statewide Bioterrorism Surveillance System. PMID: 18046025. http://www.infectioncontroltoday.com/ 12. Vandenbroucke JP. Observational research, news/2004/07/public-private-partnership-creates- randomised trials, and two views of medical statewide-biot.aspx. Accessed August 14, 2012. science. PLoS Med. 2008 Mar 11;5(3):e67. 24. Ray WA, Stein CM. Reform of drug regulation-- PMID: 18336067. PMCID: 2265762. beyond an independent drug-safety board. N Engl 13. Alter DA, Venkatesh V, Chong A, et al. Evaluating J Med. 2006 Jan 12;354(2):194-201. the performance of the Global Registry of Acute PMID: 16407517. Coronary Events risk-adjustment index across 25. Strom BL. How the US drug safety system should socioeconomic strata among patients discharged be changed. JAMA. 2006 May 3;295(17):2072-5. from the hospital after acute myocardial PMID: 16670415. infarction. Am Heart J. 2006 Feb;151(2):323-31. PMID: 16442894. 26. Okie S. Safety in numbers--monitoring risk in approved drugs. N Engl J Med. 2005 Mar 14. Hernan MA, Hernandez-Diaz S, Werler MM, et 24;352(12):1173-6. PMID: 15788493. al. Causal knowledge as a prerequisite for confounding evaluation: an application to birth 27. Rawlins M. De Testimonio: on the evidence for defects epidemiology. Am J Epidemiol. 2002 Jan decisions about the use of therapeutic 15;155(2):176-84. PMID: 11790682. interventions. Clin Med. 2008 Dec;8(6):579-88. PMID: 19149278. 15. Lash TL, Fox MP, Fink AK. Applying quantitative bias analysis to epidemiologic data. New York, 28. Lyratzopoulos G, Patrick H, Campbell B. NY: Springer; 2009. Registers needed for new interventional procedures. Lancet. 2008 May 16. Retchin SM, Wenzel RP. Electronic medical 24;371(9626):1734-6. PMID: 18502284. record systems at academic health centers: advantages and implementation issues. Acad Med. 29. American Recovery and Reinvestment Act of 1999 May;74(5):493-8. PMID: 10353279. 2009, H.R.1. http://www.gpo.gov/fdsys/pkg/ BILLS-111hr1enr/pdf/BILLS-111hr1enr.pdf. 17. Andersen MR, Schroeder T, Gaul M, et al. Using a Accessed August 14, 2012. population-based cancer registry for recruitment of newly diagnosed patients with ovarian cancer. 30. Food and Drug Administration Amendments Act Am J Clin Oncol. 2005 Feb;28(1):17-20. of 2007., Pub. L. No. 110-85 (2007), Title VI. PMID: 15685029. Sect. 603. Critical Path Public-Private Partnerships.http://www.gpo.gov/fdsys/pkg/ 18. Tunis SR. A clinical research strategy to support PLAW-110publ85/html/PLAW-110publ85.htm. shared decision making. Health Aff (Millwood). Accessed August 14, 2012. 2005 Jan-Feb;24(1):180-4. PMID: 15647229. 31. Davidoff F, DeAngelis CD, Drazen JM, et al. 19. INTERMACS. http://www.uab.edu/intermacs/. Sponsorship, authorship, and accountability. Ann Accessed August 14, 2012. Intern Med. 2001 Sep 18;135(6):463-6. PMID: 11560460.

44 Chapter 2. Planning a Registry

32. International Committee of Medical Journal 35. Mangano DT, Tudor IC, Dietzel C, et al. The risk Editors. Uniform Requirements for Manuscripts associated with aprotinin in cardiac surgery. Submitted to Biomedical Journals: Writing and N Engl J Med. 2006 Jan 26;354(4):353-65. Editing for Biomedical Publication. http://www. PMID: 16436767. icmje.org. Accessed August 14, 2012. 36. Newton J, Garner S. Disease registers in England. 33. Woodward M. Sample size determination. Report commissioned by the Department of Epidemiology: study design and data analysis. Health Policy Research Programme. Institute of Boca Raton (FL): Chapman & Hall/CRC Press; Health Sciences. University of Oxford; Feb, 2002. 2005:chapter 8. http://www.sepho.org.uk/Download/ Public/5445/1/disease_registers_in_england.pdf. 34. Guidelines on Pharmacovigilance of Medical Accessed August 27, 2012. Products for Human Use, Volume 9A of the Rules Governing Medicinal Products in the European Union. Sep2008. http://ec.europa.eu/health/ documents/eudralex/vol-9/index_en.htm. Accessed August 14, 2012.

45

Chapter 3. Registry Design

1. Introduction in this chapter. Throughout the design process, registry planners may want to discuss options and This chapter is intended as a high-level practical decisions with the registry stakeholders and guide to the application of epidemiologic methods relevant experts to ensure that sound decisions are that are particularly useful in the design of made. The choice of groups to be consulted during registries that evaluate patient outcomes. Since it is the design phase generally depends on the nature not intended to replace a basic textbook on of the registry, the registry funding source and epidemiologic design, readers are encouraged to funding mechanism, and the intended audience for seek more information from textbooks and registry reporting. More detailed discussions of scientific articles. Table 3–1 summarizes the key registry design for specific types of registries are considerations for study design that are discussed provided in Chapters 19, 20, 21, 22, and 23.

Table 3–1. Considerations for study design

Construct Relevant Questions

Research question What are the clinical and/or public health questions of interest? Resources What resources, in terms of funding, sites, clinicians, and patients, are available for the study? Exposures and outcomes How do the clinical questions of interest translate into measurable exposures and outcomes? Data sources Where can the necessary data elements be found? Study design What types of design can be used to answer the questions or fulfill the purpose? Study population What types of patients are needed for study? Is a comparison group needed? How should patients be selected for study? Sampling How should the study population be sampled, taking into account the target populations and study design? Study size and duration For how long should data be collected, and for how many patients? Internal and external validity What are the potential biases? What are the concerns about generalizability of the results (external validity)?

2. Research Questions effectiveness or assessing safety or harm are generally hypothesis driven and concentrate on Appropriate for Registries evaluating the effects of specific treatments on The questions typically addressed in registries patient outcomes. Research questions should range from purely descriptive questions aimed at address the registry’s purposes, as broadly understanding the characteristics of people who described in Table 3–2. develop the disease and how the disease generally Observational studies derived from registries (or progresses, to highly focused questions intended to “registry-based studies”) are an important part of support decisionmaking. Registries focused on the research armamentarium alongside determining clinical effectiveness or cost- interventional studies, such as randomized

47 Section I. Creating Registries

controlled trials (RCTs) or pragmatic trials, and complementary than competitive, precisely retrospective studies, such as studies derived because some research questions are better exclusively from administrative claims data. Each answered by one method than the other. of these study designs has strengths and Interventional studies are considered by many to limitations, and the selection of a study design provide the highest grade evidence for evaluating should be guided by the research questions of whether a drug has the ability to bring about an interest. (See Chapter 2, Section 2.2, for a intended effect in optimal or “ideal world” discussion of the factors that influence the study situations, a concept also known as “efficacy.”1 design decision.) In some cases, multiple studies Observational designs, on the other hand, are with different designs or a hybrid study that particularly well suited for studying broader combines study designs will be necessary to populations, understanding actual results (e.g., address a research question. In fact, this more some safety outcomes) in real-world practice (see comprehensive approach to evidence development Case Example 2), and for obtaining more is likely to become more common as researchers representative quality-of-life information. This is strive to address multiple questions for multiple particularly true when the factors surrounding the stakeholders most efficiently. Observational decision to treat are an important aspect of studies and interventional studies are more understanding treatment effectiveness.2

Table 3–2. Overview of registry purposes

• Assessing natural history, including estimating the magnitude of a problem; determining the underlying incidence or prevalence rate of a condition; examining trends of disease over time; conducting surveillance; assessing service delivery and identifying groups at high risk; documenting the types of patients served by a health provider; and describing and estimating survival. • Determining clinical effectiveness, cost-effectiveness, or comparative effectiveness of a test or treatment, including for the purpose of determining reimbursement. • Measuring or monitoring safety and harm associated with the use of specific products and treatments, including conducting comparative evaluation of safety and effectiveness. • Measuring or improving quality of care, including conducting programs to measure and/or improve the practice of medicine and/or public health.

In many situations, nonrandomized comparisons (See Chapter 21 and Case Examples 49, 50, 51, either are sufficient to address the research and 52.) question or, in some cases, may be necessary • Practicality: Will patients enroll in a study because of the following issues with randomizing where they might not receive the treatment, or patients to a specific treatment: might not receive what is likely to be the best • Equipoise: Can providers ethically introduce treatment? How can compliance and adherence randomization between treatments when the to a treatment be studied, if not by observing treatments may not be clinically equivalent? what people do in real-world situations? • Ethics: If reasonable suspicion about the safety Registries are particularly suitable for some types of a product has become known, would it be of research questions, such as: ethical to conduct a trial that deliberately • Natural history studies where the goal is to exposes patients to potential harm? For observe clinical practice and patient experience example, can pregnant women be ethically but not to introduce any intervention. exposed to drugs that may be teratogenic?

48 Chapter 3. Registry Design

• Measures of clinical effectiveness, especially as Registry studies may also include embedded related to compliance, where the purpose is to substudies as part of their overall design. These learn about what patients and practitioners substudies can themselves have various designs actually do and how their actions affect real- (e.g., highly detailed prospective data collection on world outcomes. This is especially important a subset of registry participants, or a case-control for treatments that have poor compliance. study focused on either incident or prevalent cases • Studies of effectiveness and safety for which identified within the registry). (See Case Examples clinician training and technique are part of the 3 and 47.) Registries can also be used as sampling study of the treatment (e.g., a procedure such as frames for RCTs. placement of carotid stent). 3. Translating Clinical • Studies of heterogeneous patient populations, since unlike randomized trials, registries Questions Into Measurable generally have much broader inclusion criteria Exposures and Outcomes and fewer exclusion criteria. These characteristics lead to studies with greater The specific clinical questions of interest in a generalizability (external validity) and may registry will guide the definitions of study allow for assessment of subgroup differences in subjects, exposure, and outcome measures, as well treatment effects. as the study design, data collection, and analysis. In the context of registries, the term “exposure” is • Followup for delayed or long-term benefits or used broadly to include treatments and procedures, harm, since registries can extend over much health care services, diseases, and conditions. longer periods than most clinical trials (because of their generally lower costs to run and lesser The clinical questions of interest can be defined by burden on participants). reviewing published clinical information, soliciting experts’ opinions, and evaluating the expressed • Surveillance for rare events or of rare diseases. needs of the patients, health care providers, and • Studies for treatments in which randomization payers. Examples of research questions, key is unethical, such as intentional exposure to outcome and exposure variables, and sources of potential harm (as in safety studies of marketed data are shown in Table 3–3. As these examples products that are suspected of being harmful). show, the outcomes (generally beneficial or • Studies for treatments in which randomization deleterious outcomes) are the main endpoints of is not necessary, such as when certain therapies interest posed in the research question. These are only available in certain places owing to typically represent measures of health or onset of high cost or other restrictions (e.g., proton illness or adverse events, but also commonly beam therapy). include quality of life measures, and measures of health care utilization and costs. • Studies for which blinding is challenging or unethical (e.g., studies of surgical interventions, Relevant exposures also derive from the main acupuncture). research question and relate to why a patient might experience benefit or harm. Evaluation of an • Studies of rapidly changing technology. exposure includes collection of information that • Studies of conditions with complex treatment affects or augments the main exposure, such as patterns and treatment combinations. dose, duration of exposure, route of exposure, or adherence. Other variables of interest include • Studies of health care access and barriers to independent risk factors for the outcomes of care. interest (e.g., comorbidities, age), as well as • Evaluations of actual standard medical practice. variables known as potential confounding (See Case Example 58.) variables, that are related to both the exposure and the outcome and are necessary for conducting

49 Section I. Creating Registries

valid statistical analyses. Confounding can result new drug (meaning that doctors will be more in inaccurate estimates of association between the likely to try a new drug in patients who have failed study exposure and outcome through mixing of other therapies) and the likelihood of having a effects. For example, in a study of asthma poorer outcome (e.g., hospitalization). Refer to medications, prior history of treatment resistance Chapter 4 for a discussion of selecting data should be collected or else results may be biased. elements and Chapter 5 for a discussion of The bias could occur because treatment resistance selecting patient-reported outcomes. may relate both to the likelihood of receiving the

Table 3–3. Examples of research questions and key exposures and outcomes

Key Exposure (source of Research Question data) Key Outcome (source of data)

What is the expected time to rejection All immunosuppressants, Organ rejection (clinician or medical for first kidney transplants among adults, including dosage and duration record) and how does that differ according to (clinician or medical record) immunosuppressive regimen? Are patients using a particular treatment Treatments for disease of Ability to independently perform better able to perform activities of daily interest (clinician or medical key activities related to daily living living than others? record) (patient) Do patients undergoing gastric bypass Surgery (clinician or medical Number of inpatient and outpatient surgery for weight loss use fewer health record) visits, medications dispensed, care resources in the year following associated costs (administrative surgery? databases, clinician, or medical record) Are patients using a particular drug more Drug use by mother during Pregnancy outcome (clinician, likely to have serious adverse pregnancy pregnancy (clinician, medical medical record, or patient) outcomes? record, or patient)

4. Finding the Necessary Data clinicians. Registry data can be obtained from patients, clinicians, medical records, and linkage The identification of key outcome and exposure with other sources (in particular, extant databases), variables and patients will drive the strategy for depending on the available budget. (See Chapters data collection, including the choice of data 6, 15, and 16.) sources. A key challenge to registries is that it is Examples of patient-reported data include health- generally not possible to collect all desired data. related quality of life; utilities (i.e., patient As discussed in Chapter 4, data collection should preferences); symptoms; use of over-the-counter be both parsimonious and broadly applicable. For (OTC), complementary, and alternative example, while experimental imaging studies may medication; behavioral data (e.g., smoking and provide interesting data, if the imaging technology alcohol use); family history; and biological is not widely available, the data will not be specimens. These data may rely on the subjective available for enough patients to be useful for interpretation and reporting of the patient (e.g., analysis. Moreover, the registry findings will not health-related quality of life, utilities, symptoms be generalizable if only sophisticated centers that such as pain or fatigue); may be difficult to have such technology participate. Instead, otherwise track (e.g., use of complementary and registries should focus on collecting relevant data alternative medication, smoking, and alcohol use); with relatively modest burden on patients and

50 Chapter 3. Registry Design

or may be unique to the patient (e.g., biological Permanente has created several registries of specimens). Health care resource utilization is patients receiving total joint replacement, bariatric another important construct that reflects both cost surgery, and nonsurgical conditions of care (burden of illness) and health-related quality (e.g., diabetes), all of which rely heavily on of life. For example, more frequent office visits, existing electronic health record data. As discussed procedures, or hospitalizations may result in further in Chapter 15, the availability of medical reduced health-related quality of life for the patient. records data in electronic format does not, by The primary advantage of this form of data itself, guarantee consistency of terminology and collection is that it provides direct information from coding. the entity that is ultimately of the most interest— Examples of other data sources include health the patient. The primary disadvantages are that the insurance claims, pharmacy data, laboratory data, patient is not necessarily a trained observer and that other registries, and national data sets, such as various forms of bias, such as recall bias, may Medicare claims data and the National Death influence subjective information. For example, Index. These sources can be used to supplement people may selectively recall certain exposures registries with data that may otherwise be difficult because they believe they have a disease that was to obtain, subject to recall bias, not collected caused by that exposure, or their recall may be because of loss to followup, or likely inaccurate by influenced by recent news stories claiming cause- self-report (e.g., in those patients with diseases and-effect relationships. (See Case Example 4.) affecting recall, cognition, or mental status). See Examples of clinician data include clinical Table 6–1 (in Chapter 6) for more information on impressions, clinical diagnoses, clinical signs, data sources. differential diagnoses, laboratory results, and staging. The primary advantage of clinician data is 5. Resources and Efficiency that clinicians are trained observers. Even so, the primary disadvantages are that clinicians are not Ideally, a study is designed to optimally answer a necessarily accurate reporters of patient research question of interest and funded perceptions, and their responses may also be adequately to achieve the objectives based on the subject to recall bias. Moreover, the time that busy requirements of the design. Frequently, however, clinicians can devote to registry data collection is finite resources are available at the outset of a often limited. project that constrain the approaches that may be Medical records also are a repository of clinician- pursued. Often, through efficiencies in the derived data. Certain data about treatments, risk selection of a study design and patient population factors, and effect modifiers are often not (observational vs. RCT, case-control vs. consistently captured in medical records of any prospective cohort), selection of data sources (e.g., type, but where available, can be useful. Examples medical-records–based studies vs. information of such data that are difficult to find elsewhere collected directly from clinicians or patients), include OTC medications, smoking and alcohol restriction of the number of study sites, or other use, complementary and alternative medicines, and approaches, studies may be planned that provide counseling activities by the clinician on lifestyle adequate evidence for addressing a research modifications. Medical records are often relied question, in spite of limited resources. upon as a source of detailed clinical information Section 6 below discusses how certain designs for adjudication by external reviewers of medical may be more efficient for addressing some diagnoses corresponding to study endpoints. research questions. Electronic medical records, increasingly available, improve access to the data within medical records. The increasing use of electronic health records has facilitated the development of a number of registries within large health plans. Kaiser

51 Section I. Creating Registries

6. Study Designs for Registries 6.1 Case Series Design Using a registry population to develop case series Although studies derived from registries are, by is a straightforward application that does not definition, observational studies, the framework for require sophisticated analytics. Depending on the how the data will be analyzed drives the data generalizability of the registry itself, case series collection and choices of patients for inclusion in drawn from the registry can be used to describe the the study. characteristics to be used in comparison to other The study models of case series, cohort, case- case series (e.g., from spontaneous adverse event control, and case-cohort are commonly applied to reports). Self-controlled methods, including registry data and are described briefly here. When self-controlled case series, are a relatively new set case-control or case-cohort designs are applied to of methods that lends itself well to registry registry data, additional data may be collected to analyses as it focuses on only those subjects who facilitate examination of questions that arise. have experienced the event of interest and uses an Before adding new data elements, whether in a internal comparison to derive the relative (not nested substudy or for a new objective, several of absolute) incidence of the event during the time the steps outlined in Chapter 2, including assessing the subject is “exposed” compared with the feasibility, considering the necessary scope and incidence during the time when they are rigor, and evaluating the regulatory/ethical impact, “unexposed.”4 This design implicitly controls for should be undertaken. Other models that are also all confounders that do not vary over the followup useful in some situations, but are not covered here, time (e.g., gender, genetics, geographic area), as include: case-crossover studies, which are efficient the subject serves as his or her own control. The designs for studying the effects of intermittent self-controlled case series design may also be very exposures (e.g., use of erectile dysfunction drugs) useful in those circumstances where a comparison on conditions with sudden onset, and quasi- group is not available. Self-controlled case series experimental studies or “pragmatic trials.” For require that the probability of exposure is not example, in a pragmatic trial, providers may be affected by the occurrence of an outcome; in randomized as to which intervention or quality addition, for non-recurrent events, the method improvement tools they use, but patients are works only when the event risk is small and varies observed without further intervention. Also, there over the followup time. Derivative methods, has been recent interest in applying the concept of grouped as self-controlled cohort methods, include design to registries. An observational screening5 and temporal pattern adaptive design has been defined as a design that discovery.6 These methods compare the rate of allows adaptations or modifications to some events post-exposure with the rate of events aspects of a clinical trial after its initiation without pre-exposure among patients with at least one undermining the validity and integrity of the trial.3 exposure. While many long-term registries are modified after initiation, the more formal aspects of adaptive trial 6.2 Cohort Design design have yet to be applied to registries and Cohort studies follow, over time, a group of people observational studies. who possess a characteristic, to see if individuals Determining what framework will be used to in the group develop a particular endpoint or analyze the data is important in designing the outcome. The cohort design is used for descriptive registry and the registry data collection studies as well as for studies seeking to evaluate procedures. Readers are encouraged to consult comparative effectiveness and/or safety or quality textbooks of epidemiology and of care. Cohort studies may include only people pharmacoepidemiology for more information. with exposures (such as to a particular drug or Many of the references in Chapters 13 and 18 class of drugs) or disease of interest. Cohort relate to study design and analysis. studies may also include one or more comparison groups for which data are collected using the same

52 Chapter 3. Registry Design

methods during the same period. A single cohort requirement of arising from the same source study may in fact include multiple cohorts, each population as the cases to which they will be defined by a common disease or exposure. Cohorts compared. Matching in case-control designs—for may be small, such as those focused on rare example, ensuring that patient characteristics such diseases, but often they target large groups of as age and gender are similar in the cases and their people (e.g., in safety studies), such as all users of controls—may yield additional efficiency, in that a a particular drug or device. Some limitations of smaller number of subjects may be required to registry-based cohort studies may include limited answer the study question with a given power. availability of treatment data and underreporting However, matching does not eliminate of outcomes if a patient leaves the registry or is not confounding and must be undertaken with care. adequately followed up.7 These pitfalls should be Matching variables must be accounted for in the considered and addressed when planning a study. analysis, because a form of selection bias similar to confounding will have been introduced by the 6.3 Case-Control Design matching.11 A case-control study gathers patients who have a Properly executed, a case-control study can add particular outcome or who have suffered an efficiency to a registry if more extensive data are adverse event (“cases”) and “controls” who have collected by the registry only for the smaller not but are representative of the source population number of subjects selected for the case-control from which the cases arise.8 If properly designed study. This design is sometimes referred to as a and conducted, it should yield results similar to “nested” case-control study, since subjects are those expected from a of the taken from a larger cohort. It is generally applied population from which the cases were derived. The because of budgetary or logistical concerns case-control design is often employed for relating to the additional data desired. Nested understanding the etiology of rare diseases9 case-control studies have been conducted in a wide because of its efficiency. In studies where range of patient registries, from studying the expensive data collection is required, such as some association between oral contraceptives and genetic analyses or other sophisticated testing, the various types of cancer using the Surveillance case-control design is more efficient and cost Epidemiology and End Results (SEER) effective than a cohort study because a case- program12-14 to evaluating the possible association control design collects information only from of depression with Alzheimer’s disease. As an cases and a sample of noncases. However, if no de example, in the latter case-control study design, novo data collection is required, the use of the probable cases were enrolled from an Alzheimer’s cohort design may be preferable since it avoids the disease registry and compared with randomly challenge of selecting a suitable control group and selected nondemented controls from the same base the concomitant danger of introducing more bias. population.15 Depending on the outcome or event of interest, Case-control studies present special challenges cases and controls may be identifiable within a with regard to control selection. More information single registry. For example, in the evaluation of on considerations and strategies can be found in a restenosis after coronary angioplasty in patients set of papers by Wacholder.16-18 with end-stage renal disease, investigators identified both cases and controls from an 6.4 Case-Cohort Design institutional percutaneous transluminal coronary The case-cohort design is a variant of the case- angioplasty registry; in this example, controls were control study. As in a case-control study, a case- randomly selected from the registry and matched 10 cohort study enrolls patients who have a particular by age and gender. Alternatively, cases can be outcome or who have suffered an adverse event identified in the registry and controls chosen from (“cases”), and “controls” who have not, but who outside the registry. Care must be taken, however, are representative of the source population from that the controls from outside the registry meet the

53 Section I. Creating Registries

which the cases arise. In nested case-control –– Examples include studies of the occurrence studies where controls are selected via risk-set of cancer or rare diseases, pregnancy sampling, each person in the source population has outcomes, and recruitment pools for clinical a probability of being selected as a control that is, trials. ideally, in proportion to his or her person-time • Those with a particular exposure. (These contribution to the cohort. In a case-cohort study, exposures may be to a product, procedure, or however, each control has an equal probability of other health service.) being sampled from the source population.19 This allows for collection of pertinent data for cases –– Examples include general surveillance and for a sample of the full cohort, instead of the registries, pregnancy registries for particular whole cohort. For example, in a case-cohort study drug exposures, and studies of exposure to of histopathologic and microbiological indicators medications and to devices such as stents.21 of chorioamnionitis, which included identification They also include studies of people who of specific microorganisms in the placenta, cases were treated under a quality improvement consisted of extreme preterm infants with cerebral program, as well as studies of a particular palsy. Controls, which can be thought of as a exposure that requires controlled randomly selected subcohort of subjects at risk of distribution, such as drugs with serious the event of interest, were selected from among all safety concerns (e.g., isotretinoin, clozapine, infants enrolled in a long-term study of preterm natalizumab [Tysabri®]), where the infants.20 participants in the registry are identified because of their participation in a controlled With the assumptions that competing risks and distribution/risk management program. loss to followup are not associated with the exposure or the risk of disease, the case-cohort • Those who were part of a program evaluation, design allows for the selection of one control disease management effort, or quality group that can be compared with various case improvement project. series since the controls are selected at the –– An example is the evaluation of the beginning of followup. Analogous to a cohort effectiveness of evidence-based program study where every subject in the source population guidelines on improving treatment. is at risk for the disease at the start of followup, the control series in a case-cohort design 7.1 Target Population represents a sample of the exposed and unexposed Selecting patients for registries can be thought of in the source population who are disease-free at as a multistage process that begins with the start of followup. understanding the target population (the population to which the findings are meant to 7. Choosing Patients for Study apply, such as all patients with a disease or a common exposure) and then selecting a sample of The purpose of a registry is to provide information this population for study. Some registries will or describe events and patterns, and often to enroll all, or nearly all, of the target population, generate hypotheses about a specific patient but most registries will enroll only a sample of the population to whom study results are meant to target population. The accessible study population apply. Studies can be conducted of people who is that portion of the target population to which the share common characteristics, with or without the participating sites have access. The actual study inclusion of comparison groups. For example, population is the subset of those who can actually studies can be conducted of: be identified and invited and who agree to • People with a particular disease/outcome or participate.22 While it is desirable for the patients condition. (These are focused on characteristics who participate in a study to be representative of of the person.) the target population, it is rarely possible to study groups that are fully representative from a

54 Chapter 3. Registry Design

statistical sampling perspective, either for Depending on the purpose of the registry, internal, budgetary reasons or for reasons of practicality. An external, or historical groups can be used to exception is registries composed of all users of a strengthen the understanding of whether the product (as in postmarketing surveillance studies observed effects are real and in fact different from where registry participation is required as a what would have occurred under other condition of receiving an intervention), an circumstances. Comparison groups are most useful approach which is becoming more common to in registries where it is important to distinguish manage expensive interventions and/or to track between alternative decisions or to assess potential safety issues. differences, the magnitude of differences, or the Certain populations pose greater difficulties in strength of associations between groups. Registries assembling an actual study population that is truly without comparison groups can be used for representative of the target population. Children descriptive purposes, such as characterizing the and other vulnerable populations present special natural history of a disease or condition, or for challenges in recruitment, as they typically will hypothesis generation. The addition of a have more restrictions imposed by institutional comparison group may add significant complexity, review boards and other oversight groups. time, and cost to a registry. As with any research study, very clear definitions Although it may be appealing to use more than one of the inclusion and exclusion criteria are comparison group in an effort to overcome the necessary and should be well documented, limitations that may result from using a single including the rationale for these criteria. A group, multiple comparison groups pose their own common feature of registries is that they typically challenges to the interpretation of registry results. have few inclusion and exclusion criteria, which For example, the results of comparative safety and enhances their applicability to broader populations. effectiveness evaluations may differ depending on Restriction, the strategy of limiting eligibility for the comparison group used. Generally, it is entry to individuals within a certain range of preferable to make judgments about the “best” values for a confounding factor, such as age, may comparison group for study during the design be considered in order to reduce the effect of a phase and then concentrate resources on these confounding factor when it cannot otherwise be selected subjects. Alternatively, sensitivity analyses controlled, but this strategy may reduce the can be used to test inferences against alternative generalizability of results to other patients. reference groups to determine the robustness of the findings. (See Chapter 13, Section 5.) These criteria will largely be driven by the study objectives and any sampling strategy. For a more The choice of comparison groups is more complex detailed description of target populations and their in registries than in clinical trials. Whereas clinical subpopulations, and how these choices affect trials use randomization to try to achieve an equal generalizability and interpretation, see Chapter 13. distribution of known and unknown risk factors that can confound the drug-outcome association, Once the patient population has been identified, registry studies need to use various design and attention shifts to selecting the institutions and analytic strategies to control for the confounders providers from which patients will be selected. For that they have measured. The concern for more information on recruiting patients and observational studies is that people who receive a providers, see Chapter 10. new drug or device have different risk factors for adverse events than those who choose other 7.2 Comparison Groups treatments or receive no treatment at all. In other Once the target population has been selected and words, the treatment choices are often related to the mechanism for their identification (e.g., by demographic and lifestyle characteristics and the providers) is decided, the next decision involves presence of coexisting conditions that affect determining whether to collect data on clinician decisionmaking about whom to treat.23 comparators (sometimes called parallel cohorts).

55 Section I. Creating Registries

One design strategy that is used frequently to comparisons, such as for studying the effects in ensure comparability of groups is individual certain age categories or among people with matching of exposed patients and comparators similar comorbidities. However, the information with regard to key demographic factors such as value and utility of these comparisons depend age and gender. Compatibility is also achieved by largely on having adequate sample sizes within inclusion criteria that could, for example, restrict subgroups, and such analyses may need to be the registry focus to patients who have had the specified a priori to ensure that recruitment disease for a similar duration or are receiving their supports them. Internal comparisons are first drug treatment for a new condition. These particularly useful because data are collected inclusion criteria make the patient groups more during the same observation period as for all study similar but may add constraints to the external subjects, which will account for time-related validity by defining the target population more influences that may be external to the study. For narrowly. Other design techniques include example, if an important scientific article is matching study subjects on the basis of a large published that affects general clinical practice, and number of risk factors, by using statistical the publication occurs during the period in which techniques (e.g., propensity scoring) to create the study is being conducted, clinical practice may strata of patients with similar risks. As an example, change. The effects may be comparable for groups consider a recent study of a rare side effect in observed during the same period through the same coronary artery surgery for patients with acute system, whereas information from historical coronary syndrome. In this instance, the main comparisons, for example, would be expected to exposure of interest was the use of antifibrinolytic reflect different practices. agents during revascularization surgery, a practice An external comparison group is a group of that had become standard for such surgeries. The patients similar to those who are the focus of sickest patients, who were most likely to have interest, but who do not have the condition or adverse events, were much less likely to be treated exposure of interest, and for whom relevant data with antifibrinolytic agents. To address this, the that have been collected outside of the registry are investigators measured more than 200 covariates available. For example, the SEER program (by drug and outcome) per patient and used this maintains national data about cancer and has information in a propensity score analysis. The provided useful comparison information for many results of this large-scale observational study registries where cancer is an outcome of interest.25 revealed that the traditionally accepted practice External comparison groups can provide (aprotinin) was associated with serious end-organ informative benchmarks for understanding effects damage and that the less expensive generic 24 observed, as well as for assessing generalizability. medications were safe alternatives. Incorporation Additionally, large clinical and administrative of propensity scores in analysis is discussed claims databases can contribute useful information further in Chapter 13, Section 5. on comparable subjects for a relatively low cost. A An internal comparison group refers to drawback of external comparison groups is that the simultaneous data collection for patients who are data are generally not collected the same way and similar to the focus of interest (i.e., those with a the same information may not be available. The particular disease or exposure in common), but underlying populations may also be different from who do not have the condition or exposure of the registry population. In addition, plans to merge interest. For example, a registry might collect data from other databases require the proper information on patients with arthritis who are privacy safeguards to comply with legal using acetaminophen for pain control. An internal requirements for patient data; Chapter 7 covers comparison group could be arthritis patients who patient privacy rules in detail. are using other medications for pain control. Data A historical comparison group refers to patients regarding similar patients, collected during the who are similar to the focus of interest, but who do same calendar period and using the same data not have the condition or exposure of interest, and collection methods, are useful for subgroup

56 Chapter 3. Registry Design

for whom information was collected in the past impossible and a historical comparison may be (such as before the introduction of an exposure or considered: treatment or development of a condition). • When one cannot ethically continue the use of Historical controls may actually be the same older treatments or practices, or when clinicians patients who later become exposed, or they may and/or patients refuse to continue their use, so consist of a completely different group of patients. that the researcher cannot identify relevant sites For example, historical comparators are often used using the older treatments. for pregnancy studies since there is a large body of population-based surveillance data available, such • When uptake of a new medical practice has as the Metropolitan Atlanta Congenital Defects been rapid, concurrent comparisons may differ Program (MACDP).26 This design provides weak so markedly from treated patients, in regard to evidence because symmetry is not assured (i.e., the factors related to outcomes of interest, that they patients in different time periods may not be as cannot serve as valid comparison subjects due similar as desired). Historical controls are to intractable confounding. susceptible to bias by changes over time in • When conventional treatment has been uncontrollable, confounding risk factors, such as consistently unsuccessful and the effect of new differences in climate, management practices, and intervention is obvious and dramatic (e.g., first nutrition. Bias stemming from differences in use of a new product for a previously measuring procedures over time may also account untreatable condition). for observed differences. • When collecting the comparison data is too An approach related to the use of historical expensive. comparisons is the use of Objective Performance Criteria (OPC) as a comparator. This research • When the Hawthorne effect (a phenomenon method has been described as an alternative to that refers to changes in the behavior of randomized trials, particularly for the study of subjects because they know they are being devices.27 OPC are “performance criteria based on studied or observed) makes it impossible to broad sets of data from historical databases (e.g., replicate actual practice in a comparison group literature or registries) that are generally during the same period. recognized as acceptable values. These criteria • When the desired comparison is to usual care may be used for surrogate or clinical endpoints in or “expected” outcomes at a population level, demonstrating the safety or effectiveness of a and data collection is too expensive due to the device.”28 A U.S. Food and Drug Administration distribution or size of that population. guidance document on medical devices includes a description of study designs that should be 8. Sampling considered as alternatives to randomized clinical trials and that may meet the statutory criteria for Various sampling strategies for patients and sites preapproval as well as postapproval evidence.28 can be considered. Each of these has tradeoffs in Registries serve as a source of reliable historical terms of validity and information yield. The data in this context. New registries with safety or representativeness of the sample, with regard to effectiveness endpoints may also be planned that the range of characteristics that are reflective of the will incorporate previously existing OPC as broader target population, is often a consideration, comparators (e.g., for a safety endpoint for a new but representativeness mainly affects cardiac device). Such registries might use prior generalizability rather than the internal validity of clinical study data to set the “complication-free the results. Representativeness should be rate” for comparison. considered in terms of patients (e.g., men and There are several situations in which conventional women, children, the elderly, different racial or prospective design for comparison selection is ethnic groups) and sites (academic medical centers, community practices). For sites (health

57 Section I. Creating Registries

care providers, hospitals, etc.), representativeness –– Cluster (area) sampling: The population is is often considered in terms of geography, practice divided into clusters, these clusters are size, and academic or private practice type. randomly sampled, and then some or all Reviewing and refining the research question can patients within selected clusters are help researchers define an appropriate target sampled. This technique is particularly population and a realistic strategy for subject useful in large geographic areas or when selection. cluster-level interventions are being studied. To ensure that enough meaningful information will –– Multistage sampling: Multistage sampling be available for analysis, registry studies often can include any combination of the restrict eligibility for entry to individuals within a sampling techniques described above. certain range of characteristics. Alternatively, they • Nonprobability sampling: Selection is may use some form of sampling: random selection, systematic or haphazard but not random. The systematic sampling, or a nonrandom approach. following sampling strategies affect the type of Often-used sampling strategies include the inferences that can be drawn; for example, it following: would be preferable to have a random sample if • Probability sampling: Some form of random the goal were to estimate the prevalence of a selection is used, wherein each person in the condition in a population. However, systematic population must have a known (often equal) sampling of “typical” patients can generate probability of being selected.29-32 useful data for many purposes, and is often used in situations where probability sampling is –– Census: A census sample includes every 33 individual in a population or group (e.g., all not feasible. known cases). A census is not feasible when –– Case series or consecutive (quota) sampling: the group is large relative to the costs of All consecutive eligible patients treated at a obtaining information from individuals. given practice or by a given clinician are –– Simple random sampling: The sample is enrolled until the enrollment target is selected in such a way that each person has reached. This approach is intended to reduce the same probability of being sampled. conscious or unconscious selection bias on the part of clinicians as to whom to enroll in –– Stratified random sampling: The group from the study, especially with regard to factors which the sample is to be taken is first that may be related to prognosis. stratified into subgroups on the basis of an important, related characteristic (e.g., age, –– Haphazard, convenience, volunteer, or parity, weight) so that each individual in a judgmental sampling: This includes any subgroup has the same probability of being sampling not involving a truly random included in the sample, but the probabilities mechanism. A hallmark of this form of for different subgroups or strata are sampling is that the probability that a given different. Stratified random sampling individual will be in the sample is unknown ensures that the different categories of before sampling. The theoretical basis for characteristics that are the basis of the strata is lost, and the result is are sufficiently represented in the sample. inevitably biased in unknown ways. However, the resulting data must be –– Modal instance: The most typical subject is analyzed using more complicated statistical sampled. procedures (such as Mantel-Haenszel) in –– Purposive: Several predefined groups are which the stratification is taken into deliberately sampled. account. –– Expert: A panel of experts judges the –– Systematic sampling: Every nth person in a representativeness of the sample or is the population is sampled. source that contributes subjects to a registry.

58 Chapter 3. Registry Design

Individual matching of cases and controls is desired power) and by time-related considerations. sometimes used as a sampling strategy for The induction period for some outcomes of controls. Controls are matched with individual interest must be considered, and sufficient cases who have similar confounding factors, such followup time allowed for the exposure under as age, to reduce the effect of the confounding study to have induced or promoted the outcome. factors on the association being investigated. Biological models of disease etiology and Patients may be recruited in a fashion that allows causation usually indicate the required time period for individual matching. For example, if a 69-year- of observation for an effect to become apparent. old “case” participates in the registry, a control Calendar time may be a consideration in studies of near in age will be sought. Individual matching for changes in clinical practice or interventions that prospective recruitment is challenging and not have a clear beginning and end. The need for customarily used. More often, matching is used to evidence to inform policy may also determine a create subgroups for supplemental data collection timeframe within which the evidence must be for case-control studies and cohort studies when made available to decisionmakers. subjects are limited and/or stratification is unlikely A detailed discussion of the topic of sample size to provide enough subjects in each stratum for calculations for registries is provided in Appendix meaningful evaluation. A. For present purposes it is sufficient to briefly A number of other sampling strategies have arisen describe some of the critical inputs to these from research (e.g., snowball, calculations that must be provided by the registry heterogeneity), but they are of less relevance to developers: registries. • The expected timeframe of the registry and the time intervals at which analyses of registry data 9. Registry Size and Duration will be performed. • Either the size of clinically important effects Precision in measurement and estimation (e.g., minimum clinically important corresponds to the reduction of random error; it differences) or the desired precision associated can be improved by increasing the size of the study with registry-based estimates. and modifying the design of the study to increase the efficiency with which information is obtained • Whether or not the registry is intended to from a given number of subjects.29 support regulatory decisionmaking. If the results from the registry will affect regulatory During the registry design stage, it is critical to action—for example, the likelihood that a explicitly state how large the registry will be, how product may be pulled from the market—then long patients should be followed, and what the the precision of the overall risk estimate is justifications are for these decisions. These important, as is the necessity to predict and decisions are based on the overall purpose of the account for attrition. registry. For example, in addressing specific questions of product safety or effectiveness, the In a classical calculation of sample size, the desired level of precision to confirm or rule out the crucial inputs that must be provided by the existence of an important effect should be investigators include either the size of clinically specified, and ideally should be linked to policy or important effects or their required precision. For practice decisions that will be made based on the example, suppose that the primary goal of the evidence. For registries with aims that are registry is to compare surgical complication rates descriptive or hypothesis generating, study size in general practice with those in randomized trials. may be arrived at through other considerations. The inputs to the power calculations would include the complication rates from the randomized trials The duration of registry enrollment and followup (e.g., 4 percent) and the complication rate in should be determined both by required sample size general practice, which would reflect a meaningful (number of patients or person-years to achieve the

59 Section I. Creating Registries

departure from this rate (e.g., 6 percent). If, on the intervention. To the extent they are not other hand, the goal of the registry is simply to independent, a measure of interdependence, the track complication rates (and not to compare the intraclass correlation (ICC), and so-called “design registry with an external standard), then the effect” must be considered in generating the investigators should specify the required width of overall sample size calculation. A reference the associated with those rates. addressing sample size considerations for a study For example, in a large registry, the 95-percent incorporating a cluster-randomized intervention is confidence interval for a 5-percent complication provided.34 A hierarchical or multilevel analysis rate might extend from 4.5 percent to 5.5 percent. may be required to account for one or more levels If all of the points in this confidence interval lead of “grouping” of individual patients, discussed to the same decision, then an interval of ±0.5 further in Chapter 13, Section 5. One approach to percent is considered sufficiently precise, and this addressing multiple comparisons in the surgical is the input required for the estimation of sample complication rate example above is to use control size. chart methodology, a statistical approach used in Specifying the above inputs to sample size process measurement to examine the observed calculations is a substantial matter and usually variability and determine whether out-of-control involves a combination of quantitative and conditions are occurring. qualitative reasoning. The issues involved in methodology is also used in sample size making this specification are essentially similar for estimation, largely for studies with repeated registries and other study designs, though for measurements, to adjust the sample size as needed registries designed to address multiple questions of and therefore maintain reasonably precise interest, one or more primary objectives or estimates of confidence limits around the point endpoints must be selected that will drive the estimate. Accordingly, for registries that involve selection of a minimum sample size to meet those ongoing evaluation, sample size per time interval objectives. could be determined by the precision associated with the related confidence interval, and decision Other considerations that should sometimes be rules for identifying problems could then be based taken into account when estimating sample sizes on control chart methodology. include— Although most of the emphasis in estimating study • whether individual patients can be considered size requirements is focused on patients, it is “independent,” or whether they share factors equally important to consider the number of sites that would lead to correlation in measures needed to recruit and retain enough patients to between them; achieve a reasonably informative number of • whether multiple comparisons are being made person-years for analysis. Many factors are and subjected to statistical testing; and involved in estimating the number of sites needed for a given study, including the number of eligible • whether levels of expected attrition or lack of patients seen in a given practice during the relevant adherence to therapy may require a larger time period, desired representativeness of sites number of patients to achieve the desired with regard to geography, practice size, or other number of person-years of followup or features, and the timeframe within which study exposure. results are required, which may also limit the In some cases, patients under study who share timeframe for patient recruitment. some group characteristics, such as patients treated In summary, the aims of a registry, the desired by the same clinician or practice, or at the same precision of information sought, and the institution, may not be entirely independent from hypotheses to be tested, if any, determine the one another with regard to some outcomes of process and inputs for arriving at a target sample interest or when studying a practice-level size and specifying the duration of followup.

60 Chapter 3. Registry Design

Registries with mainly descriptive aims, or those minimize a real difference—generating a that provide quality metrics for clinicians or distortion known as “bias toward the null”—by medical centers, may not require the choice of a including the experience of people who did not target sample size to be arrived at through power adhere to the recommended study product along calculations. In either case, the costs of obtaining with those who did. study data, in monetary terms and in terms of Another principal difference between registries researcher, clinician, and patient time and effort, and RCTs is that RCTs are often focused on a may set upper as well as lower limits on study size. relatively homogeneous pool of patients from Limits to study budgets and the number of sites which significant numbers of patients are and patients that could be recruited may be purposefully excluded at the cost of external apparent at the outset of the study. However, an validity—that is, generalizability to the target underpowered study involving substantial data population of disease sufferers. Registries, in collection that is ultimately unable to satisfactorily contrast, usually focus on generalizability so that answer the research question(s) may prove to be a their population will be representative and relevant waste of finite monetary as well as human to decisionmakers. resources that could better be applied elsewhere. 10.1 Generalizability 10. Internal and External The strong external validity of registries is Validity achieved by the fact that they include typical patients, which often include more heterogeneous The potential for bias refers to opportunities for populations than those participating in RCTs (e.g., systematic errors to influence the results. Internal wide variety of age, ethnicity, and comorbidities). validity is the extent to which study results are free Therefore, registry data can provide a good from bias, and the reported association between description of the course of disease and impact of exposure and outcome is not due to unmeasured or interventions in actual practice and, for some uncontrolled-for variables. Generalizability, also purposes, may be more relevant for known as external validity, is a concept that refers decisionmaking than the data derived from the to the utility of the inferences for the broader artificial constructs of the clinical trial. In fact, population that the study subjects are intended to even though registries have more opportunities to represent. In considering potential biases and introduce bias (systematic error) because of their generalizability, we discuss the differences nonexperimental methodology, well designed between RCTs and registries, since these are the observational studies can approximate the effects two principal approaches to conducting clinically of interventions observed in RCTs on the same relevant prospective research. topic35, 36 and, in particular, in the evaluation of The strong internal validity that earns RCTs high health care effectiveness in many instances.37 grades for evidence comes largely from the The choice of groups from which patients will be randomization of exposures that helps ensure that selected directly affects generalizability. No the groups receiving the different treatments are particular method will ensure that an approach to similar in all measured or unmeasured patient recruitment is adequate, but it is characteristics, and that, therefore, any differences worthwhile to note that the way in which patients in outcome (beyond those attributable to chance) are recruited, classified, and followed can either can be reasonably attributed to differences in the enhance or diminish the external validity of a efficacy or safety of the treatments. However, it is registry. Some examples of how these methods of worth noting that RCTs are not without their own patient recruitment and followup can lead to biases, as illustrated by the “intent-to-treat” systematic error follow. analytic approach, in which people are considered to have used the assigned treatment, regardless of actual compliance. The intent-to-treat analyses can

61 Section I. Creating Registries

10.2 Information Bias study that is different from the estimate that is obtainable from the target population.38 Selection If the registry’s principal goal is the estimation of bias may be introduced if certain subgroups of risk, it is possible that adverse events or the patients are routinely included or excluded from number of patients experiencing them will be the registry. underreported if the reporter will be viewed negatively for reporting them. It is also possible 10.4 Channeling Bias (Confounding by for those collecting data to introduce bias by Indication) misreporting the outcome of an intervention if they have a vested interest in doing so. This type of Channeling bias, also called confounding by bias is referred to as information bias (also called indication, is a form of selection bias in which detection, observer, ascertainment, or assessment drugs with similar therapeutic indications are bias), and it addresses the extent to which the data prescribed to groups of patients with prognostic that are collected are valid (represent what they are differences.39 For example, physicians may intended to represent) and accurate. This bias prescribe new treatments more often to those arises if the outcome assessment can be interfered patients who have failed on traditional first-line with, intentionally or unintentionally. On the other treatments. hand, if the outcome is objective, such as whether One approach to designing studies to address or not a patient died or the results of a lab test, channeling bias is to conduct a prospective review then the data are unlikely to be biased. of cases, in which external reviewers are blinded as to the treatments that were employed and are asked 10.3 Selection Bias to determine whether a particular type of therapy A registry may create the incentive to enroll only is indicated and to rate the overall prognosis for patients who either are at low risk of complications the patient.40 This method of blinded prospective or who are known not to have suffered such review was developed to support research on complications, biasing the results of the registry ruptured cerebral aneurysms, a rare and serious toward lower event rates. Those registries whose situation. The results of the blinded review were participants derive some sort of benefit from used to create risk strata for analysis so that reporting low complication rates, for example, comparisons could be conducted only for those with surgeons participating are at candidates for whom both therapies under study particularly high risk for this type of bias. Another were indicated, a procedure much like the example of how patient selection methods can lead application of additional inclusion and exclusion to bias is the use of patient volunteers, a practice criteria in a clinical trial. that may lead to selective participation from A computed “propensity score” (i.e., the predicted subjects most likely to perceive a benefit, probability of use of one therapy over another distorting results for studies of patient-reported based on medical history, health care utilization, outcomes. and other characteristics measured prior to the Enrolling patients who share a common exposure initiation of therapy) is increasingly incorporated history, such as having used a drug that has been into study designs to address this type of publicly linked to a serious adverse effect, could confounding.41, 42 Propensity scores may be used distort effect estimates for cohort and case-control to create cohorts of initiators of two different analyses. Registries can also selectively enroll treatments matched with respect to probability of people who are at higher risk of developing serious use of one of the two therapies, for stratification or side effects, since having a high-risk profile can for inclusion as a covariate in a multivariate motivate a patient to participate in a registry. analysis. Studies incorporating propensity scores as part of their design may be planned prior to and The term selection bias refers to situations where implemented shortly following launch of a new the procedures used to select study subjects lead to drug as part of a risk management program, with an effect estimate among those participating in the matched comparators being selected over time, so

62 Chapter 3. Registry Design

that differences in prescribing patterns following those who have tolerated or benefited from the drug launch may be taken into account.43 intervention, and would not necessarily capture the Instrumental variables, or factors strongly full spectrum of experience and outcomes. associated with treatment but related to outcome Selecting only existing users may introduce any only through their association with treatment, may number of biases, including incidence/prevalence provide additional means of adjustment for bias, survivorship bias, and followup bias. By confounding by indication, as well as unmeasured enrolling new users (an inception or incidence confounding.44 Types of instrumental variables cohort), a study ensures that the longitudinal include providers’ preferences for one therapy over experience of all users will be captured, and that the ascertainment of their experience and another—a variable which exploits variation in 48 practice as a type of ; variation outcomes will be comparable. or changes in insurance coverage or economic 10.6 Loss to Followup factors (e.g., cigarette taxes) associated with an exposure; or geographic distance from a specific Loss to followup or attrition of patients and sites type of service.45, 46 Variables that serve as threatens generalizability as well as internal effective instruments of this nature are not always validity if there is differential loss; for example, available and may be difficult to identify. While loss of participants with a particular exposure or use of clinician or study site may, in some specific disease, or with particular outcomes. Loss to cases, offer potential as an instrumental variable followup and attrition are generally a serious for analysis, the requirement that use of one concern only when they are nonrandom (that is, therapy over another be very strongly associated when there are systematic differences between with the instrument is often difficult to meet in those who leave or are lost and those who remain). real-world settings. In most cases, instrumental The magnitude of loss to followup or attrition variable analysis provides an alternative for determines the potential impact of any bias. Given secondary analysis of study data. Instrumental that the differences between patients who remain variable analysis may either support the enrolled and those who are lost to followup are conclusions drawn on the basis of the initial often unknown (unmeasurable), preventing loss to analysis, or it may raise additional questions followup in long-term studies to the fullest extent regarding the potential impact of confounding by possible will increase the credibility and validity indication.42 of the results.49 Attrition should be considered with regard to both patients and study sites, as In some cases, however, differences in disease results may be biased or less generalizable if only severity or prognosis between patients receiving some sites (e.g., teaching hospitals) remain in the one therapy rather than another may be so extreme study while others discontinue participation. and/or unmeasurable that confounding by indication is not remediable in an observational 10.7 Assessing the Magnitude of Bias design.47 This represents special challenges for observational studies of comparative effectiveness, Remaining alert for any source of bias is as the severity of underlying illness may be a important, and the value of a registry is enhanced strong determinant of both choice of treatment and by its ability to provide a formal assessment of the treatment outcome. likely magnitude of all potential sources of bias. Any information that can be generated regarding 10.5 Bias from Study of Existing nonrespondents, missing respondents, and the like, Rather Than New Product Users is helpful, even if it is just an estimation of their raw numbers. As with many types of survey If there is any potential for tolerance to affect the research, an assessment of differential response use of a product, such that only those who perceive rates and patient selection can sometimes be benefit from it or are free from harm continue undertaken when key data elements are available using it, the recruitment of existing users rather for both registry enrollees and nonparticipants. than new users may lead to the inclusion of only Such analyses can easily be undertaken when the

63 Section I. Creating Registries

initial data source or population pool is that of a designed for the purpose of assessing adherence to health care organization, employer, or practice that lipid screening guidelines requires that its sites has access to data in addition to key selection have a sophisticated electronic medical record in criteria (e.g., demographic data or data on order to collect data, it will probably report better comorbidities). Another tool is the use of adherence than usual practice because this same sequential screening logs, in which all subjects electronic medical record facilitates the generation fitting the inclusion criteria are enumerated and a of real-time reminders to engage in screening. In few key data elements are recorded for all those this case, a report of rates of adherence to other who are screened. This technique allows some screening guidelines (for which there were no quantitative analysis of nonparticipants and reminders), even if these are outside the direct assessments of the effects, if any, on scope of inquiry, would provide some insight into representativeness. Whenever possible, the degree of overestimation. quantitative assessment of the likely impact of bias Finally, and most importantly, whether or not study is desirable to determine the sensitivity of the subjects need to be evaluated on their findings to varying assumptions. A text on representativeness depends on the purpose and quantitative analysis of bias through validation kind of inference needed. For example, sampling studies, and on probabilistic approaches to data in proportion to the underlying distribution in the analysis, provides a guide for planning and 50 population is not necessary to understand implementing these methods. biological effects. However, if the study purpose Qualitative assessments, although not as rigorous were to estimate a rate of occurrence of a as quantitative approaches, may give users of the particular event, then sampling would be necessary research a framework for drawing their own to reflect the appropriate underlying distributions. conclusions regarding the effects of bias on study results if the basis for the assessment is made 11. Summary explicit in reporting the results. Accordingly, two items that can be reported to help In summary, the key points to consider in the user assess the generalizability of research designing a registry include study design, data results based on registry data are a description of sources, patient selection, comparison groups, the criteria used to select the registry sites, and the sampling strategies, and considerations of possible characteristics of these sites, particularly those sources of bias and ways to address them, to the characteristics that might have an impact on the extent that is practical and achievable. purpose of the registry. For example, if a registry

64 Chapter 3. Registry Design

Case Examples for Chapter 3

Case Example 2. Designing a registry for a scars than the Nuss procedure. Surgeons also tend health technology assessment to perform either only the Nuss procedure or only Description The Nuss procedure registry another procedure, a factor that would complicate was a short-term registry randomization efforts. In addition, only a small designed specifically for the number of procedures are done in the United health technology assessment of Kingdom. The sample for a randomized trial the Nuss procedure, a novel, would likely be very small, making it difficult to minimally invasive procedure detect rare adverse events. for the repair of pectus Due to these limitations, NICE decided to excavatum, a congenital develop a short-term registry to gather evidence malformation of the chest. The on the Nuss procedure. The advantages of a registry collected procedure registry were its ability to gather data on all outcomes, patient-reported patients undergoing the procedure in the United outcomes, and safety outcomes. Kingdom to provide a more complete safety Sponsor National Institute for Health and assessment, and its ability to collect patient- Clinical Excellence (NICE), reported outcomes. United Kingdom The registry was developed by an academic Year Started 2004 partner, with input from clinicians. Hospitals performing the procedure were identified and Year Ended 2007 asked to enter into the registry data on all patients No. of Sites 13 hospitals undergoing the intervention. Once the registry was underway, the cases in the registry were No. of Patients 260 compared against cases included in the Hospital Challenge Episodes Statistics (HES) database, a nationwide source of routine data on hospital activity, and The Nuss procedure is a minimally invasive nonparticipating hospitals were identified and intervention for the repair of pectus excavatum. prompted to enter their data. During a review of the evidence supporting this procedure conducted in 2003, the National Results Institute for Health and Clinical Excellence NICE conducted a reassessment of the Nuss (NICE) determined that the existing data included procedure in 2009, comparing data from the relatively few patients and few quality of life registry with other published evidence on safety outcomes, and did not sufficiently address safety and efficacy. The quantity of published literature concerns. NICE concluded in the 2003 review had increased substantially between 2003 and that the evidence was not adequate for routine use 2009. The new publications primarily focused on and that more evidence was needed to make a technical and safety outcomes, while the registry complete assessment of the procedure. included patient-reported outcomes. The Proposed Solution literature and the registry reported similar rates of major adverse events such as bar displacement Gathering additional evidence through a (from 2 to 10 percent). Based on the registry data randomized controlled trial was not feasible for and the new literature, the review committee several reasons. First, a blinded trial would be found that the evidence was now sufficient to difficult because the other procedures for the support routine use of the Nuss procedure, and no repair of pectus excavatum produce much larger

65 Section I. Creating Registries

Case Example 2. Designing a registry for a Key Point health technology assessment (continued) The Nuss registry demonstrated that a small, Results (continued) short-term, focused registry with recommended further review of the guidance is planned. (but not automatic or mandatory) submission can Committee members considered that the registry produce useful data, both about safety and about made a useful contribution to guidance patient-reported outcomes. development.

Case Example 3. Developing prospective Proposed Solution nested studies in existing registries In collaboration with Genentech, the CORRONA Description The Consortium of investigators are utilizing the registry in two Rheumatology Researchers of separate prospective, nested substudies: the North America (CORRONA) Comparative Effectiveness Registry to study is a national disease registry Therapies for Arthritis and Inflammatory of patients with rheumatoid CoNditions (CERTAIN) and the Treat to Target arthritis (RA) and psoriatic (T2T) study. Based on the study eligibility criteria arthritis (PsA). and the capabilities of CORRONA sites, different Sponsor CORRONA Investigators and patients and sites are being selected to participate Genentech in CERTAIN and T2T. Year Started 2001 The CERTAIN study is a nested comparative effectiveness and safety study evaluating real- Year Ended Ongoing world differences in classes of biologic agents No. of Sites Over 100 sites in the United among RA patients initiating either tumor States necrosis factor (TNF) antagonists or non-TNF- inhibitor biologic agents. The study is enrolling No. of Patients As of March 31 2012: 36,922 approximately 2,750 patients over three years to (31,701 RA patients and 5,221 address comparative effectiveness questions. PsA patients) Long-term safety followup data will be collected Challenge through lifelong patient participation in the CORRONA registry after CERTAIN study In 2001, the CORRONA data collection program completion. Data are collected at mandated was established to collect longitudinal, physician- 3-month intervals and include standard validated and patient-reported safety and effectiveness data physician- and patient-derived outcomes and for the treatment and management of RA and centrally-processed laboratory measures such as PsA. Any patient with RA or PsA upon diagnosis complete blood counts, metabolic panel, high can participate in the registry, and participation in sensitivity CRP, lipids with direct (nonfasting) the registry is lifelong unless the patient LDL, immunoglobulin levels and serology (CCP withdraws consent. With an existing and RF). Serum, plasma, DNA and RNA will be infrastructure and its representative, real-world stored for future research. In addition, adverse nature, the disease registry can be used as a event data are being obtained with inclusion of robust opportunity for nested trials at sites that primary “source” documents, followed by a have been trained in data collection and robust process of verification and adjudication. verification.

66 Chapter 3. Registry Design

Case Example 3. Developing prospective The two nested substudies require additional nested studies in existing registries (continued) patient consent and site reimbursement, as they Proposed Solution (continued) collect blood samples that increase the time required to complete a study visit. CORRONA The T2T study is a cluster-randomized, open- collaborates with an academic institution to label study comparing treatment acceleration collect personal identifiers and patient consent to (i.e., monthly visits with a change in therapeutic release medical records, thereby facilitating agent, dosage or route of administration in order verification of serious adverse event for patients to achieve a target metric of disease activity) participating in CERTAIN. While this feature against usual care (i.e., no mandated changes to adds value to CERTAIN’s ability to address therapy or visit frequencies beyond what the long-term safety questions, it entailed treating physician considers appropriate for the establishing a new mechanism to ensure that the patient). This study will attempt to determine two databases (CORRONA and a database for both the feasibility and outcomes of treating to personal identifiers) remain separate from each target in a large U.S. population. This one-year other in a highly secure way. New enrollment and study is enrolling 888 patients. Data collection screening instructions were developed for each includes standard measures of disease activity substudy, with mandated completion of required such as Clinical-Disease Activity Index (CDAI) training for participating physicians and research score, Disease Activity Score-28 (DAS28), and coordinators. Routine Assessment of Patient Index Data-3 (RAPID 3), as well as rates of acceleration, Key Point frequency of visits, and suspected RA-drug- Designing a prospective, nested study within an related toxicities. The purpose of the trial is to established disease registry has many benefits: the test the hypothesis that accelerated aggressive study leverages existing infrastructure, patient therapy of RA correlates with better long-term and site staff are familiar with the registry, and patient outcomes. site relationships are already in place. Substudies Results need to be well planned and address a compelling clinical issue. Registry personnel must provide The CERTAIN and T2T studies, now in the sufficient guidance, instructions, and rationale to enrollment phase, exemplify the key advantages sites to ensure that the transition is smooth and and the unique operational synergies of that the distinction from core registry operations successfully nesting studies within an existing is maintained in order to achieve the goal of disease registry. This design approach has the high-quality research. advantage of minimizing the usual study start-up and implementation challenges. The registry For More Information allows real-time identification of eligible patients Kremer J. The CORRONA database. Ann Rheum typically seen in a U.S. clinical practice, a Dis. 2005 Nov;64 Suppl 4:iv37-41. capability that can facilitate patient recruitment. The Consortium of Rheumatology Researchers of Both CERTAIN and T2T have broad inclusion North America, Inc. (CORRONA). criteria to increase representativeness of the http://www.corrona.org. population enrolled. Established registry sites include investigators, staff, and patients already US National Institutes of Health, ClinicalTrials. experienced with the registry questionnaires and gov. http://www.clinicaltrials.gov/ct2/show/ research activities. NCT01407419?term=CORRONA& rank=2.

67 Section I. Creating Registries

Case Example 4. Designing a registry to provider. In addition, because of the often address unique patient enrollment challenges unsettling and traumatic nature of their Description The Anesthesia Awareness experience, even patients who recognize their Registry is a survey-based anesthesia awareness before being discharged registry that collects detailed from the hospital may not feel comfortable data about patient experiences reporting it to their surgeon or other health care of anesthesia awareness. Patient providers. medical records are used With ongoing coverage in the media, to assess anesthetic factors anesthesiologists were facing increasing concern associated with the patient’s and fear about anesthesia awareness among their experience. An optional set of patients. The American Society of psychological assessment Anesthesiologists sought a patient-oriented instruments measure potential approach to this problem. trauma-related sequelae Proposed Solution including depression and post- traumatic stress disorder (PTSD). Because this population of patients is not always immediately recognized in the health care setting, Sponsor American Society of the registry was created to collect case reports of Anesthesiologists anesthesia awareness directly from patients. A Year Started 2007 patient advocate was invited to consult in the Year Ended Ongoing registry’s development and provides ongoing advice from the patient perspective. The registry No. of Sites Not applicable hosts a Web site that provides information about No. of Patients 265 anesthesia awareness and directions for enrolling in the registry. Any patient who believes they Challenge have experienced anesthesia awareness may Anesthesia awareness is a recognized voluntarily submit a survey and medical records complication of general anesthesia, defined as the to the registry. Psychological assessments are unintended experience and explicit recall of optional. An optional open-ended discussion events during surgery. The incidence of about the patient’s anesthesia awareness anesthesia awareness has been estimated at 1–2 experience provides patients with an opportunity patients per 1,000 anesthetics and may result in to share information that may not be elicited development of serious and long-term through the survey. psychological sequelae including PTSD. The Results causes of the phenomenon and preventive strategies have been studied, but there is The registry has enrolled 265 patients since 2007. disagreement in the scientific community about Patients who enroll are self-selected, and the the effectiveness of monitoring devices for sample is likely biased towards patients with prevention of anesthesia awareness. emotional sequelae. While the information provided to potential enrollees clearly states that The population of patients experiencing eligibility is restricted to awareness during anesthesia awareness is difficult to identify. general anesthesia, a surprising number of Although standard short questionnaires designed enrollments are patients who were supposed to be to identify anesthesia awareness are sometimes awake during regional anesthesia or sedation. administered to patients postoperatively, many This revealed a different side to the problem of patients experience delayed recollection and do anesthesia awareness: clearly, some patients did not realize that they were awake during their not understand the nature of the anesthetic that procedure until several weeks later. These patients would be provided for their procedure, or patients may or may not report their experience to their had expectations that were not met by their

68 Chapter 3. Registry Design

Case Example 4. Designing a registry to Domino, KB. Update on the Anesthesia address unique patient enrollment challenges Awareness Registry. ASA Newsletter 72(11): 32, (continued) 36, 2008. Results (continued) Kent CD, Bruchas RR, Posner KL, et al. anesthesia providers. Most enrollees experienced Anesthesia Awareness Registry Update. long-term psychological sequelae regardless of Anesthesiology. 2009;111:A1518. anesthetic technique. Kent CD. Awareness during general anesthesia: Key Point ASA Closed Claims Database and Anesthesia Awareness Registry. ASA Newsletter. 2010; Allowing the registry’s purpose to drive its design 74(2): 14-16. produces a registry that is responsive to the expected patient population. Employing direct-to- Kent CD, Metzger NA, Posner KL, et al. patient recruitment can be an effective way of Anesthesia Awareness Registry: psychological reaching a patient population that otherwise impacts for patients. Anesthesiology. 2011; would not be enrolled in the registry, and can A003. yield surprising and important insights into Domino KB, Metzger NA, Mashour GA. patient experience. Anesthesia Awareness Registry: patient responses For More Information to awareness. Br J Anaesth. 2012;108(2):338P. http://www.awaredb.org Domino, KB. Committee on Professional Liability opens anesthesia awareness registry. ASA Newsletter. 2007;71(3): 29, 34.

References for Chapter 3 6. Norén GN, Hopstadius J, Bate A, et al. Temporal pattern discovery in longitudinal electronic patient 1. Strom BL. Pharmacoepidemiology. 3rd ed. records. Data Min Knowl Discov. Chichester, England: John Wiley; 2000. 2010;20(3):361-87. 2. Gliklich R. A New Framework For Comprehensive 7. Travis LB, Rabkin CS, Brown LM, et al. Cancer Evidence Development. IN VIVO. Oct 22, 2012. survivorship—genetic susceptibility and second http://www.elsevierbi.com/publications/in- primary cancers: research strategies and vivo/30/9/a-new-framework-for-comprehensive- recommendations. J Natl Cancer Inst. 2006 Jan evidence-development. Accessed June 17, 2013. 4;98(1):15-25. PMID: 16391368. 3. Chow SC, Chang M, Pong A. Statistical 8. Sackett DL, Haynes RB, Tugwell P. Clinical consideration of adaptive methods in clinical epidemiology. Boston: Little, Brown and development. J Biopharm Stat. 2005;15(4):575-91. Company; 1985. p. 228. PMID: 16022164. 9. Hennekens CH, Buring JE. Epidemiology in 4. Farrington CP. Relative incidence estimation from medicine. 1st ed. Boston: Little, Brown and case series for vaccine safety evaluation. Company; 1987. Biometrics. 1995 Mar;51(1):228-35. PMID: 7766778. 10. Schoebel FC, Gradaus F, Ivens K, et al. Restenosis after elective coronary balloon angioplasty in 5. Ryan PB, Powell GE, Pattishall EN, et al. patients with end stage renal disease: a case- Performance of screening multiple observational control study using quantitative coronary databases for active drug safety surveillance. angiography. Heart. 1997 Oct;78(4):337-42. Providence, RI: International Society of PMID: 9404246. PMCID: 1892250. Pharmacoepidemiology; 2009.

69 Section I. Creating Registries

11. Rothman K, Greenland S. Modern Epidemiology. 23. Hunter D. First, gather the data. N Engl J Med. 3rd ed. Philadelphia: Lippincott Williams & 2006 Jan 26;354(4):329-31. PMID: 16436764. Wilkins; 1998. p. 175–9. 24. Mangano DT, Tudor IC, Dietzel C, et al. The risk 12. Oral contraceptive use and the risk of endometrial associated with aprotinin in cardiac surgery. cancer. The Centers for Disease Control Cancer N Engl J Med. 2006 Jan 26;354(4):353-65. and Steroid Hormone Study. JAMA. 1983 Mar PMID: 16436767. 25;249(12):1600-4. PMID: 6338265. 25. National Cancer Institute. Surveillance 13. Oral contraceptive use and the risk of ovarian Epidemiology and End Results. http://seer.cancer. cancer. The Centers for Disease Control Cancer gov. Accessed August 27, 2012. and Steroid Hormone Study. JAMA. 1983 Mar 26. Metropolitan Atlanta Congenital Defects Program 25;249(12):1596-9. PMID: 6338264. (MACDP). National Center on Birth Defects and 14. Long-term oral contraceptive use and the risk of Developmental Disabilities. Centers for Disease breast cancer. The Centers for Disease Control Control and Prevention. http://www.cdc.gov/ Cancer and Steroid Hormone Study. JAMA. 1983 ncbddd/birthdefects/MACDP.html. Accessed Mar 25;249(12):1591-5. PMID: 6338262. August 27, 2012. 15. Speck CE, Kukull WA, Brenner DE, et al. History 27. Chen E, Sapirstein W, Ahn C, et al. FDA of depression as a risk factor for Alzheimer’s perspective on clinical trial design for disease. Epidemiology. 1995 Jul;6(4):366-9. cardiovascular devices. Ann Thorac Surg. 2006 PMID: 7548342. Sep;82(3):773-5. PMID: 16928481. 16. Wacholder S, McLaughlin JK, Silverman DT, et 28. U.S. Food and Drug Administration, Center for al. Selection of controls in case-control studies. I. Devices and Radiological Health. The Least Principles. Am J Epidemiol. 1992 May Burdensome Provisions of the FDA 1;135(9):1019-28. PMID: 1595688. Modernization Act of 1997; Concept and Principles: Final Guidance for FDA and Industry. 17. Wacholder S, Silverman DT, McLaughlin JK, et Document issued October 4, 2002. http://www. al. Selection of controls in case-control studies. II. fda.gov/MedicalDevices/ Types of controls. Am J Epidemiol. 1992 May DeviceRegulationandGuidance/ 1;135(9):1029-41. PMID: 1595689. GuidanceDocuments/ucm085994.htm#h3. 18. Wacholder S, Silverman DT, McLaughlin JK, et Accessed August 14, 2012. al. Selection of controls in case-control studies. 29. Cochran WG. Sampling Techniques. 3rd ed. New III. Design options. Am J Epidemiol. 1992 May York: John Wiley & Sons; 1977. 1;135(9):1042-50. PMID: 1595690. 30. Lohr SL. Sampling: Design and Analysis. Boston: 19. Rothman K, Greenland S. Modern Epidemiology. Duxbury; 1999. 3rd ed. Philadelphia: Lippincott Williams & Wilkins; 1998. p. 108. 31. Sudman S. Applied Sampling. New York: Academic Press; 1976. 20. Vigneswaran R, Aitchison SJ, McDonald HM, et al. Cerebral palsy and placental infection: a 32. Henry GT. Practical Sampling. Newbury Park, case-cohort study. BMC Pregnancy Childbirth. CA: Sage; 1990. 2004 Jan 27;4(1):1. PMID: 15005809. 33. Rothman K, Greenland S. Modern Epidemiology. PMCID: 343280. 3rd ed. Philadelphia: Lippincott Williams & 21. Ong AT, Daemen J, van Hout BA, et al. Cost- Wilkins; 1998. p. 116. effectiveness of the unrestricted use of sirolimus- 34. Raudenbush SW. Statistical analysis and optimal eluting stents vs. bare metal stents at 1 and 2-year design for cluster randomized trials. Psychological follow-up: results from the RESEARCH Registry. Methods. 1997;2(2):173-85. Eur Heart J. 2006 Dec;27(24):2996-3003. PMID: 17114234. 35. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the 22. Hulley SB, Cumming SR. Designing clinical hierarchy of research designs. N Engl J Med. 2000 research. Baltimore: Williams & Wilkins; 1988. Jun 22;342(25):1887-92. PMID: 10861325. PMCID: 1557642.

70 Chapter 3. Registry Design

36. Benson K, Hartz AJ. A comparison of 44. Brookhart, M. Alan. Instrumental Variables for observational studies and randomized, controlled Comparative Effectiveness Research: A Review of trials. N Engl J Med. 2000 Jun 22;342(25):1878- Applications. Slide Presentation from the AHRQ 86. PMID: 10861324. 2008 Annual Conference (Text Version); Rockville, MD: Agency for Healthcare Research 37. Black N. Why we need observational studies to and Quality; Jan, 2009. http://www.ahrq.gov/ evaluate the effectiveness of health care. BMJ. about/annualmtg08/090908slides/Brookhart.htm. 1996 May 11;312(7040):1215-8. PMID: 8634569. Accessed August 14, 2012. PMCID: 2350940. 45. Evans WN, Ringel JS. Can higher cigarette taxes 38. Rothman K. Modern Epidemiology. Boston: Little improve birth outcomes? J Public Economics. Brown and Company; 1986. p. 83. 1999;72(1):135-54. 39. Petri H, Urquhart J. Channeling bias in the 46. Schneeweiss S, Seeger JD, Landon J, et al. interpretation of drug effects. Stat Med. 1991 Aprotinin during coronary-artery bypass grafting Apr;10(4):577-81. PMID: 2057656. and risk of death. N Engl J Med. 2008 Feb 40. Johnston SC. Identifying confounding by 21;358(8):771-83. PMID: 18287600. indication through blinded prospective review. Am 47. Bosco JL, Silliman RA, Thwin SS, et al. A most J Epidemiol. 2001 Aug 1;154(3):276-84. stubborn bias: no adjustment method fully PMID: 11479193. resolves confounding by indication in 41. Sturmer T, Joshi M, Glynn RJ, et al. A review of observational studies. J Clin Epidemiol. 2010 the application of propensity score methods Jan;63(1):64-74. PMID: 19457638. yielded increasing use, advantages in specific PMCID: 2789188. settings, but not substantially different estimates 48. Ray WA. Evaluating medication effects outside of compared with conventional multivariable clinical trials: new-user designs. Am J Epidemiol. methods. J Clin Epidemiol. 2006 May;59(5): 2003 Nov 1;158(9):915-20. PMID: 14585769. 437-47. PMID: 16632131. PMCID: 1448214. 49. Kristman V, Manno M, Cote P. Loss to follow-up 42. Glynn RJ, Schneeweiss S, Sturmer T. Indications in cohort studies: how much is too much? Eur J for propensity scores and review of their use in Epidemiol. 2004;19(8):751-60. PMID: 15469032. pharmacoepidemiology. Basic Clin Pharmacol Toxicol. 2006 Mar;98(3):253-9. PMID: 16611199. 50. Lash TL, Fox MP, Fink AK. Applying quantitative PMCID: 1790968. bias analysis to epidemiologic data: Springer; 2009. 43. Loughlin J, Seeger JD, Eng PM, et al. Risk of hyperkalemia in women taking ethinylestradiol/ drospirenone and other oral contraceptives. Contraception. 2008 Nov;78(5):377-83. PMID: 18929734.

71

Chapter 4. Data Elements for Registries

1. Introduction domain consists of data that describe the patient, such as information on patient demographics, Selection of data elements for a registry requires a medical history, health status, and any necessary balancing of potentially competing considerations. patient identifiers. The exposure domain describes These considerations include the importance of the the patient’s experience with the product, disease, data elements to the integrity of the registry, their device, procedure, or service of interest to the reliability, their necessity for the analysis of the registry. Exposure can also include other primary outcomes, their contribution to the overall treatments that are known to influence outcome response burden, and the incremental costs but are not necessarily the focus of the study, so associated with their collection. Registries are that their confounding influence can be adjusted generally designed for a specific purpose, and data for in the planned analyses. The outcomes domain elements not critical to the successful execution of consists of information on the patient outcomes the registry or to the core planned analyses should that are of interest to the registry; this domain not be collected unless there are explicit plans for should include both the primary endpoints and any their analysis. secondary endpoints that are part of the overall registry goals. The selection of data elements for a registry begins with the identification of the domains that must be In addition to the goals and desired outcomes, it is quantified to accomplish the registry purpose. The necessary to consider the need to create important specific data elements can then be selected, with subsets when defining the domains. Measuring consideration given to clinical data standards, potential confounding factors (variables that are common data definitions, and the use of patient linked with both the exposure and outcome) identifiers. Next, the data element list can be should be taken into account in this stage of refined to include only those elements that are registry development. Collecting data on potential necessary for the registry purpose. Once the confounders will allow for analytic or design selected elements have been incorporated into a control. (See Chapters 3 and 13.) data collection tool, the tool can be pilot tested to Understanding the time reference for all variables identify potential issues, such as the time required that can change over time is critical in order to to complete the form, data that may be more distinguish cause-and-effect relationships. For difficult to access than realized during the design example, a drug taken after an outcome is phase, and practical issues in data quality (such as observed cannot possibly have contributed to the appropriate range checks). This information can development of that outcome. Time reference then be used to modify the data elements and reach periods can be addressed by including start and a final set of elements. stop dates for variables that can change; they can also be addressed categorically, as is done in some 2. Identifying Domains quality improvement registries. For example, the Paul Coverdell National Acute Stroke Registry Registry design requires explicit articulation of the organized its patient-level information into goals of the registry and close collaboration among categories to reflect the timeframe of the stroke disciplines, such as epidemiology, health event from onset through treatment to followup. In outcomes, statistics, and clinical specialties. Once this case, the domains were categorized as the goals of the study are determined, the domains prehospital, emergency evaluation and treatment, most likely to influence the desired outcomes must in-hospital evaluation and treatment, discharge be defined. Registries generally include personal, information, and postdischarge followup.1 exposure, and outcomes information. The personal

73 Section I. Creating Registries

3. Selecting Data Elements cardioverter defibrillators and leads, which derived their publically posted data elements and Once the domains have been identified, the process definitions from the American College of of selecting data elements begins with Cardiology/American Heart Association (ACC/ identification of the data elements that best AHA) Key Data Elements and Definitions for quantify that domain and the source(s) from which Electrophysiological Studies and Procedures.6 The those data elements can be collected. When National Cancer Institute (NCI) provides the selecting data elements, gaining consensus among Cancer Data Standards Registry and Repository the registry stakeholders is important, but this (caDSR), which includes the caBIG® (Cancer must be achieved without undermining the Biomedical Informatics Grid®)–NCI data standards purpose of the registry by including elements and the Cancer Therapy Evaluation Program solely to please a stakeholder. Each data element (CTEP) common data element initiative.7, 8 The should support the purpose of the registry and North American Association of Central Cancer answer an explicit scientific question or address a Registries (NAACCR) has developed a set of specific issue or need. The most effective way to standard data elements and a data dictionary, and it select data elements is to start with the study promotes and certifies the use of these standards.9 purpose and objective, and then decide what types The American College of Surgeons National of groupings, measurements, or calculations will Cancer Database (NCDB) considers its data be needed to analyze that objective. Once the plan elements to be nationally standardized and open of analysis is clear, it is possible to work backward source.10 to define the data elements necessary to implement To a lesser extent, other disease areas also have that analysis plan. This process keeps the group begun to catalog data element lists and definitions. focused on the registry purpose and limits the In the area of trauma, the International Spinal number of extraneous (“nice to know”) data Cord Society has developed an International 2 elements that may be included. (See Case Spinal Cord Injury Core data set to facilitate Example 5.) comparison of studies from different countries,11 The data element selection process can be and the National Center for Injury Prevention and simplified if clinical data standards for a disease Control has developed Data Elements for area exist. (See Case Example 7.) While there is a Emergency Department Systems (DEEDS), which great need for common core data sets for are uniform specifications for data entered into conditions, there are few consensus or broadly emergency department patient records.12 In the accepted sets of standard data elements and data area of neurological disorders, the National definitions for most disease areas. Thus, different Institute of Neurological Disorders and Stroke studies of the same disease state may use different (NINDS) maintains a list of several hundred data definitions of fundamental concepts, such as the elements and definitions (Common Data diagnosis of myocardial infarction or the definition Elements).13 In the area of infection control, the of worsening renal function. National Vaccine Advisory Committee (NVAC) in 2007 approved a new set of core data elements for To address this problem and to support more immunization information systems, which are used consistent data elements so that comparisons as functional standards by groups such as the across studies can be more easily accomplished, American Immunization Registry Association some specialty societies and organizations are (AIRA).14, 15 Currently, there are more than one beginning to compile clinical data standards. For set of lists for some conditions (e.g., cancer) and example, the American College of Cardiology no central method to search broadly across disease (ACC) has created clinical data standards for acute areas. coronary syndromes, heart failure, and atrial fibrillation.3-5 These are used by registries such as Some standards organizations are also working on the National Cardiovascular Data Registry core data sets. The Clinical Data Interchange (NDCR)® ICD Registry ™ for implantable Standards Consortium (CDISC) Clinical Data

74 Chapter 4. Data Elements for Registries

Acquisition Standards Harmonization (CDASH) is (DAM), meaning that it provides a common a global, consensus-based effort to recommend representation of the semantics of protocol-driven minimal data sets in 16 domains. While developed clinical and preclinical research, along with the primarily for clinical trials, these domains have associated data, resources, rules, and processes significant utility for patient registries. They used to formally assess a drug, treatment, or comprise adverse events, comments, prior and procedure.17 The BRIDG model is freely available concomitant medications, demographics, to the public as part of an open-source project at disposition, drug accountability, electrocardiogram www.bridgemodel.org. It is hoped that the BRIDG test results, exposure, inclusion and exclusion model will guide clinical researchers in selecting criteria, laboratory test results, medical history, approaches that will enable their data to be physical examination, protocol deviations, subject compared with other clinical data, regardless of characteristics, substance abuse, and vital signs. the study phase or data collection method.18 The CDASH Standards information also includes In cases where clinical data standards for the a table on best practices for developing case report 16 disease area do not exist, established data sets may forms. be widely used in the field. For example, United The use of established data standards, when Network of Organ Sharing (UNOS) collects a available, is essential so that registries can large amount of data on organ transplant patients. maximally contribute to evolving medical Creators of a registry in the transplant field should knowledge. Standard terminologies—and to a consider aligning their data definitions and data greater degree, higher level groupings into core element formats with those of UNOS to simplify data sets for specific conditions—not only improve the training and data abstraction process for sites. efficiency in establishing registries but also Other examples of widely used data sets are the promote more effective sharing, combining, or Joint Commission and the Centers for Medicare & linking of data sets from different sources. Medicaid Services (CMS) data elements for Furthermore, the use of well-defined standards for hospital data submission programs. These data sets data elements and data structure ensures that the cover a range of procedures and diseases, from meaning of information captured in different heart failure and acute myocardial infarction to systems is the same. This is critical for “semantic” pregnancy and surgical infection prevention. interoperability between information systems, Hospital-based registries that collect data on these which will be increasingly important as health conditions may want to align their data sets with information system use grows. This is discussed the Joint Commission and CMS. However, one more in Chapter 15, Section 6.2. limitation of tying elements and definitions to Clinical data standards are important to allow another data collection program rather than a fixed comparisons between studies, but when different standard is that these programs may change their sets of standards overlap (i.e., are not harmonized), elements or definitions. With Joint Commission the lack of alignment may cause confusion during core measure elements, for example, this has analyses. To consolidate and align standards that occurred with some frequency. have been developed for clinical research, CDISC, If clinical data standards for the disease area and the HL7 (Health Level 7) Regulated Clinical established data sets do not exist, it is still possible Research Information Management Technical to incorporate standard terminology into a registry. Committee (RCRIM TC), NCI, and the U.S. Food This will make it easier to compare the registry and Drug Administration (FDA) have collaborated data with the data of other registries and reduce the to create the Biomedical Research Integrated training needs and data abstraction burden on sites. Domain Group (BRIDG) model. The purpose of Examples of several standard terminologies used this project is to provide an overarching model that to classify important data elements are listed in can be used to harmonize standards between the Table 4–1. Standard terminologies and suggestions clinical research domain and the health care for minimal data sets specific to pregnancy domain. BRIDG is a domain analysis model registries are provided in Chapter 21.

75 Section I. Creating Registries

Table 4–1. Standard terminologies

Standard Acronym Description and Web Site Developer

Billing-related Current Procedural CPT® Medical service and procedure codes commonly American Medical Terminology used in public and private health insurance plans Association and claims processing. Web site: http://www.ama- assn.org/ama/pub/category/3113.html International ICD, ICD-O, International standard for classifying diseases and World Health Classification of ICECI, ICF, other health problems recorded on health and vital Organization Diseases ICPC records. ICD-9-CM, a modified version of the ICD-9 standard, is used for billing and claims data in the United States, which will transition to ICD- 10-CM in 2014. The ICD is also used to code and classify mortality data from death certificates in the United States. ICD adaptations include ICD-O (oncology), ICECI (External Causes of Injury), ICF (Functioning, Disability and Health), and ICPC-2 (Primary Care, Second Edition). Web site: http://www.who.int/classifications/icd/en Clinical Systemized SNOMED CT Clinical health care terminology that maps clinical International Nomenclature of concepts with standard descriptive terms. Formerly Health Terminology Medicine SNOMED RT and SNOP. Web site: http://www. Standards ihtsdo.org/snomed-ct Development Organization Unified Medical UMLS Database of 100 medical terminologies with National Library of Language System concept mapping tools.19 Web site: http://www. Medicine nlm.nih.gov/research/umls/ Classification of OPCS-4 Code for operations, surgical procedures, and Office of Population, Interventions and interventions. Mandatory for use in National , and Procedures Health Service (England). Web site: http:// Surveys www.datadictionary.nhs.uk/web_site_content/ supporting_information/clinical_coding/opcs_ classification_of_interventions_and_procedures. asp Diagnostic and DSM The standard classification of mental disorders American Psychiatric Statistical Manual used in the United States by a wide range of health Association and mental health professionals. The version currently in use is the DSM-IV. Web site: http:// www.psych.org/MainMenu/Research/DSMIV.aspx Drugs Medical Dictionary MedDRA Terminology covering all phases of drug International for Regulatory development, excluding animal toxicology. Also Conference on Activities covers health effects and malfunctions of devices. Harmonisation (ICH) Replaced COSTART (Coding Symbols for a Thesaurus of Adverse Reaction Terms). Web site: http://www.meddramsso.com

76 Chapter 4. Data Elements for Registries

Table 4–1. Standard terminologies (continued)

Standard Acronym Description and Web Site Developer

VA National Drug NDF-RT Extension of the VA National Drug File; used U.S. Department of File Reference for modeling drug characteristics, including Veterans Affairs Terminology ingredients, chemical structure, dose form, physiologic effect, mechanism of action, pharmacokinetics, and related diseases. Web site not available. National Drug Code NDC Unique 3-segment number used as the universal U.S. Food and Drug identifier for human drugs. Web site: http://www. Administration fda.gov/cder/ndc/ RxNorm RxNorm Standardized nomenclature for clinical drugs. National Library of The name of a drug combines its ingredients, Medicine strengths, and/or form. Links to many of the drug vocabularies commonly used in pharmacy management and drug interaction software. Web site: http://www.nlm.nih.gov/research/umls/ rxnorm/ World Health WHODRUG International drug dictionary. Web site: http://www. World Health Organization Drug who-umc.org/DynPage.aspx?id=98105&mn1=734 Organization Dictionary 7&mn2=7252&mn3=7254&mn4=7338 Lab-Specific Logical Observation LOINC® Concept-based terminology for lab orders and Regenstrief Institute Identifiers Names results.19 Web site: http://www.regenstrief.org/ for Health Care and Codes loinc/ Other HUGO Gene HGNC Recognized standard for human gene nomenclature. Human Genome Nomenclature Web site: http://www.genenames.org/ Organization Committee Dietary Reference DRIs Nutrient reference values developed by the Institute Institute of Medicine Intakes of Medicine to provide the scientific basis for the Food and Nutrition development of food guidelines in Canada and the Board United States. Web site: http://fnic.nal.usda.gov/ dietary-guidance/dietary-reference-intakes/dri- tables Substance Registry SRS The central system for standards identification of, Environmental Services and information about, all substances tracked or Protection Agency regulated by the Environmental Protection Agency. Web site: http://iaspub.epa.gov/sor_internet/ registry/substreg/searchandretrieve/searchbylist/ search.do

77 Section I. Creating Registries

In addition to these standard terminologies, most appropriate order of data collection. Data numerous useful commercial code listings target elements that are related to each other in time specific needs, such as proficiency in checking for (e.g., dietary information and a fasting blood drug interactions or compatibility with widely sample for glucose or lipids) should be collected in used electronic medical record systems. Mappings the same visit rather than in different visit case between many of these element lists are also report forms. increasingly available. For example, SNOMED ® International clinician and patient participation CT (Systemized Nomenclature of Medicine may be required to meet certain registry data Clinical Terminology) can currently be mapped to objectives. In such situations, it is desirable to ICD-9-CM (International Classification of consider the international participation when Diseases, 9th Revision, Clinical Modification), and selecting data elements, especially if it will be mapping between other standards is planned or 20 necessary to collect and compare data from underway. individual countries. Examination and laboratory After investigating clinical data standards, registry test results or units may differ among countries, planners may find that there are no useful and standardization of data elements may become standards or established data sets for the registry, necessary at the data-entry level. Data elements or that these standards comprise only a small relating to cost-effectiveness studies may be portion of the data set. In these cases, the registry particularly challenging, since there is substantial will need to define and select data elements with variation among countries in health care delivery the guidance of its project team, which may systems and practice patterns, as well as in the cost include an advisory board. of medical resources that are used as “inputs.” When selecting data elements, it is often helpful to Alternatively, if capture of internationally gather input from statisticians, epidemiologists, standardized data elements is not desirable or psychometricians, and experts in health outcomes cannot be achieved, registry stakeholders should assessment who will be analyzing the data, as they consider provisions to capture data elements may notice potential analysis issues that need to be according to local standards. Later, separate data considered at the time of data element selection. conversions and merging outside the database for Data elements may also be selected based on uniform reporting or comparison of data elements performance or quality measures in a clinical area. captured in multiple countries can be evaluated (See Case Examples 6 and 53.) and performed as needed if the study design When beginning the process of defining and ensures that all data necessary for such selecting data elements, it can be useful to start by conversions have been collected. considering the registry design. Since many Table 4–2 provides examples of possible baseline registries are longitudinal, sites often collect data data elements. The actual baseline data elements at multiple visits. In these cases, it is necessary to selected for a specific registry will vary depending determine which data elements can be collected on the design, nature, and goals of the registry. once and which data elements should be collected Examples listed include patient identifiers (e.g., at every visit. Data elements that can be collected for linkage to other databases), contact information once are often collected at the baseline visit. (e.g., for followup), and residence location of In other cases, the registry may be collecting data enrollee (e.g., for geographic comparisons). Other at an event level, so all of the data elements will be administrative data elements that may be collected collected during the course of the event rather than include the source of enrollment, enrollee in separate visits. In considering when to collect a sociodemographic characteristics, and information data element, it is also important to determine the on provider locations.

78 Chapter 4. Data Elements for Registries

Table 4–2. Examples of possible baseline data elements

Enrollee contact information • Enrollee contact information for registries with direct-to-enrollee contact • Another individual who can be reached for followup (address, telephone, email) Enrollment data elements • Patient identifiers (e.g., name [last, first, middle initial], date of birth, place of birth, Social Security number) • Permission/consent • Source of enrollment (e.g., provider, institution, phone number, address, contact information) • Enrollment criteria • Sociodemographic characteristics, including race, gender, and age or date of birth • Education and/or economic status, insurance, etc. • Preferred language • Place of birth • Location of residence at enrollment • Source of information • Country, State, city, county, ZIP Code of residence

Depending on the purpose of a registry, other sets elements; again, the data elements selected for a of data elements may be required. Table 4–3 specific registry will vary and should be driven by provides examples of possible additional data the design and purpose of the registry.

Table 4–3. Examples of possible additional enrollee, provider, and environmental data elements

Pre-Enrollment History Medical history • Morbidities/conditions • Onset/duration • Severity • Treatment history • Medications • Adherence • Health care resource utilization • Diagnostic tests and results • Procedures and outcomes • Emergency room visits, hospitalizations (including length of stay), long-term care, or stays in skilled nursing facilities • Genetic information • Comorbidities • Development (pediatric/adolescent) Environmental exposures • Places of residence

79 Section I. Creating Registries

Table 4–3. Examples of possible additional enrollee, provider, and environmental data elements (continued)

Patient characteristics • Functional status (including ability to perform tasks related to daily living), quality of life, symptoms • Health behaviors (alcohol, tobacco use, physical activity, diet) • Social history • Marital status • Family history • Work history • Employment, industry, job category • Social support networks • Economic status, income, living situation • Sexual history • Foreign travel, citizenship • Legal characteristics (e.g., incarceration, legal status) • Reproductive history • Health literacy • Individual understanding of medical conditions and the risks and benefits of interventions • Social environment (e.g., community services) • Enrollment in clinical trials (if patients enrolled in clinical trials are eligible for the registry) Provider/system • Geographical coverage characteristics • Access barriers • Quality improvement programs • Disease management, case management • Compliance programs • Information technology use (e.g., computerized physician order entry, e-prescribing, electronic medical records) Financial/economic • Disability, work attendance (days lost from work), or absenteeism/presenteeism information Out-of-pocket costs • Health care utilization behavior, including outpatient visits, hospitalizations (and length of stay), and visits to the emergency room or urgent care • Patients’ assessments of the degree to which they avoid health care because of its costs • Patients’ reports of insurance coverage to assist/cover the costs of outpatient medications • Destination when discharged from a hospitalization (home, skilled nursing facility, long-term care, etc.) • Medical costs, often derived from data clinician office visits, hospitalizations (especially length of stay), and/or procedures Followup Key primary outcomes • Safety: adverse events (see Chapter 12) • Quality measurement/improvement: key selected measures at appropriate intervals • Effectiveness and value: intermediate and endpoint outcomes; health case resource use and hospitalizations, diagnostic tests and results. Particularly important are outcomes meaningful to patients, including survival, symptoms, function, and patient-reported outcomes, such as health-related quality-of-life measures. • Natural history: progression of disease severity; use of health care services; diagnostic tests, procedures, and results; quality of life; mortality; cause/date of death

80 Chapter 4. Data Elements for Registries

Table 4–3. Examples of possible additional enrollee, provider, and environmental data elements (continued)

Key secondary outcomes • Economic status • Social functioning Other potentially • Changes in medical status important information • Changes in patient characteristics • Changes in provider characteristics • Changes in financial status • Residence • Changes to, additions to, or discontinuation of exposures (medications, environment, behaviors, procedures) • Changes in health insurance coverage • Sources of care (e.g., where hospitalized) • Changes in individual attitudes, behaviors

In addition, data elements that may be needed for confounding by indication, such as the reason specific types of registries are outlined here: for prescribing a medication. In addition to the • For registries examining questions of safety for data elements mentioned above for safety, data drugs, vaccines, procedures, or devices, key elements may include individual behaviors and information includes history of the exposure provider and/or system characteristics. For and data elements that will permit analysis of assessment of cost-effectiveness, information potential confounding factors that may affect may be recorded on the financial and economic observed outcomes, such as enrollee burden of illness, such as office visits, visits to characteristics (e.g., comorbidities, concomitant urgent care or the emergency room, and therapies, socioeconomic status, ethnicity, hospitalizations, including length of stay. environmental and social factors) and provider Information on indirect or productivity costs characteristics. For drug exposures, data on use (such as absenteeism and disability) may also (start and stop dates), as well as data providing be collected. For some studies, a quality-of-life continuing evidence that the drug was actually instrument that can be analyzed to provide used (data on medication persistence and/or quality-adjusted life years or similar adherence), may be important. In some comparative data across conditions may be instances, it is also useful to record reasons for useful. discontinuation and whether pills were split or • For registries assessing quality of care and shared with others. Refer to Chapter 19 for quality improvement, data that categorize and more information on using registries for possibly differentiate among the services product safety assessments. For registries provided (e.g., equipment, training, or designed to study devices, unique device experience level of providers, type of health identifier information may be collected. See care system) may be sought, as well as Chapter 23 for more information on issues information that identifies individual patients as specific to medical devices. potential candidates for the treatment (Chapter • For registries examining questions of 22). In addition, patient-reported outcomes are effectiveness and cost-effectiveness, key valuable to assess the patients’ perception of information includes the history of exposure quality of care (Chapter 5). and data elements that will permit analysis of • For registries examining the natural history of a potential confounding factors that may affect condition, the selection of data elements would observed outcomes. It may be particularly be similar to those of effectiveness registries. useful to collect information to assess

81 Section I. Creating Registries

If one goal of a registry is to identify patient invaluable. In addition, identifier needs will differ subsets that are at higher risk for particular based on the registry goals. For example, a registry outcomes, more detailed information on patient that tracks children will need identifiers related to and provider characteristics should be collected, the parents, and registries that are likely to include and a higher sample size also may be required. twins (e.g., immunization registries) should plan This information may be important in registries for the duplication of birth dates and other that look at the usage of a procedure or treatment. identifiers. In selecting patient identifiers for use in Quality improvement registries also use this a registry, registry planners will need to determine information to understand how improvement what data are necessary for their purpose and plan differs across many types of institutions. for potential inaccurate and changing data. Another question that may arise during data Generally, patient identifiers can simplify the element selection relates to endpoint adjudication. process of identifying and tracking patients for Some significant endpoints may either be difficult followup. Patient identifiers also allow for the to confirm without review of the medical record possibility of identifying patients who are lost to (e.g., stroke) or may not be specific to a single followup due to death (i.e., through the National disease and therefore difficult to attribute without Death Index) and linking to birth certificates for such review (e.g., mortality). While clinical trials studies in children. In addition, unique patient commonly use an adjudication process for such identifiers allow for analysis to remove duplicate endpoints to better assess the endpoint or the most patients. likely cause, this is much less common in When considering the advantages of patient registries. The use of adjudication for endpoints identifiers, it is important to take into account the will depend on the purpose of the registry. potential challenges that collecting patient identifiers can present and the privacy and security 3.1 Patient Identifiers concerns associated with the collection and use of When selecting patient identifiers, there are a patient identifiers. Obtaining consent for the use of variety of options to use (e.g., the patient’s name, patient-identifiable information can be an obstacle date of birth, or some combination thereof) that to enrollment, as it can lead to the refusal of are subject to legal and security considerations. patients to participate. Chapter 7 contains more When the planned analyses require linkage to information on the ethical and legal considerations other data (such as medical records), more specific of using patient identifiers. patient information may be needed, depending on the planned method of linkage (e.g., probabilistic In addition to the data points related to primary or deterministic). (For more information on and secondary outcomes, it is important to plan for linkage considerations, see Chapter 16.) In patients who will leave the registry. While the selecting patient identifiers, some thought should intention of a registry is generally for all patients be given to the possibility that patient identifiers to remain in the study until planned followup is may change during the course of the registry. For completed, planning for patients to leave the study example, patients may change their names during before completion of full followup may reduce the course of the registry following marriage/ analysis problems. By designing a final study visit divorce, or patients may move or change their form, registry planners can more clearly document telephone numbers. Patient identifiers can also be when losses to followup occurred and possibly inaccurate because of intentional falsification by collect important information about why patients the patient (e.g., for privacy reasons in a sexually left the study. Not all registries will need a study transmitted disease registry), unintentional discontinuation form, as some studies collect data misreporting by the patient or a parent (e.g., wrong on the patient only once and do not include date of birth), or typographical errors by clerical followup information (e.g., in-hospital procedure staff. In these cases, having more than one patient registries). identifier for linking patient records can be

82 Chapter 4. Data Elements for Registries

3.2 Data Definitions their perspective on health and disease. PROs have become an increasingly important avenue of Creating explicit data definitions for each variable investigation, particularly in light of the 2001 to be collected is essential to the process of Institute of Medicine report calling for a more selecting data elements. This is important to ensure patient-centered health care system.22 The FDA internal validity of the proposed study so that all also noted the importance of PRO data in participants in data collection are acquiring the understanding certain treatment effects in its 2009 requisite information in the same reproducible guidance document.23 The use of PROs in way. (See Chapter 11.) The data definitions should registries is discussed in more detail in Chapter 5. include the ranges and acceptable values for each individual data element, as well as the potential When using an instrument to gather data on PROs, interplay of different data elements. For example, it is important both to collect the individual logic checks for the validity of data capture may be question responses and to calculate the summary created for data elements that should be mutually or composite score. The summary score, which exclusive. may be for the entire instrument or for individual domains, is ultimately used to report results. When deciding on data definitions, it is important However, if the registry collects only the summary to determine which data elements are required and score, it will not be possible to examine how the which elements may be optional. This is patients scored on different components of the particularly true in cases where the registry may instrument during the registry analysis phase. collect a few additional “nice to know” data elements. The determination will differ depending on whether the registry is using existing medical 4. Registry Data Map record documentation to obtain a particular data element or whether the clinician is being asked Once data elements have been selected, a data map directly. For example, the New York Heart should be created. The data map identifies all Association Functional Class for heart failure is an sources of data (Chapter 6) and explains how the important staging element but is often not sources of data will be integrated. Data maps are documented.21 However, if clinicians are asked to useful to defend the validity and/or reliability of provide the data point prospectively, they can the data, and they are typically an integral part of readily do so. Consideration should also be given the data management plan (Chapter 11, Section to accounting for missing or unknown data. In 2.5). some cases, a data element may be unknown or not documented for a particular patient, and followup 5. Pilot Testing with the patient to answer the question may not be possible. Including an option on the form for “not After the data elements have been selected and the documented” or “unknown” will allow the person data map created, it is important to pilot test the completing the case report form to provide a data collection tools to determine the time needed response to each question rather than leaving it to complete the form and the resulting subject/ blank. Depending on the analysis plans for the abstractor burden. For example, through pilot registry, the distinction between undocumented testing, registry planners might determine that it is data and missing data may be important. wise to collect certain data elements that are either highly burdensome or only “nice to know” in only 3.3 Patient-Reported Outcomes a subset of participating sites (nested registry) that agree to the more intensive data collection, so as When collecting data for patient outcomes not to endanger participation in the registry as a analysis, it is important to use patient-reported whole. Pilot testing should also help to identify the outcomes (PROs) that are valid, reliable, rate of missing data and any validity issues with responsive, interpretable, and translatable. PROs the data collection system. reflect the patients’ perceptions of their status and

83 Section I. Creating Registries

The burden of form collection is a major factor interpretation was recorded in the chart. The two determining a registry’s success or failure, with time points can have significant variation, major implications for the cost of participation and depending on the documentation practices of the for the overall acceptance of the registry by institution. hospitals and health care personnel. Moreover, Pilot testing ranges in practice from ad hoc knowing the anticipated time needed for patient assessments of the face validity of instruments and recruitment/enrollment will allow better materials in clinical sites, to trial runs of the communication to potential sites regarding the registry in small numbers of sites, to highly scope and magnitude of commitment required to structured evaluations of inter-rater agreement. participate in the study. Registries that obtain The level of pilot testing is determined by multiple information directly from patients include the factors. Accuracy of data entry is a key criterion to additional issue of participant burden, with the evaluate during the pilot phase of the registry. potential for participant fatigue, leading to failure When a “gold standard” exists, the level of to answer all items in the registry. Highly agreement with a reference standard (construct burdensome questions can be collected in a validity) may be measured.24 Data collected by prespecified subset of subjects. The purpose of seasoned abstractors or auditors following strict these added questions should be carefully operational criteria can serve as the gold standard considered when determining the subset so that by which to judge accuracy of abstraction for useful and accurate conclusions can be achieved. chart-based registries.25 Pilot testing the registry also allows the In instances where no reference standard is opportunity to identify issues and make available, reproducibility of responses to registry refinements in the registry-specific data collection elements by abstractors (inter-rater reliability) or tools, including alterations in the format or order test-retest agreement of subject responses may be of data elements and clarification of item assessed.26 Reliability and/or validity of a data definitions. Alterations to validated PRO measures element should be tested in the pilot phase are generally not advised unless they are whenever the element is collected in new revalidated. Validated PRO measures that are not populations or for new applications. Similar used in the validated format may be perceived as mechanisms to those used during the pilot phase invalid or unreliable. can be used during data quality assurance (Chapter Piloting may also uncover problems in registry 11, Section 3). A kappa measure of how logistics, such as the ability to accurately or much the level of agreement between two or more comprehensively identify subjects for inclusion. A observers exceeds the amount of agreement fundamental aspect of pilot testing is evaluation of expected by chance alone is the most common the accuracy and completeness of registry method for measuring reliability of categorical and questions and the comprehensiveness of both ordinal data. The intraclass correlation coefficient, instructional materials and training in addressing or inter-rater reliability coefficient, provides these potential issues. Gaps in clarity concerning information on the degree of agreement for questions can result in missing or misclassified continuous data. It is a proportion that ranges from data, which in turn may cause bias and result in zero to one. Item-specific agreement represents the inaccurate or misleading conclusions. For highest standard for registries; it has been example, time points, such as time to radiologic employed in cancer registries and to assess the interpretation of imaging test, may be difficult to quality of data in statewide stroke registries. Other obtain retrospectively and, if they do exist in the methods, such as the Bland and Altman method,26 chart, may not be consistently documented. may also be chosen, depending upon the type of Without additional instruction, some hospitals may data and registry purpose. indicate the time the image was read by the radiologist and others may use the time when the

84 Chapter 4. Data Elements for Registries

6. Summary standards, common data definitions, and whether patient identifiers will be used. It is also important The selection of data elements requires balancing to determine which elements are absolutely such factors as their importance for the integrity of necessary and which are desirable but not the registry and for the analysis of primary essential. Once data elements have been selected, a outcomes, their reliability, their contribution to the data map should be created, and the data collection overall burden for respondents, and the tools should be pilot tested. Overall, the choice of incremental costs associated with their collection. data elements should be guided by parsimony, Data elements should be selected with validity, and a focus on achieving the registry’s consideration for established clinical data purpose.

Case Examples for Chapter 4

Case Example 5. Selecting data elements for following intervention with ESTs, little a registry information was available on real-world practice Description The Dosing and Outcomes Study patterns and outcomes in oncology patients. The of Erythropoiesis-stimulating registry team determined that a prospective Therapies (DOSE) Registry was observational effectiveness study in this designed to understand anemia therapeutic area was needed to gain this management patterns and information. The three key challenges were to clinical, economic, and patient- make the study representative of real-world reported outcomes in oncology practices and settings (e.g., hospital-based clinics, patients treated in outpatient community oncology clinics); to collect data oncology practice settings across elements that were straightforward so as to the United States. The minimize potential data collection errors; and to prospective design of the DOSE collect sufficient data to study effectiveness, Registry enabled data capture while ensuring that the data collection remained from oncology patients treated feasible and time efficient for outpatient oncology with erythropoiesis-stimulating clinics. therapies. Proposed Solution Sponsor Centocor Ortho Biotech The registry team began selecting data elements Services, LLC by completing a thorough literature review. Year Started 2003 Because this would be one of the first prospective observational studies in this therapeutic area, the Year Ended 2009 team wanted to ensure that study results could be No. of Sites 71 presented to health care professionals and decisionmakers in a manner consistent with No. of Patients 2,354 clinical trials, of which there were many. The Challenge team also intended to make the data reports from this study comparable with clinical trial reports. Epoetin alfa was approved for patients with To meet these objectives, data elements (e.g., chemotherapy-induced anemia in 1994. In 2002, baseline demographics, dosing patterns, the U.S. Food and Drug Administration approved hemoglobin levels) similar to those in clinical a second erythropoiesis-stimulating therapy trials were selected whenever possible, based on a (EST), darbepoetin alfa, for a similar indication. thorough literature review. While multiple clinical trials described outcomes

85 Section I. Creating Registries

Case Example 5. Selecting data elements for minimize confusion and enable a variety of site a registry (continued) personnel, as well as the patients, to complete the Proposed Solution (continued) registry data collection. For the patient-reported outcomes component of Results the registry, the team incorporated standard The registry was launched in 2003 as one of the validated instruments. This decision allowed the first prospective observational effectiveness team to avoid developing and validating new studies in this therapeutic area. Seventy-one sites instruments and supported consistency with and 2,354 patients enrolled in the study. The sites clinical trial literature, as many trials had participating in the registry represented a wide incorporated these instruments. To capture geographic distribution and a mixture of patient-reported data, the team selected two outpatient practice settings. instruments, the Functional Assessment of Cancer Key Point Therapy–Anemia (FACT-An) and the Linear Analog Scale Assessment (LASA) tool. The Use of common data elements, guided by a FACT-An tool, developed from the FACT-General literature review, and validated patient-reported scale, had been designed and validated to outcomes instruments enhanced data measure the impact of anemia in cancer patients. generalizability and comparability with clinical The LASA tool enables patients to report their trial data. A multidisciplinary advisory board also energy level, activity level, and overall quality of helped to ensure collection of key data elements life on a scale of 0 to 100. Both tools are in an appropriate manner from both a clinical and commonly used to gather patient-reported practical standpoint. outcomes data for cancer patients. For More Information Following the literature review, an advisory board Larholt K, Burton TM, Hoaglin DC. et al. was convened to discuss the registry objectives, Clinical and patient-reported outcomes based on data elements, and study execution. The advisory achieved hemoglobin levels in chemotherapy- board included representatives from the medical treated cancer patients receiving erythropoiesis- and nursing professions. The multidisciplinary stimulating agents. Commun Oncol. 2009;6: board provided insights into both the practical 403–8. and clinical aspects of the registry procedures and data elements. Throughout the process, the Larholt K, Pashos CL, Wang Q. et al. Dosing and registry team remained focused on both the Outcomes Study of Erythropoiesis-Stimulating overall registry objectives and user-friendly data Therapies (DOSE): a registry for characterizing collection. In particular, the team worked to make anaemia management and outcomes in oncology each question clear and unambiguous in order to patients. Clin Drug Invest. 2008;28(3):159–67.

86 Chapter 4. Data Elements for Registries

Case Example 6. Understanding the needs processes of MI care affect patients’ health status. and goals of registry participants To develop the registry data set, the team began Description The Prospective Registry by clearly defining the phases of care and Evaluating Myocardial recovery and identifying the clinical Infarction: Events and Recovery characteristics that were important in each of (PREMIER) studied the health these phases. These included patient status of patients for one year characteristics upon hospital arrival, details of after discharge for a myocardial inpatient care, and details of outpatient care. The infarction. The registry focused team felt that information on each of these phases on developing a rich was necessary, since the variability of any understanding of the patients’ outcome over 1 year may be explained by patient, symptoms, functional status, inpatient treatment, or outpatient factors. Health and squality of life by collecting status also includes many determinants beyond extensive baseline data in the the clinical status of disease, such as access to hospital and completing care, socioeconomic status, and social support; followup interviews at 1, 6, and the registry needed to collect these additional 12 months. data in order to fully understand the health status Sponsor CV Therapeutics and CV outcomes. Outcomes Proposed Solution Year Started 2003 While registries often try to include as many Year Ended 2004 eligible patients and sites as possible by reducing the burden of data entry, this registry took an No. of Sites 19 alternative approach. The team designed a data No. of Patients 2,498 set that included more than 650 baseline data elements and more than 200 followup interview- Challenge assessed data elements. Instead of allowing With the significant advances in myocardial retrospective chart abstraction, the registry infarction (MI) care over the past 20 years, many required hospitals to complete a five-page patient studies have documented the improved mortality interview while the patient was in the hospital. and morbidity associated with these new The registry demanded significant resources from treatments. These studies typically have focused the participating sites. For each patient, the on in-hospital care, with little to no followup registry required about 4 hours of time, with 15 component. As a result, information on the minutes for screening, 2 hours for chart transition from inpatient to outpatient care has abstraction, 45 minutes for interviews, 45 been lacking, as have data on health status minutes for data entry, and 15 minutes of a outcomes. cardiologist’s time to interpret the electrocardiograms and angiograms. A detailed, PREMIER was designed to address these gaps by prespecified sampling plan was developed by collecting detailed information on MI patients each site and approved by the data coordinating during the hospital stay and through followup center to ensure that the patients enrolled at each telephone interviews conducted at 1, 6, and 12 center were representative of all of the patients months. The goal of the registry was to provide a seen at that site. rich understanding of patients’ health status (their symptoms, function, and quality of life) 1 year The registry team developed this extremely after an acute MI. The registry also proposed to detailed data set and data collection process quantify the prevalence, determinants, and through extensive consultations with the registry consequences of patient and clinical factors in participants. The coordinators and steering order to understand how the structures and

87 Section I. Creating Registries

Case Example 6. Understanding the needs collection, sites enjoyed the academic and goals of registry participants (continued) productivity and collaborative nature of the study. Proposed Solution (continued) The data coordinating center created a Web site that offered private groups for the principal committees reviewed the data set multiple times, investigators, so that each investigator had access with some sites giving extensive feedback. to all of the abstract ideas and all of the research Throughout the development process, there was that was being done. This structure provided an ongoing dialog among the registry designers, nurturing and support for the investigators, and the steering committee, and the registry sites. they viewed the registry as a way to engage The registry team also used standard definitions themselves and their institutions in research with and established instruments whenever possible to a prominent, highly respected team. enable the registry data to be cross-referenced to On the patient side, the registry met followup other studies and to minimize the training burden. goals. More than 85 percent of participants The team used the American College of provided 12-month followup information. The Cardiology Data Standards for Acute Coronary registry team attributed this followup rate to the Syndromes for data definitions of any overlapping strong rapport that the interviewers developed fields. To measure other areas of the patient with the patients during the course of the experience, the team used the Patient Health followup period. Questionnaire to examine depression, the ENRICHD Social Support Inventory to measure Key Point social support, the Short Form-12 to quantify This example illustrates that there is no maximum overall mental and physical health, and the Seattle or minimum number of data elements for a Angina Questionnaire (SAQ) to understand the successful registry. Instead, a registry can best patients’ perspective on how coronary disease achieve its goals by ensuring that sufficient affects their life. information is collected to achieve the purpose of Results the registry while remaining feasible for the participants. An open, ongoing dialog with the The data collection burden posed some participants or a subgroup of participants can challenges. Two of the 19 sites dropped out of the help determine what is feasible for a particular registry soon after it began. Two other sites fell registry and to ensure that the registry will retain behind on their chart abstractions. Turnover of the participants for the life of the study. personnel and multiple commitments at participating sites also delayed the study. For More Information Despite these challenges, the registry experienced Spertus JA, Peterson E, Rumsfeld JS. et al. The very little loss of enthusiasm or loss of sites once Prospective Registry Evaluating Myocardial it was up and running. The remaining 17 sites Infarction: Events and Recovery (PREMIER)– completed the registry and collected data on evaluating the impact of myocardial infarction on nearly 2,500 patients. In return for this data patient outcomes. Am Heart J. 2006;151(3): 589–97.

88 Chapter 4. Data Elements for Registries

Case Example 7. Using standardized data During the planning phase of the registry, the elements in a registry sponsor elected to use standardized data elements Description The Caris Registry is a national, wherever possible, in order to maintain flexibility multicenter, Web-based registry and to anticipate multiple future uses of registry that tracks long-term outcomes data. The National Cancer Institute’s Cancer for patients who have undergone Data Standards Registry and Repository (caDSR) Caris Molecular Intelligence™ standardized data dictionary contains common Services. data elements (CDEs) that can be reused for multiple purposes. The registry used some of Sponsor Caris Life Sciences these CDEs exactly as they appear in the caDSR Year Started 2009 (e.g., demographics). Other data elements that the sponsor wished to collect were not present in Year Ended Ongoing the caDSR (e.g., “Did the patient receive No. of Sites 96 molecular-guided therapy?”). For these elements, the sponsor collaborated with the Center for No. of Patients >1400 Biomedical Informatics and Information Challenge Technology group to create new CDEs that were Molecular biomarker data may be valuable in incorporated into the caDSR data dictionary. Of guiding treatment decisions for cancer patients. the 100 clinical data elements in the registry, 87 Caris Life Sciences offers a commercial were incorporated directly from the caDSR data molecular profiling service (Caris Molecular dictionary and 13 were added to the data Intelligence™) that combines biomarker analysis dictionary through collaboration with the of a patient’s tumor with an analysis of the National Cancer Institute. published scientific literature in order to report Results personalized, evidence-based treatment options. To date, 1,400 patients from 96 centers across the These data may impact the physicians’ and United States have been enrolled in the Caris patients’ treatment decisions at one point in time, Registry. At least 1,124 of these patients have but collection of longitudinal data would allow followup data capturing disease status, treatments for correlation of treatment recommendations to and clinical outcomes and 500 of those have clinical outcomes. In addition, longitudinal data completed end of study reports capturing vital could support collaborative investigator-initiated status and cancer related deaths. research that may be focused on using molecular profiling as a tool to improve treatment selection In the first half of 2013, Caris agreed to and associated outcomes for patients with cancer. participate in a retrospective study of registry data titled, “A Retrospective Investigation To Proposed Solution Evaluate The Use Of Target Now™ Assay in The Caris Registry employs a scientifically valid Selecting Treatment in Patients with Advanced and regulatory-compliant protocol that is Stage Metastatic Cancer.” intended to capture clinical disease, treatment, Key Point and outcome data over the course of five years from patients who have had Caris Molecular Common data elements endorsed by recognized Intelligence™ Services performed. Medical standards organizations are available for registry history, disease status, treatments, and outcomes planners and may be useful for registries in some are captured at enrollment (defined as the date of disease areas. Use of CDEs can increase the report) and every 9 months for 5 years. The opportunities for standardized collaboration, registry is maintained as a limited data set and all linkage, and additional exploratory analysis. biological and laboratory data is de-identified.

89 Section I. Creating Registries

Case Example 7. Using standardized data presented at the 2012 ASCO Annual Meeting. elements in a registry (continued) June 1–5, 2012. https://asco.confex.com/ For More Information asco/2012/sci/papers/viewonly.cgi?username=971 55&password=340328. Accessed August 27, Sanders S, Schroeder W, Wright A, et al. The 2012. Caris registry: Building a biomarker-focused database to advance patient care. Abstract http://www.carislifesciences.com/caris-registry

References for Chapter 4 6. National Cardiovascular Disease Registry. “ICD Registry Data Collection.”. https://www.ncdr.com/ 1. Wattigney WA, Croft JB, Mensah GA, et al. webncdr/icd/home/datacollection. Accessed July Establishing data elements for the Paul Coverdell 16, 2013. National Acute Stroke Registry: Part 1: 7. National Cancer Institute. Cancer Data Standards proceedings of an expert panel. Stroke. 2003 Registry and Repository (caDSR). http://ncicb.nci. Jan;34(1):151-6. PMID: 12511767. nih.gov/infrastructure/cacore_overview/cadsr. 2. Good PI. A manager’s guide to the design and Accessed August 15, 2012. conduct of clinical trials. New York: John Wiley & 8. National Cancer Institute. CTEP Common Data Sons, Inc.; 2002. Elements. https://wiki.nci.nih.gov/display/caDSR/ 3. Cannon CP, Battler A, Brindis RG, et al. American CTEP+Common+Data+Elements. Accessed July College of Cardiology key data elements and 16, 2013. definitions for measuring the clinical management 9. The North American Association of Central and outcomes of patients with acute coronary Cancer Registries. Data Standards & Data syndromes. A report of the American College of Dictionary (Volume II). http://www.naaccr.org/ Cardiology Task Force on Clinical Data Standards StandardsandRegistryOperations/VolumeII.aspx. (Acute Coronary Syndromes Writing Committee). Accessed August 15, 2012. J Am Coll Cardiol. 2001 Dec;38(7):2114-30. PMID: 11738323. 10. The American College of Surgeons Commission on Cancer. National Quality Forum Endorsed 4. McNamara RL, Brass LM, Drozda JP, Jr., et al. Commission on Cancer Measures for Quality of ACC/AHA key data elements and definitions for Cancer Care for Breast and Colorectal Cancers. measuring the clinical management and outcomes http://www.facs.org/cancer/qualitymeasures.html. of patients with atrial fibrillation: a report of the Accessed August 15, 2012. American College of Cardiology/American Heart Association Task Force on Clinical Data Standards 11. DeVivo M, Biering-Sorensen F, Charlifue S, et al. (Writing Committee to Develop Data Standards International Spinal Cord Injury Core Data Set. on Atrial Fibrillation). Circulation. 2004 Jun Spinal Cord. 2006 Sep;44(9):535-40. 29;109(25):3223-43. PMID: 15226233. PMID: 16955073. 5. Radford MJ, Arnold JM, Bennett SJ, et al. ACC/ 12. National Center for Injury Prevention and Control. AHA key data elements and definitions for DEEDS – Data Elements for Emergency measuring the clinical management and outcomes Department Systems. http://www.cdc.gov/ncipc/ of patients with chronic heart failure: a report of pub-res/deedspage.htm. Accessed August 15, the American College of Cardiology/American 2012. Heart Association Task Force on Clinical Data 13. National Institute of Neurological Disorders and Standards (Writing Committee to Develop Heart Stroke. Common Data Elements. http://www. Failure Clinical Data Standards): developed in commondataelements.ninds.nih. collaboration with the American College of Chest gov/#page=Default. Accessed August 15, 2012. Physicians and the International Society for Heart and Lung Transplantation: endorsed by the Heart Failure Society of America. Circulation. 2005 Sep 20;112(12):1888-916. PMID: 16162914.

90 Chapter 4. Data Elements for Registries

14. Centers for Disease Control and Prevention. 21. Yancy CW, Fonarow GC, Albert NM, et al. Vaccines & Immunizations. IIS Recommended Influence of patient age and sex on delivery of Core Data Elements. http://www.cdc.gov/ guideline-recommended heart failure care in the vaccines/programs/iis/core-data-elements.html. outpatient cardiology practice setting: findings Accessed August 15, 2012. from IMPROVE HF. Am Heart J. 2009 Apr;157(4):754-62 e2. PMID: 19332206. 15. American Immunization Registry Association. Immunization Registry Functional Standards. 22. Institute of Medicine, Committee on Quality of http://www.immregistries.org/resources/standards/ Health Care in America. Crossing the Quality functional-standards. Accessed July 16, 2013. Chasm: A New Health System for the 21st Century. Washington DC: National Academy 16. Clinical Data Interchange Standards Consortium. Press, Institute of Medicine; 2001. Clinical Data Acquisition Standards and Harmonization (CDASH). http://www.cdisc.org/ 23. U.S. Food and Drug Administration. Guidance for cdash. Accessed August 15, 2012. Industry: Patient Reported Outcome Measures: Use in Medical Product Development and 17. Biomedical Research Integrated Domain Group Labeling Claims. December 2009. http://www.fda. (BRIDG). http://www.bridgmodel.org. Accessed gov/downloads/Drugs/ August 15, 2012. GuidanceComplianceRegulatoryInformation/ 18. HL7. HL7 and CDISC mark first anniversary of Guidances/UCM193282.pdf. Accessed August 15, renewed associate charter agreement, joint 2012. projects result from important healthcare-clinical 24. Goldberg J, Gelfand HM, Levy PS. Registry research industry collaboration [press release]. evaluation methods: a review and case study. http://www.hl7.org/documentcenter/public_temp_ Epidemiol Rev. 1980;2:210-20. PMID: 7000537. A7EA2C6E-1C23-BA17-0CA05816D86FD311/ pressreleases/20051012b.pdf. Accessed August 25. Sorensen HT, Sabroe S, Olsen J. A framework for 15, 2012. evaluation of secondary data sources for epidemiological research. Int J Epidemiol. 1996 19. Kim K. iHealthReports. California HealthCare Apr;25(2):435-42. PMID: 9119571. Foundation. Clinical data standards in health care: five case studies. http://www.chcf.org/topics/view. 26. Bland JM, Altman DG. Statistical methods for cfm?itemID=112795. Accessed August 15, 2012. assessing agreement between two methods of clinical measurement. Lancet. 1986 Feb 20. Imel M. A closer look: the SNOMED clinical 8;1(8476):307-10. PMID: 2868172. terms to ICD-9-CM mapping. J AHIMA. 2002 Jun;73(6):66-9; quiz 71-2. PMID: 12066386.

91

Chapter 5. Use of Patient-Reported Outcomes in Registries

1. Introduction emerged as the preferential choice in recent literature, and there are cogent arguments As the medical system refocuses on delivering surrounding its use. However, the more general patient-centered care, the importance of measuring “QOL” reflects the fact that health status affects and reporting those aspects of health and well- numerous aspects of daily life and influences being that are best described by patients overall QOL. Thus, further discussions in this themselves, whether related to disease, treatment, chapter will consistently use the term QOL. 1-4 or both is increasingly recognized. While widespread adoption of PROs as a key Discrepancies exist between patient and clinician component in clinical research has not occurred, estimates of both the prevalence and severity of there is increasing recognition of their role in patients’ symptoms as well as functional complementing traditional clinical and impairments, highlighting the importance of direct administrative data. To this end, the importance of 3, 5-9 patient reporting. According to the U.S. Food incorporating PROs into clinical research has been and Drug Administration (FDA), a patient- highlighted by a number of national policymaking reported outcome (PRO) is defined as a organizations.2, 37 Recently, the FDA identified measurement based on a report that comes directly PROs as the regulatory standard for supporting from the patient (i.e., the study subject) about the subjective endpoints, like symptoms, in drug status of a patient’s health condition, without approval and labeling, and their updated guidance amendment or interpretation of the patient’s distributed in December 2009 (hereafter referred 10 response by a clinician or anyone else. (See Table to as “the FDA guidance document”) provides 5–1.) PROs are a subgroup of patient outcomes. clear instructions on PRO measurement in drug The latter category is more general and refers to development trials.10 While the purposes of PROs any outcome related to a patient, whether reported in registry studies are not for supporting labeling by the patient or described by a third party (e.g., claims, the guidance provided by the FDA has by an imaging report, laboratory evaluation, or helped refine the definition of PROs and expand clinician assessment). the sphere of interest surrounding their use. Most Over the past 20 years, an expanding body of importantly, the FDA guidance document10 has literature has demonstrated that PROs are established a benchmark, albeit a high one, for associated with traditional outcomes, such as PRO data and has been the focus of much recent overall survival11-16 and tumor response.17 PROs PRO-related literature (references too numerous to themselves are also increasingly recognized as list). For this reason, the standards set by the FDA valid outcomes (e.g., quality of life [QOL], pain, are heavily referenced in the following discussion. 18-27 breathlessness, physical functioning). Presently, there are no evidence-based guidelines Systematic collection of PROs in clinical trials, for inclusion of PROs in registries; consequently, patient registries, and usual clinical care is feasible there is substantial heterogeneity in capture and 28-32 and efficient. PROs are more reflective of reporting of PROs in this setting (see, for example, 33 underlying health status than physician reporting the review about some large registries in and facilitate discussion of important symptoms rheumatoid arthritis).38 Recent initiatives to define 34 and QOL with clinicians. Additionally, they have how PROs should be used in oncology been shown to serve as supporting comparative effectiveness research (CER) are 29 documentation, improve symptom instructive,39 as they reflect current, collaborative 35 management, and potentially impact clinical opinions of many different stakeholders, and may 30, 36 decisionmaking, all of which are viewed serve as a template for inclusion of PROs in 30 favorably. As a matter of terminology, the term registries (Table 5–2). “health-related quality of life” (HRQOL) has

93 Section I. Creating Registries

Table 5–1. Definitions of commonly encountered terms within PRO-related literature

Term Definition

Ability to detect Evidence that a patient-reported outcome (PRO) instrument can identify differences change in scores over time in individuals or groups who have changed with respect to the measurement concept.10 Clinician reported Outcomes that are either observed by the physician (e.g., cure of infection and absence of outcome (ClinRO) lesions) or require physician interpretation (e.g., radiologic results and tumor response). In addition, ClinROs may include formal or informal scales completed by the physician using information about the patient.40 Concept The specific measurement goal, or the thing that is measured by a PRO.10 Conceptual Explicitly defines the concepts measured by the instrument in a diagram that presents framework a description of the relationships between items, domain (subconcepts), and concepts measured and the scores produced by a PRO instrument.10 Construct validity The degree to which what was measured reflects the a priori conceptualization of what should be measured.41 Content validity The extent to which the instrument actually measures the concepts of interest.42 Criterion validity The extent to which the scores of PRO measure reflect the gold standard measure of the same concept.10 Domain A subconcept represented by a score of an instrument that measures a larger concept that includes multiple domains.10 HRQOL The subjective assessment of the impact of disease and treatment across the physical, psychological, social, and somatic domains of functioning and well-being.43 Instrument A means to capture data (e.g., a questionnaire) plus all the information and documentation that supports its use. Generally, this includes clearly defined methods and instruction for administration or responding, a standard format for data collection, and well-documented methods for scoring, analysis, and interpretation of results in the target population.10 Item An individual question, statement, or task (and its standardized response options) that is evaluated by the patient to address a particular concept.10 Item bank A comprehensive collection of questions (and their response options) designed to measure an underlying construct across its entire continuum.44 Metadata Structured information that describes, explains, locates, or otherwise makes it easier to retrieve, use, or manage an information source.45 PRO A measurement based on a report that comes directly from the patient (i.e., the study subject) about the status of a patient’s health condition, without amendment or interpretation of the patient’s response by a clinician or anyone else.10 Proxy-reported A measurement based on a report by someone other than the patient reporting as if he or outcome she is the patient.10 QOL An individual’s perception of their position in life in the context of the culture and the value systems in which they live and in relation to their goals, expectations, standards, and concerns. It is a broad-ranging concept affected in a complex way by the person’s physical health, psychological state, level of independence, social relationships, personal beliefs, and relationship to salient features of the environment.46 Recall period The period of time patients are asked to consider in responding to a PRO item or question.10

94 Chapter 5. Use of Patient-Reported Outcomes in Registries

Table 5–1. Definitions of commonly encountered terms within PRO-related literature (continued)

Term Definition

Reliability The ability of an instrument to yield the same result on serial administrations when no change in the concept being measured is expected.42 Scale The system of numbers of verbal anchors by which a value or score is derived for an item. Examples include VAS, Likert scales, and rating scales.10 Score A number derived from a patient’s response to items in a questionnaire. A score is computed based on a prespecified, validated scoring algorithm and is subsequently used in statistical analyses of clinical results.10

Table 5–2. Example of guidelines for PRO incorporation into product-labeling claims in oncologya

Selection of Measures 1. Include PROs in all prospectively designed CER and post-marketing studies in adult oncology (including registries, observational cohorts, and controlled trials). 2. Include systematic assessment of the following 14 patient-reported symptoms (the “Core” symptom set) in all CER and postmarketing clinical studies in adult oncology: anorexia, anxiety, constipation, depression, diarrhea, dyspnea, fatigue, insomnia, mucositis, nausea, pain, sensory neuropathy, rash, vomiting. 3. Include additional patient-reported symptoms as appropriate to a specific study’s population, intervention, context, objectives, and setting (in addition to the Core symptom set), and incorporate a process that allows individual patients to report unsolicited symptoms. 4. Measure QOL, either via a single-item or multi-item questionnaire, in all prospective CER and post- marketing clinical studies. Inclusion of a measure that enables cost-utility analysis is encouraged. 5. Selected measures to assess symptoms or QOL should have demonstrated content validity (based on direct patient input), criterion validity, reliability, and sensitivity in the intended patient population (including assessment of the meaningfulness of specific score changes and the ability to detect change over time), as well as an appropriate recall period. Translations from other languages should be conducted in accordance with existing methodological standards. Implementation Methods 6. Limit PRO data collection so that the average patient can complete the process within 20 minutes at the initial (baseline) visit and within 10 minutes at any subsequent time points. 7. Collect PROs as frequently as necessary to meet research objectives without overburdening patients. When using PROs to assess potential treatment benefits, collection of PROs at baseline and following treatment completion or study withdrawal as well as at selected long-term time points should be considered a minimum standard. When using PROs to assess treatment toxicities/harms or comparative tolerability, more frequent assessment is merited, such as at baseline and every 1-4 weeks during active therapy as well as at selected long-term time points. 8. Collect PROs via electronic means whenever possible. 9. Establish measurement equivalence when mixing modes of PRO measure administration in a study (e.g., Web, telephone/interactive voice response [IVRS], handheld device, and/or paper). 10. Employ methods to minimize missing PRO data, including education of local site personnel, training of patients, and real-time monitoring of adherence with backup data collection.

95 Section I. Creating Registries

Table 5–2. Example of guidelines for PRO incorporation into product-labeling claims in oncology (continued)

Data Analysis and Reporting 11. Include in the protocol a plan for analyzing and reporting missing PRO data. 12. For each PRO measure, report the proportion of patients experiencing a change from baseline demonstrated as being meaningful to patients. 13. Evaluate the cumulative distribution of responses for each PRO measure and include cumulative distribution curves in reports and publications. 14. Include a mechanism for alerting clinical staff in real-time about symptoms of concern reported by patients during study participation. 15. Analyze and publish results of PRO analyses simultaneously with other clinical outcomes. a Adapted from the Center for Medical Technology Policy (CMTP), Effectiveness Guidance Document: Recommendations for Incorporating Patient-Reported Outcomes into the Design of Post-Marketing Clinical Trials in Adult Oncology.39 Used with permission. Copyright restrictions apply.

2. The Role of PROs in Patient-centered outcomes research helps people make informed health care decisions Registries and allows their voice to be heard in assessing the value of health care options. This research 2.1 Relationship Between PROs and answers patient-focused questions: (1) “Given CER my personal characteristics, conditions and CER was recently defined by the Institute of preferences, what should I expect to happen to Medicine as: me?” (2) “What are my options and what are the benefits and harms of those options?” “… the generation and synthesis of evidence (3) “What can I do to improve the outcomes that compares the benefits and harms of that are most important to me?” (4) “How can alternative methods to prevent, diagnose, treat, the health care system improve my chances of and monitor a clinical condition or to improve achieving the outcomes I prefer?”50 the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and By definition, PCOR is impossible to pursue policy makers to make informed decisions that without including the patient voice, and PROs are will improve health care at both the individual an important tool for capturing the patient voice. and population levels.”47 As PCOR is effectively a subset of CER (and will not be referred to independently from this point), Central to this definition is that the information PROs are therefore critical components of CER as generated by CER should assist consumers of well. The importance of PROs in CER is health care (i.e., patients) in making decisions. Of highlighted by the interest in the patient experience great interest to patients are factors like QOL, of the multiple stakeholders who ultimately use symptom burden, and functional status, which are results of CER.49 best described directly by patients, thereby implicitly emphasizing the importance of PROs to 2.2 Relationship Between CER and CER.48, 49 The strength of this relationship is Registries furthered by the term patient-centered outcomes research (PCOR), which has emerged after passage While clinical trials are generally felt to represent of the Patient Protection and Affordable Care Act the gold standard of evidence to support clinical that established the Patient-Centered Outcomes decisions, many clinical trials are conducted under Research Institute (PCORI). According to PCORI: conditions that limit generalizability or do not

96 Chapter 5. Use of Patient-Reported Outcomes in Registries

emphasize factors that are important to patients 2.3.1 Describing the Natural History of Disease and clinicians in the course of actual practice. A requirement of registries intended to describe Clinicians and patients face challenging decisions natural history of disease is adequate information regarding treatment choices and toxicity profiles about symptom burden and related QOL that are unaddressed by traditional clinical trials, trajectories, especially in the setting of rare and these are exactly the types of questions that diseases, inherited diseases with increasing life CER is intended to address. Registries are span (e.g., cystic fibrosis, sickle cell disease), and important tools for answering such questions. They heterogeneous diseases (e.g., chronic obstructive can evaluate effects in a more “real-world” pulmonary disease, breast cancer). Registries can population, improving generalizability. In provide useful information on the expected course uncommon diseases, where traditional clinical of health even in the absence of treatment— trials are unrealistic because of small numbers, information which could provide useful registries can help fill the information void on any information regarding the need for and timing of number of issues, including treatment options and treatment. Understanding how new therapies responses, natural history, and QOL. Registries impact patient experience can also be captured can be designed to answer specific questions that under this rubric. For example, metastatic renal affect clinical practice but were unaddressed by cell carcinoma is a relatively uncommon pivotal clinical trials. Importantly, when partnered malignancy for which the FDA has approved six with electronic health records (EHRs), registries targeted therapies within the past decade. All have can capitalize on the massive amounts of data different toxicity profiles and different symptom collected as part of routine clinical care to create alleviation profiles; insufficient information can be data sets that more realistically represent the array derived from the pivotal clinical trials to develop of inputs that clinicians and patients assimilate in optimal strategies for the sequencing and timing of almost every clinical encounter. Electronic PRO these therapies.51 Registries of patients receiving instruments that are directly incorporated into routine care with these different agents (i.e., routine clinical care, and thus directly into an “real-world” registries), especially when EHR, are potentially important sources of PRO containing PRO data, can help inform sequencing, data for registry studies. Collection and analysis of timing, and impact of treatments, providing critical such data sets, in the form of registries, offers the information where there is an explosion of opportunity to inform clinical care in ways that are treatment options but a dearth of comparative meaningful to all stakeholders in the health care information. system. 2.3.2 Determining Effectiveness 2.3 Importance of PROs in Registries In registries designed to determine effectiveness, Given the centrality of PROs to CER and the role PROs also figure prominently, especially of registries in CER, the importance of PROs to considering the importance placed upon the patient registries is apparent. Inclusion of PROs in experience as a meaningful outcome in the IOM’s prospectively collected registries is almost always definition of CER. Beyond traditional outcome appropriate. PROs contribute information across measures such as overall survival and risk the spectrum of registry purposes described in reduction, QOL is a valid marker of efficacy by Chapter 1: describing the natural history of itself and is best captured by PRO measures. disease, determining effectiveness, measuring or Patient-reported symptoms can be indicators of monitoring safety or harm, and measuring quality. adverse consequences of therapy (e.g., toxicity As one reads down the list of nominated purposes monitoring), targets for meaningful intervention of registries, the substantive role of PROs in (e.g., symptom control intervention), and means of registry design becomes increasingly clear. understanding how patient perceptions of toxicities or effectiveness impact effectiveness (e.g., through adherence behavior). Consider a prospective

97 Section I. Creating Registries

registry intended to support CER for the 2.4 PROs in Prospective Registries management of early-stage prostate cancer. For Versus Retrospective Studies these patients, differentiating between and Having established the role for PROs across a comparing surgery and radiation is best achieved spectrum of registries, it is important to consider from patient-reported information on symptoms of the roles of PROs in prospective registries and radiation proctitis, sexual health, pain, and urinary retrospective studies. Patients’ experiences are function, as well as the relationship of these transient and are best captured “in the .” factors to overall QOL and patient preference. They cannot be recreated or recalled precisely—a Within the area of toxicity monitoring, PROs are limitation that highlights the need to routinely and likely to take a place on center stage. The National systemically capture PROs for prospective Cancer Institute has recently developed a patient- registries. Further, abundant evidence reported version of its Common Terminology demonstrates that third party assessments (most Criteria for Adverse Events, PRO-CTCAE,52 for notably those of clinicians) do not adequately use in cancer clinical trials. Pharmacovigilance reflect patients’ subjective experience with their studies provide another fertile area for PRO care.7, 8, 28, 56 For example, in patients with lung implementation. Perhaps even more powerful are cancer receiving chemotherapy, Basch and efforts to link PROs to genomic and proteomic colleagues showed that, when compared with data in order to understand the biologic basis for physician assessments, patient reports of toxicity phenotype. Registries intended for safety symptoms were more reflective of daily health monitoring offer potential for a much more robust status as measured by EuroQoL EQ-5D.33 As understanding of long-term safety than typical rapid-learning health care systems57-59 become clinical efficacy trials, and when coupled with data standard, routine capture of longitudinal and on effectiveness may help answer difficult systematic PROs will happen as part of routine questions such as “Was the intervention worth it?” care, thereby making it easier to prospectively especially as viewed through the patient’s lens. capture PROs for registry support. 2.3.3 Quality Measurement In contrast to prospective registries, which can be Registries intended to measure quality can designed to collect PROs as data accrue, studies incorporate PROs in numerous ways, and PROs constructed by manual chart extraction or from can contribute to quality assessment. In some EHR queries should not attempt to retrospectively instances, established quality standards do not add PRO data that was not originally collected. exist, and registries can be used to establish Additionally, researchers should not ask patients to realistic and acceptable standards. For example, provide recalled/recreated PROs for missing data there is an impetus to initiate quality monitoring in in such studies, as this may introduce recall bias. palliative medicine programs, but the evidence The exact length of time over which recall bias base is insufficient to establish benchmarks to develops is unclear, and seems to vary for different 60 define quality.53 In such a setting, registries experiences. For pain, single-item assessments incorporating PROs would serve an important role reflecting the prior week do not seem to represent in establishing definitions for quality, and could actual pain levels or a mean of daily pain levels 61 then be used in real-time to monitor quality. Some collected for the same 1-week period. Thus, quality metrics focused on the patient experience asking patients to precisely recall their symptom already exist. For example, in the American experience associated with a clinic visit at some Society of Clinical Oncology’s Quality Oncology arbitrary point in the past is fraught with pitfalls. Practice Initiative assessment and management of 2.5 Other General Considerations on pain, nausea/vomiting and dyspnea are core Inclusion of PROs in Registries metrics; this requires both PRO assessment and response to findings.54, 55 Including PROs in registries offers numerous advantages. First, incorporation of the patient voice helps keep care and research patient-

98 Chapter 5. Use of Patient-Reported Outcomes in Registries

centered, acknowledging the balance and tension collected within the context of clinical research between traditional outcomes and PROs. Further, should be used to inform care and change daily symptom burden, QOL, and satisfaction with care practice patterns. Without appropriate are dynamic variables that cannot be recreated infrastructure for responding to critical reports, accurately through retrospection; they are collection of PROs may pose a liability if critical essentially lost if not captured “in the moment.” data do not receive appropriate clinician attention For this reason, routine, systematic, and and response. For example, significant liability longitudinal collection is recommended and should could result if a patient reports a constellation of be a standard of practice. The importance of symptoms known to be strongly associated with longitudinal collection cannot be overstated; it suicidal behavior and there is inadequate clinical allows patients to serve as their own controls; that intervention. Further, it is possible that PROs is, each patient serves as his or her own could lead to decreased satisfaction with care if comparator over time. Changes from baseline are patients expect that their PROs will be reviewed tracked over time and linked to other interventions, and addressed, but they are unmet or such as initiation or discontinuation of a drug; or unacknowledged in the clinical encounter.29 to other outcomes, such as change in disease status (e.g., cancer progression, cardiac event). Serial 3. What Methods Are PROs address a number of critical issues. They: (1) improve our understanding of the trajectory of Available To Collect PROs and individual patient’s symptom burden and QOL Which Is Best? over the course of disease (or treatment); (2) remind clinicians of the variability between Often, choice of PRO instrument and of patients; (3) provide information on the value that administration are considered jointly; however, the individual patient places on their health state; they need not be, as administration methods and, (4) are central to the efforts of CER, simply provide a platform for collecting and pharmacovigilance studies, and quality presenting information. There are two main ways monitoring. When routine and systematic of collecting PRO data—on paper and collection of PROs is incorporated into registries, electronically. the health care community can improve the efficiency of routine care through support of 3.1 Paper-Based Methods billing and clinical documentation functions. Historically, PROs were collected via paper forms Certainly, including PROs in registries poses and were developed based on this collection challenges. Collection of PROs can generate method. From a practical standpoint, collection of significant amounts of data and adds another layer PRO data via paper-based methods is relatively of complexity to already complex data sets. straightforward. After selecting the instrument(s) Clinician acceptance may lag slightly for several to be used (discussed further in Section 4), reasons.62 Although the history (patient reports consistency is the guiding principle. Items should filtered through a clinician’s lens) and physical be presented in the same order for every exam are central to clinical diagnosis and collection. If the PRO measurement selected is a decisionmaking,63 long-standing and deeply single-item tool, this is automatic, but if multiple ingrained beliefs persist that clinician assessment instruments are employed, presenting them in the alone is objective and unbiased, casting doubt same order is important. Patients should complete upon the value and validity of unfiltered, direct forms in a confidential space, without fear that patient reports. Regardless, collection of PROs “wandering eyes” will see responses. Once forms generates more data for clinicians to consider and are completed, they should be reviewed multiple incorporate into care, which could be viewed as times for completeness. For those instruments onerous and burdensome, especially since PROs completed in clinic, this review should be done by are not yet ubiquitous or the standard of care. staff collecting the instruments, nurses involved in More importantly, it is largely unclear how PROs patient intake and rooming, and clinicians

99 Section I. Creating Registries

reviewing responses. Once forms are submitted to manual data entry, which is administratively the research team for data entry, completeness burdensome and subject to transcription errors. should be reassessed. Patients who fail to complete Manual entry also generates a lag time in a pre-defined percentage of questions (there is no monitoring response rates, complicating the consensus on an acceptable percentage), should process of reducing missing data.66 Overall, there receive a followup telephone inquiry to attempt to is a threshold beyond which the continuing data minimize missing data. Finally, data should be collection and quality assurance costs of paper- entered into electronic forms using double data based PROs surpass the upfront technology costs entry or similar techniques to enhance for electronic data capture, making electronic transcription accuracy. These techniques should PROs the more efficient and reliable approach. ideally be augmented with near real-time exploratory analyses to examine the believability 3.2 Electronic Capture Methods of the data within the clinical context.64 With the advent of portable and more cost- Paper forms are the historical gold standard for effective electronic capture methods, the presence PRO collection. For this reason, patients are of such methods within the literature has grown. inherently familiar with them. Their use is not Similar to traditional paper-based collection, limited by unfamiliarity or unease with new electronic collection begins with instrument(s) technologies, although unfamiliarity with new selection. Integral to the choice of instruments is technology dissipates quickly and patients are the choice of platform, as not all instruments are increasingly familiar with technology as advances tested across multiple platforms, nor is every continue to disseminate. Moreover, paper forms do instrument amenable to every platform. Electronic not require significant upfront capital investment, PRO (ePRO) capture has been demonstrated on a in terms of devices or software. There are many variety of platforms, including Web-based, measurement instruments across a variety of electronic tablets, interactive voice response disease states that have been extensively evaluated system (IVRS), handheld device, and digital pen. and are available for immediate use. For ePRO collection using tablet computers or handheld devices in the clinic setting, patients are However, paper forms have many limitations. They provided the device at the time of check-in to require research personnel to sort, distribute, and clinic with pre-loaded PRO measures such that collect, introducing risk for inconsistencies and a patients simply select their response to each item source of ongoing cost. Paper forms collected as as it is presented. With the digital pen, patients part of routine/scheduled clinic visits are generally select responses on a specially designed paper straightforward, but this approach systematically survey, with responses electronically recorded by misses participants unwilling or unable to attend a the pen. With IVRS, patients call a telephone clinic appointment. Collection between visits is number and are prompted, via an automated logistically difficult with paper forms; delivery of transcript, to select a preferred language, provide the paper forms either requires that participants an identifier and then are guided through the PRO take paper booklets home with them or that measure, providing verbal responses to each item. research personnel coordinate timely delivery of Access to Web-based platforms can be provided at booklets through the postal service. With either “confidential” computer stations in clinic waiting approach, obtaining a time/date stamp for at-home, rooms, or in the exam room itself, as well as from paper-based administration remains a challenge. any web-enabled device including home Relying on at-home paper booklets risks computers, handheld devices, and mobile participants completing multiple days of reporting 65 telephones. Regardless of platform, data are all at once (i.e., the so-called “parking lot” effect transmitted to a central, secure repository in which all responses for the past month are immediately upon submission and can be accessed completed immediately before a visit while sitting for “real-time” incorporation into routine care, if in the parking lot). Paper forms often include desired. Both Web-based and IVRS collection illegible or uninterpretable responses and require platforms can extend beyond the clinic and capture

100 Chapter 5. Use of Patient-Reported Outcomes in Registries

PROs between visits. Factors influencing platform trails and compliance with all FDA guidance for selection include budget and technical support, PROs. Alternatively, consider, for example, a technology literacy of the registry’s target health care system with an extensive EHR that population, collection logistics (in-clinic, between- plans a registry to monitor the impact of a series of visit, or combination), and the instrument(s) clinical pathways to lessen the debilitation chosen.2, 66 following major abdominal surgery. Such an Electronic methods of PRO capture have been organization may elect to develop or modify a widely shown to be feasible in a variety of practice PRO system to be directly integrated with their settings, disease states, and age ranges.29, 30, 67 EHR without involving an ePRO vendor. Recently developed PRO measures have either Compared with paper methods, delivery of ePROs been created specifically for electronic data can be automated, minimizing the risk of capture or include features to capitalize on inconsistent presentation of materials or electronic capture technologies, such as the mishandling paper forms. Electronic collection of Patient-Reported Outcomes Measurement responses provides immediate and accurate time/ Information System (PROMIS),68-70 the PRO- date stamps, and facilitates real-time monitoring of CTCAE,8, 52 and the Patient Care Monitor, version response rates and review for missing data.66 2 (PCM).71 The PROMIS and PRO-CTCAE tools Additionally, electronic platforms may provide a take advantage of electronic functionalities such as safer environment for patients to disclose sensitive skip logic or computerized adaptive testing, which concerns, such as sexual function.72 can reduce the number of items patients have to Not all PRO measures were developed for, or have complete, while the PCM also fulfills clinical been tested on, electronic administration documentation needs for clinical review of platforms. The transition of paper-based measures systems and provides triggers for accompanying to electronic platforms is referred to as patient education. “migration,” and guidelines were recently As to obtaining hardware or software for these developed to assess the equivalence of measures purposes, hardware often requires an upfront that have migrated from one collection mode to investment. Again, the size of the investment another.73 In general, paper-to-electronic migration depends largely upon the scope and scale of the yields between-mode equivalence comparable to registry. Some software packages are publicly the test-retest reliability of the original mode, but available (e.g., PROMIS Initiative items) while this is not always the case and should be tested.74 others are proprietary. Third-party commercial When incorporating a migrated PRO measure into vendors specializing in design and implementation a registry, registry developers should verify that of PROs offer a variety of products. The decision the ePRO measure has demonstrated validity in the to involve a commercial vendor depends upon intended mode of administration or reasonable factors like the rationale for including PROs in the equivalence with the mode for which validity, registry, the size of the registry and number of reliability, and sensitivity were initially involved sites, local technological expertise and demonstrated.39 support, whether the data will be collected as part Although electronic capture provides substantive of routine care or just for research purposes, and advantages over paper-based methods, enthusiasm the degree of psychometric analysis needed. must be tempered on several fronts. First, Although registry studies are not viewed as completion of electronically delivered PRO sufficiently rigorous for product labeling, measures requires some level of comfort with and exploratory analyses of PROs from a registry may access to newer technologies, which may prove serve as the basis for a subsequent trial for labeling challenging in certain situations. For example, in purposes, in which case having a sound PRO rural areas, using Web-based methods to collect measure in the registry could simplify the trial PROs between visits may be impractical due to process. In such a scenario, it would be prudent to unpredictable Internet access, while some geriatric use a commercial vendor to ensure adequate audit populations may be uncomfortable with tablet or

101 Section I. Creating Registries

handheld technologies. Second, if paper-electronic be considered, and testing should be conducted equivalence has not already been verified for a with the population planned for the registry. As migrated PRO instrument, the process of elaborated on http://www.usability.gov/, usability documenting equivalence can be time-consuming is not a single, one-dimensional property of the and expensive. Finally, electronic methods require interface, but rather a synthesis of the following greater up-front investment in terms of the devices elements: and software, electronic storage (meeting • Ease of learning: How fast can a user who has appropriate security standards), training, and never seen the user interface before learn it technical support. Depending upon the scale of the sufficiently well to accomplish basic tasks? registry, these issues may render electronic methods too burdensome. • Efficiency of use: Once experienced users have learned to use the system, how fast can they Software selection is a common question. While accomplish tasks? outside the scope of this chapter, some broad advice can be provided. First, there are many • Memorability: If a user has used the system companies that offer software to collect ePROs. before, can he or she remember enough to use Publically available software is also in production it effectively the next time or does the user have (e.g., PROMIS) or being developed (e.g., ePRO to start over again learning everything? CTCAE). The software solution itself is relatively • Error frequency and severity: How often do simple and expensive systems are not needed, users make errors while using the system? How unless specific features are required (e.g., the serious are these errors, and how do users requirement to be compliant with the FDA’s CFR recover from these errors? Part 11). Software should be from a credible vendor, with available security features that will • Subjective satisfaction: How much does the support compliance with applicable privacy and user like using the system? security laws. The degree of usability testing should match the In general, patients should report few items or complexity of the task. For an ePRO system, this ideally one item per screen, the screen should be process minimally includes documentation of clear and move to the next item when the answer is respondents’ ability to navigate the electronic provided, and there should not be any software platform, follow instructions, and answer delays between questions. Visually, the software questions, with an overall goal of demonstrating should present questions and response “buttons” in that respondents can complete the computerized large enough font for easy reading by mildly assessment as intended. Generally, fewer than ten visually impaired individuals. Validation code and representative patients are required to verify verifications should be built into the software, as usability. If the system is not usable, then it should well as any required clinical triggers. The software be iteratively updated until it is usable. should be easily adaptable, and easily integrated Feasibility extends usability and establishes the into the registry workflow. Reports (e.g., for practical implementation of the software system in clinicians) should be visually appealing, efficient, the local setting (e.g., clinic, home, hospital). and informative. Whenever possible, software Assessment approaches are similar and the should connect into the EHR workflow, including software goes through iterative updates until embedding data into the EHR for clinical feasible. During this process, patients can documentation and/or contributing to an enterprise contribute critical advice for the “help” manual data warehouse. and instruction sets. Finally, it is important to ensure that the software Although most often associated with questionnaire has been tested before full-scale implementation development, cognitive debriefing is also with the registry. Testing documentation should be appropriate for usability and feasibility assessment requested from the vendor, who should have through verbal probing by the interviewer (e.g., completed it. Both usability and feasibility should “What does the instruction ‘skip item’ mean to you

102 Chapter 5. Use of Patient-Reported Outcomes in Registries

here?”) and “thinking aloud” in which the suffice. When the registry is going to be large, interviewer asks the respondent to verbalize upfront investments in electronic approaches will whatever comes to mind as they conduct a task. realize substantial downstream gains in efficiency, Incorporated in usability and feasibility testing, cost, and data quality. Regardless of the ultimate cognitive debriefing helps to assess whether the choice of administration method, clear ePRO system influences the way respondents documentation of the rationale for the choice and interpret the questions, decide on an answer, and clear evidence of appropriate psychometric respond. In addition, it can help to determine assessment is strongly recommended. Assistance whether the instructions are clear or if anything is with this process may be available from internal confusing. expertise (as in many academic institutions) or may rely upon input from a commercial vendor, 3.3 Which Method Is Best? whose involvement can range from consulting only As with most other aspects of involving PROs in to nearly full control of the development and registries, the choice of PRO capture method is implementation process. highly dependent upon the design and purpose of the registry. Both paper-based and electronic 4. Which PRO Measure(s) platforms offer advantages and disadvantages, as Should Be Selected? outlined above. Ideally, when either method is shown to be valid for an instrument, both methods The process of choosing which PRO measure(s) to of PRO data collection should be available in a include in a registry can be challenging, largely study. Providing an interface familiar to or because the plethora of available measures is preferred by particular patients or populations may overwhelming. In 2007, a PubMed search for PRO reduce missing data not at random. Modes may be instrument development articles since 1995 mixed across patients in a study (e.g., each patient resulted in more than 2000 citations.75 selects a specific mode at baseline and continues to report via that mode throughout a study), or within Existing PRO measures assume a variety of forms: patients (e.g., a patient reports by Web until he • General assessment scales (e.g., health-related becomes symptomatically ill, at which point IVRS QOL) becomes preferable). One mode may be preferred • Disease-specific scales (e.g., chronic at a particular site, for example in multinational obstructive pulmonary disease, cancer studies where IVRS or Web access are [including scales for individual tumor types], heterogeneous across countries. “Real-world” arthritis, or psoriasis) registries are likely to enroll patients from a variety of settings (e.g., home, hospital, assisted living • Symptom-specific scales (e.g., pain, facility) and circumstances (e.g., independent, breathlessness, distress) caregiver-assisted), such that flexibility in mode of • Evaluations of functioning across a variety of administration facilitates capturing a broad mix of domains (e.g., physical, social, emotional) patients. Mixing modes is generally viewed as acceptable if a reasonable level of between-mode • Scales assessing satisfaction with care received equivalence has been demonstrated.39 • Other (e.g., adherence with therapy) In general, electronic capture is preferred to paper Some PRO measures are extensive, with dozens of because of its flexibility and its ability to reduce items related to a single concept (e.g., the chance that the PRO data in a registry will be breathlessness), while others have 80 or more missing. In contemporary research, paper methods items reflecting many different patient-reported are usually most cost-effective until registries start concerns constituting an entire clinical review of to grow in size or number of sites. When the systems, and yet others are single-item instruments registry is going to be intentionally small (e.g., measuring a single construct in a single question. fewer than 100 patients), paper methods will likely

103 Section I. Creating Registries

Further, there is extensive literature describing the As with any research activity, a priori specific aims important characteristics (i.e., conceptual and hypotheses to be tested must be outlined up framework, content validity, reliability, ability to front, and PRO selection appropriately aligned. detect change) of PRO measures, but consolidating Registry studies, in particular, are susceptible to this information into practical guidance for poorly defined outcomes; PRO instruments may be selecting among existing PRO measures is chosen because they are general in nature and difficult. The FDA guidance document has capture a broad range of patient-reported concerns, outlined a standard for evaluating PRO measures meeting a target goal of demonstrating that PROs for labeling claims that encompasses the salient are captured rather than capturing specific PRO points regarding development history, conceptual concepts of interest. If the objectives of the framework, and psychometric evaluation.10 The registry, intended hypotheses, and outcomes of standards outlined by the FDA may be more interest are clearly defined, the desired stringent than is necessary for certain registry characteristics of the PRO instrument become purposes, but nevertheless serve as an important more clearly delineated, facilitating a search of and well-conceived framework for discussion and existing measurement instruments. conform to accepted best practices.28 While a comprehensive review of PRO development and 4.2 Potential Sources for Identifying psychometric evaluation is beyond the scope of PRO Instruments this chapter, below is a concise overview of the Once these issues are clearly defined, identification process and concepts. For more information, of candidate PRO measures can begin in earnest. several texts provide detailed descriptions.26, 27, 76 In general, the process of PRO development is time- and resource-intensive and using existing 4.1 Getting Started and the measures whenever possible is best. It is highly Importance of Clarity unlikely that any existing instrument will perfectly The key to successfully navigating this process is suit the needs of a registry study, or that a to clearly define the following aspects of the “perfect” instrument can be developed, further registry: underscoring the importance of clearly defining • Population of interest (e.g., cancer patients the population, outcomes of interest, and purpose receiving radiotherapy for painful bony of the registry. Such clarity will allow more metastases, individuals with oxygen-dependent appropriate assessment of the relative strengths chronic obstructive pulmonary disease, and weaknesses of existing PRO measures. In children with rhinoconjunctivitis, U.S. veterans many cases, modifications to existing measures with rheumatoid arthritis) will improve the measure for use in a registry. These modifications can include changes in • Outcomes of interest, also known as the wording or order of questions, adding specific concept (e.g., specific symptom severity, overall questions, or altering the method of administration. symptom burden, treatment-related toxicities, In general, such modifications require some degree physical functioning, social functioning, QOL) of psychometric reassessment, though the degree • Intended users of the registry (e.g., clinicians, to which instrument modification requires patient advocacy groups, pharmaceutical psychometric reassessment varies and is discussed companies, insurance companies, governmental by Snyder et al.77 agencies) Traditional literature searches can yield results, but • The purpose(s) of the registry (e.g., may be quite time-consuming. The Mapi Institute pharmacovigilance, establishment of symptom maintains the Patient-Reported Outcome and trajectories, correlation of survival benefit with Quality of Life Instruments Database (http://www. QOL or symptom benefit). proqolid.org/), allowing users to search a large and relatively comprehensive database for PRO instruments that best address the specific needs

104 Chapter 5. Use of Patient-Reported Outcomes in Registries

identified. The Online Guide to Quality-of-life improvement over existing PRO instruments, Assessment (http://www.olga-qol.com/), is another concerns over regulatory acceptance, and limited database of existing QOL instruments. data about psychometric properties of item banks Additionally, the U.S. National Institutes of Health in specific populations.44 However, IRT-based item PROMIS Initiative (hhtp://www.nihpromis.org/) banks represent a promising approach, especially has been tasked with developing rigorously tested in light of the emphasis on limiting respondent item banks across a broad range of domains and burden. subdomains (functioning, disability, symptoms, distress, and role participation).68 The PROMIS 4.3 Choice of the Best PRO for the Initiative is also actively evaluating methods to Registry achieve brevity in instruments through techniques Section 4.4 describes many of the properties of such as computer adaptive testing. Importantly, PRO instruments that should be considered when these measures are publicly available through the choosing the appropriate instrument for each PROMIS Assessment Center (http://www. unique registry scenario. Whether to adhere assessmentcenter.net/). Commercial vendors can closely to the conservative FDA recommendations also aid in identifying appropriate measures; as is a frequent question, if not a source of frank with selecting a mode for administering the PRO tension. While there is no formal avenue through measure, the decision to involve a commercial which registries can support product-labeling vendor is multifactorial, depending on the factors claims, if the registry is in any way tied to trials described in Section 3.2. with aspirations of product-labeling, then the Item banks represent another option for developing answer is straightforward and the FDA PRO PRO surveys. In general, item banks contain guidance should be followed. Anchoring the FDA comprehensive collections of items that pertain to threshold as a “maximally conservative” (and a particular construct (e.g., dyspnea).44 Item banks therefore usually least practical) state, there is a generally rely on item response theory (IRT), in continuum of scenarios and a continuum of which the unit of focus is the item rather than the practical allowances to the ideal state where the entire instrument. As such, instruments can be need for precision and reduction of bias is constructed using IRT that employ only those balanced with the need for practical solutions and items which provide the most useful and relevant the reduction of missing data. Figure 5–1 shows information, eliminating questions with little that the tension between psychometric desirability added value, without compromising psychometric and logistical considerations of PRO collection in qualities.78 The PROMIS Initiative is an example registries requires a careful balance, driven of an item bank.44 Item banks may represent the primarily by the goals of the registry. Explicitly future of PRO collection, but they are currently outlining the registry objectives, population, limited by logistical issues, questions about outcomes, and intended uses as described in whether IRT-based item banks represent an Section 4.1 will help to define where the registry is on the continuum and guide decisionmaking.

105 Section I. Creating Registries

Figure 5–1. Psychometric properties and logistical considerations exist along a spectrum

EHR = Electronic Health Record; FDA = U.S. Food and Drug Administration.

4.4 Development History and imperative, from an FDA and product-labeling Conceptual Framework standpoint, that the entire development history be well documented. The cornerstone of the The PRO development history and conceptual development history is the conceptual framework. framework are inextricably linked and are discussed in close proximity for this reason. 4.4.2 Conceptual Framework 4.4.1 Development History Clear identification of the target population, purpose of the registry, and outcomes of interest The FDA guidance document strongly greatly facilitates developing a conceptual recommends transparency with respect to framework. According to the FDA Guidance development history. “Development history” document, a conceptual framework “explicitly explicitly refers to the entire process of developing defines the concepts measured by the instrument in and psychometrically evaluating a PRO measure, a diagram that presents a description of the including the conceptual framework, item relationships between items, domain development and revision history, and evidence of (subconcepts), and concepts measured and the patient input. For newly developed PRO scores produced by a PRO instrument.”10 Initially, instruments, clearly documenting the development the conceptual framework arises out of expert history is straightforward and can be integrated opinion and literature review. The framework is into the development process. This is in contrast to then refined by qualitative methods of patient using an existing measure, where the development input, such as patient interviews and focus groups, history may be very difficult, if not impossible, to which ensures that a priori hypotheses are obtain. Ideally, the development history is well consistent with patient experiences and vetted in the literature, but if the history is descriptions. The conceptual framework will be somewhat opaque, the FDA has indicated that modified iteratively.79 For complex concepts, such demonstration of content validity with specific as breathlessness, multiple domains affect the examples, including direct patient input from the overall concept, so identifying appropriate appropriate population, is an acceptable domains and then assessing these is paramount to alternative. For newly developed PROs, it is assessing the overarching concept.

106 Chapter 5. Use of Patient-Reported Outcomes in Registries

4.5 Psychometric Properties important that an instrument assess those concepts it was designed to measure. In general, qualitative Entire texts are written on and there evidence, in the form of documented patient input is an extensive literature on psychometric through focus groups, is an important standard in properties of PRO measures. An excellent series the view of the FDA.42 Construct validity arising from the Mayo/FDA Patient-Reported describes the degree to which what was measured Outcomes Consensus Group focused on PRO reflects the a priori conceptualization of what development in advance of the anticipated FDA should be measured.41 Subcomponents of guidance; it was published in a special supplement construct validity are convergent and discriminant of the November/December 2007 issue of the validity, which assess the degree of similarity journal Value in Health and provides more detailed between measures that are theoretically similar descriptions of processes and procedures needed to (convergent validity) or the extent to which implement PRO systems to meet FDA measures that are theoretically different actually expectations.80 differ (discriminant validity). For example, a new Almost every guideline regarding the use of PRO measure of anxiety would be expected to have high measures recommends selecting measures that convergent validity with the anxiety subscale of have demonstrated content validity, criterion the Hospital Anxiety and Depression Scale.82 To validity, reliability, and sensitivity (including the that end, the FDA would expect comparisons of ability to detect change over time) in the target new PRO measures with similar existing measures population.39 It is important to note that to support construct validity. Criterion validity psychometric properties are not dichotomous and describes the extent to which the scores of PRO that instruments are not completely “valid” or measure reflect the gold standard measure of the “reliable.” These properties are continuous same concept.10 Criterion validity is often difficult variables relaying incremental information. to assess in the PRO arena because it is difficult to Additionally, it is inappropriate to refer to an identify gold standard measures for many PRO instrument as “validated,” as this simply means it concepts. The FDA therefore deemphasizes has been subjected to psychometric analysis, but criterion validity. conveys no information regarding the measure’s 4.5.2 Reliability performance.42 For this reason, instruments are reflected at varying points on our continuum in Reliability reflects the ability of an instrument to Figure 5–1 to demonstrate that differing states of yield the same result on serial administrations reliability and validity may be appropriate when no change in the concept being measured is depending upon the context of the registry and the expected. The reliability of an instrument is PROs to be captured within it. The goal, ultimately, typically assessed via test-retest methods and by is to identify or develop instruments with measuring the internal consistency.42 Accordingly, acceptable psychometric properties in the two aspects of reliability can be distinguished: population of interest. • Test-retest reliability describes the ability of an 4.5.1 Validity instrument to generate the same results in the same respondent over a period of time during From a psychometric standpoint, validity has three which no change is reasonably expected.2, 10, 42 main forms: content, construct, and criterion Thus, test-retest reliability assesses the intra- validity. Content validity is the extent to which the individual variability. Identifying the optimal instrument actually measures the concepts of timeframe for retesting can be challenging, and interest. The FDA guidance document10 may vary by disease state and target understandably places significant emphasis on population.42 content validity, consistent with other groups,81 even stating that without adequate content validity, • Internal consistency reliability reflects the labeling claims cannot be supported. At face value, degree to which items within a scale measure the importance of content validity is intuitive; it is the same concept. It can be quantitatively

107 Section I. Creating Registries

assessed with Cronbach’s alpha, which Consider a registry of patients with advanced measures the internal consistency of an cancer designed to assess the impact of certain instrument. Well-established thresholds for interventions upon the development of disability. interpreting Cronbach’s alpha are available; in Upon entering the registry, a patient rates his general, coefficient alpha greater than 0.7 is the disability as severe because his reference point is a minimum acceptable threshold for comparisons previously healthy state. Four months later, he between groups.42 rates his disability as mild, though on more 4.5.3 Ability To Detect Change open-ended questioning, he notes that he can simply sit on the front porch and watch his The ability of a PRO measure to detect change is grandchildren, as he knows that any other activities intuitively important. Demonstration of this ability, are unrealistic and that his goal is to simply make according to the FDA, requires that changes in the it to the front porch. Even though the instrument PRO instrument parallel changes in other factors measures disability from the view of the patient that indicate a change in the status of the concept and would thus have adequate content validity, the of interest. For example, in patients receiving a interpretation regarding the merits of the new treatment for opioid-induced constipation, intervention would be erroneous, as the patient has changes in a PRO instrument designed to assess clearly become more disabled, but has shifted his overall bowel health may be linked with use of frame of reference, a fact which is not captured by certain other bowel products, such as enemas, to content validity. This phenomenon is commonly establish the ability to detect change. The measure referred to as “response shift” and has long been must demonstrate ability to detect both recognized as a challenge in QOL research.84 improvements and losses in health status. Further, Alternatively, all measures with marginal content it is important to detect changes throughout the validity may be cast aside without consideration of range of possible values. In registry studies, where other properties. Consider two new measures for longitudinal collection and analysis are critical, the same concept tested in different studies with understanding the concept of minimally important different methodologies, resulting in different change detected,83 rather than establishing that content validities. The measure with higher content number explicitly, may be sufficient. validity is likely to propagate, even if it is more 4.5.4 Areas of Controversy flawed, simply because of methodological issues. The emphasis placed upon content validity has These arguments on content validity are not generated some controversy as PRO developers intended to undermine the standards established by attempt to improve content validity, in part by the FDA, nor should they be viewed as rationale meticulously wording items and instructions to for not adhering to these standards, but are meant minimize variations in interpretation between to prompt careful consideration of all the patients. However, the ability to improve content psychometric properties of PRO measures, validity is likely asymptotic, in that individual especially in the context of the specific registry. variability undoubtedly influences interpretation of Remember first principles—before anything else, questions in ways that cannot be accounted for, it needs to make good sense, have face validity, be meaning that responses to an instrument capture doable, and limit patient burden. the patient’s true (and unique) perceptions. There are concerns that in the pursuit of greater content 4.6 Non-Psychometric Considerations validity, other important characteristics of PRO Beyond identifying a PRO instrument with instruments may be underdeveloped or desirable psychometric properties, consideration underappreciated.41 For example, in pursuing must be given to the people that are closely tied to greater content validity, the constraints placed completing and acting upon PRO data and the upon questions may actually limit patient tension that can exist between impacts on people perspective by forcing some degree of conformity, and psychometric desirability. or may result in misinterpretation of results.

108 Chapter 5. Use of Patient-Reported Outcomes in Registries

4.6.1 Patient Factors 4.6.2 Clinician Factors In designing registries and considering PROs for Even within the research setting, assessing the inclusion, it is important to consider the burden to impact of PRO collection on routine care is the patient the PRO measures represent. For important. Will the PRO results be made available instance, lengthy questionnaires may result in immediately as part of routine care or only increasing missing data over time, as patients grow available to research personnel? Whether or not weary of serially completing such questionnaires. PRO data are shared with clinicians in real time The capacity to answer lengthy instruments cannot should be explicitly addressed in the informed be predicted a priori and differs between groups. consent process. If data are to be made available to At Duke Cancer Institute, patients in a variety of clinicians, are appropriate support services solid tumor clinics routinely complete 80-86 item available to assist in managing newly identified instruments without significant fatigue or concerns or issues? Are there mechanisms to burnout;7, 71 time to complete the survey is support incorporation of PRO data into clinical 11 minutes, reducing to under 8 minutes after care, if it will be made available, or will it be “one several visits in the clinic using the same more thing” for which clinicians are responsible? instrument. While the FDA did not offer specific What will be the impact of the PRO collection on recommendations on questionnaire length, a workflow? guidance document from the Center for Medical Many recent guidelines recommend providing Technology Policy recommends that, for patients clinician feedback regarding patient-reported with cancer, completion of PRO instruments take information of concern, such as reports of new no more than 20 minutes at the initial visit and 39 chest pain. The thresholds for triggering a clinical fewer than 10 minutes at subsequent visits. alert, components of the alert message, and Patients should be offered a private space for method of delivering the notice to the clinician completing instruments, to minimize concerns must be carefully considered. What are the risk regarding confidentiality, especially for sensitive management concerns? How will the clinician’s questions. Instructions should be provided for response be verified? Though often mundane, every item, even if only to frame the recall period. these factors are important to consider in the The instrument should be delivered with adequate implementation phase. Teams experienced in font size and at appropriate literacy levels. embedding PROs into registries and clinical Additionally, physical assistance should be workflow can provide sage advice as to how to provided if needed, such as reading items aloud to navigate these pathways (e.g., Duke Cancer Care patients with visual impairments. While most pilot Research Program, http://www.cancer.duke.edu/ studies of PRO instruments provide a small 29, 71 dccrp/); clear guidelines do not exist. See further amount of remuneration, these studies have discussion in Section 4.7. demonstrated that the collection of PROs made patients feel encouraged that their clinicians were 4.6.3 Ensuring Data Quality seeking additional information and felt that the Collecting quality data is an implicit necessity of ePRO instrument facilitated communication any registry. Although assessing data quality can between patient and clinician.29 Outside the pilot assume many forms, for the purposes of registries, testing phase, it is not advisable to provide two concepts are critical. The first is to minimize remuneration to patients for completing PRO missing data. Missing data inevitably undermine instruments, even in the setting of a registry study. the quality of the information collected, thereby PRO responses should be shared with clinicians, as decreasing its analytic potential. It is essential to this has been shown to be an important aspect of anticipate missing data and to plan interventions to PROs to patients.29 minimize the problem. This is especially important in registry studies where time horizons may be long and the potential for missing data great. There

109 Section I. Creating Registries

are a number of steps that can be taken to properties, and scoring algorithms. In such a minimize missing data during the implementation scenario, it is imperative that the version of the phase of the registry. The most important step is to instrument completed at any given point in time be make sure that the PRO instrument chosen is identified within the data set. Further, there may be meaningful, and that its role in the registry and cases where the person completing the related work is well described, especially to questionnaire may not always be the patient (see patients and families. Ideally, the PRO measures discussion in Section 4.6.4.). For example, in a should be implemented as standard of care, such palliative care registry, patients are not always able that they become ubiquitous and desired, not only to complete a PRO instrument, even with by patients but also by clinicians.58 If this occurs, assistance. The ability of the person to complete the amount of missing data should decrease. the instrument may change over time as cognition Electronic data collection practically supports wanes. In these settings, proxy-reports involving real-time, or near-real-time, quality monitoring of close family or caregivers may become the only information being collected in order to identify available measures and the only available data to patterns of missing data, leading to development be incorporated into registries; therefore, it is of targeted interventions to reduce missing data. essential to identify, via metadata, who is Additionally, with near-real-time quality analysis, completing the instrument. backup data collection methods can—and 4.6.4 Special Populations: Are Proxy-Reports should—be deployed. For example, a central Ever Appropriate? telephone interviewer can contact individuals who did not respond to items (either individually or There are numerous situations in which patients entire instruments) to both obtain the data and are not physically or cognitively able to provide ascertain why the item was omitted. Analytic direct assessment of their experience. Obvious approaches must include a plan for managing the examples include infants and small children, unavoidable occurrence of missing data; individuals with significant cognitive impairment importantly, a “last observation carried forward” (congenital or acquired), and those at the end of approach to handling missing data should be life. In such settings, proxy-reports of QOL are avoided. often collected,20 though the literature suggests that proxy-reports demonstrate moderate The second issue related to data quality is agreement, at best, with patient-reports.85-87 consistency. In registries with long time horizons, Nevertheless, proxy-reports are viewed as valuable it is not uncommon for measurement items, or in many of these settings because caregiver or instruments, to evolve or change entirely. family perception is also an important Unfortunately, it is equally uncommon for consideration. The FDA strongly discourages notations of such changes to be embedded within proxy-reports in product-labeling claims.10 the data structure as metadata, such that future Unfortunately, such an extreme stance leaves these analyses can quickly and readily identify which vulnerable populations marginalized. By not iteration of an instrument was completed at which considering proxy-reports, symptom-based point in time. Metadata is essentially data about research and other lines of inquiry in these data. More precisely, it is “…structured populations face considerable obstacles with a information that describes, explains, locates, or potential end-result that drugs or products that otherwise makes it easier to retrieve, use, or 45 could improve symptom burden or QOL never manage an information source.” Consider a have the opportunity to gain FDA approval for long-term registry where the primary measurement such indications. The FDA’s position on proxy- instrument undergoes an iterative update to reports is emphasized because of the rigorous “version 2” to reflect new knowledge in the field standard the FDA guidance document establishes, and is quickly implemented into the registry. but that position should not devalue the potential Though the two versions are likely very similar, role for proxy-reports. Ideally, the extent of they also likely have slightly different questions (in agreement between patient- and proxy-reports can terms of structure or order), psychometric

110 Chapter 5. Use of Patient-Reported Outcomes in Registries

be established in advance of use of proxy-reports. translate an instrument into another language, as The PROMIS Initiative is investigating application the psychometric properties obtained within an of existing methods for PROs to proxy-reports to American population of patients with disease X improve performance.88, 89 are unlikely to be reproduced in a population of Japanese patients with the same disease. Thus, 4.7 Implementation Issues formal assessment of the psychometric properties Upon successful navigation of the challenging of the instrument is necessary when translating to process of selecting PRO instruments and their another language. mode of administration comes the daunting task of Another aspect of consistency in this setting implementing the selected instruments. Below is a reflects administering the same instrument over the practical framework for successful lifespan of the registry. The strength of this implementation, centered on achieving data quality recommendation depends partly upon the purpose and consistency. of the registry; for registries comparing Just as with mode of administration, implementing effectiveness, this consistency is essential, while PRO data collection is best achieved if consistency for a registry focused on quality and embedded is a central tenet, especially if the registry study is within an EHR, this recommendation is less multicenter. In this setting, consistency refers to stringent. Nevertheless, if the data are collected processes. Standard operating procedures should prospectively, the strong preference is for be established for each site of data collection that consistency in PRO instrument administered. delineate, to the extent possible, how patients, Regardless of purpose, collected data should researchers, and clinicians interact with the include metadata labels. collection system (paper or electronic). As part of Further, involving the entire health care team standard operating procedures, specific training (physicians, mid-level providers, nurses, should be provided, with accessible and easy-to- administrators, and other support staff) in the use manuals available (preferably in both text and development process is essential, especially with video format). Every aspect of the process that can respect to integrating the PRO instruments into the be standardized should be standardized, including clinical workflow and providing clinician the data set itself. That is, the data sets should feedback. As part of this integration, clinical include metadata that describe key components triggers should be established (and standardized) important for subsequent analyses and end-users, that explicitly force acknowledgement of a patient including who completed the instrument (patient report by a provider (e.g., a pain score of 8 out of or proxy), where it was completed (e.g., outpatient 10) or initiate some standardized intervention (e.g., clinic, home, inpatient ward), which version was a patient reporting a high distress level might be administered, and a flag for irregularities identified automatically contacted by a psychosocial care as part of internal . support team).88 Such standardized triggers will Ideally, for multisite studies these standard only be embraced if there is inclusion of the health operating procedures are the same at each site, care team in the implementation process. This with another set of standard operating procedures inclusive implementation process will also help for the central repository (or coordinating) site that shape the perception of the PRO data, in that delineates how often data from cooperating sites buy-in from the health care team will make the should be transmitted, how it should be compiled PRO collection process a necessary and desired and stored, how often it should undergo quality component of care, rather than simply an extra assessment, and how it should be accessed and task to complete.88 distributed for analysis. Within multisite registries, Finally, explicitly including the patient voice in the and even within some single-site registries, it may form of PROs has been shown to improve patient be necessary to select an instrument that has been well-being and enhance patient-provider translated into and validated in other languages communication.35 Building on this premise, besides English. It is not adequate to simply inclusion of PROs in observational studies may

111 Section I. Creating Registries

improve patient engagement, recruitment, and instruments and administration methods. If this retention, though there are no data directly process is navigated effectively, the stage will be supporting this. The experience of the Duke set for successful incorporation of PROs into the Cancer Care Research Program with ePRO registry. collection as part of routine cancer care has shown After the arduous process of clearly defining the remarkable response and participation rates, with population and outcomes of interest is complete, rates of missing data routinely less than 5 percent, researchers should search for existing PRO even for sensitive questions such as level of sexual instruments that will assess the outcomes of enjoyment. Certainly, more rigorous interest. (See Case Examples 8, 9, 10, and 11.) If a documentation of improved long-term patient suitable measure is not identified, options include participation with inclusion of PROs is needed modifying an existing measure or developing a before more ardent assertions can be made. new measure. (See Case Example 8.) In general, development of new PRO instruments is resource- 4.8 Summary Regarding Selecting PRO intensive, so it is preferable to use an existing Instruments measure whenever possible. After identifying (or Selecting PRO instruments for inclusion in registry developing) a measure, administration mode studies is not a one-size-fits-all process. The should be selected. Electronic administration is Center for Medical Technology Policy prepared a preferred, but not all instruments have been guidance document for inclusion of PROs in adult evaluated using electronic administration, though oncology trials39 and these recommendations are this can be accomplished. Important to the included as an example (Table 5–2). Clear and scientific basis of the registry are the psychometric careful definition of the target population, concept properties of the instrument. While the FDA to be measured, and purpose of the registry is an highly values content validity, it is possible to important first step. For a given population or effectively use an instrument with modest content context, even in a registry, it is important to have validity, depending on the purpose of the registry, some a priori hypotheses and justification for highlighting the importance of understanding and outcomes being measured, or the study risks defining the purpose of the registry. becoming a prospective fishing expedition. As such, there needs to be a systematic approach to In most registry studies, the purposes of the study selecting salient outcomes (to the extent possible and outcomes of interest will necessitate inclusion in a registry, which admittedly is sometimes of PRO data. Careful planning is essential, in exploratory by nature). In CER, the process of identifying appropriate PRO instruments for identifying meaningful outcomes requires upfront inclusion, selecting modes of instrument patient input. But regardless of how the outcomes administration, and implementing the PRO are selected, there must be a systematic approach collection system. When carried out effectively, to determining whether an outcome is best this planning process generally produces more reported by a patient (i.e., if information about a complete data sets that truly include the voices of particular symptom or overall health state or all stakeholders in the health care system and are satisfaction is sought, it is best reported from the meaningful to all stakeholders. patient/surrogate perspective; thus, a PRO instrument is appropriate). Far too frequently, the 5. Example of PRO Use in a tail wags the dog in registry studies; that is, PRO Registry instruments are selected first, prior to identifying outcomes of interest. Thus, the rational Consider the division of pulmonary medicine at an identification of outcomes of interest early in the academic university. Within the division there is a process of registry development is important. Such growing multidisciplinary cystic fibrosis (CF) an approach will quickly enable the researchers to program with a large catchment area and determine if PROs are appropriate, and will approximately 250 patients ranging in age from 21 produce a sound base for evaluating PRO to 65 years, though most patients are younger than

112 Chapter 5. Use of Patient-Reported Outcomes in Registries

age 35. As the program develops, the team plans to Because the team plans to use this registry in a implement a series of initiatives for patients with longitudinal fashion for numerous planned CF, targeting not only improved survival, but also interventions and wants to understand how specific improved functioning. Proposed interventions interventions affect certain domains impacted by include routine endocrinology consultations for all CF, it selects an established, multi-item, multi- CF-related diabetes mellitus, improved domain, CF-specific measure that incorporates an psychological services, and standardized exercise aggregate assessment of QOL, as well as several regimens during hospitalizations. The outcomes of component domains of well-being. interest for these interventions are equally broad- Since the planned settings of intervention include ranging, but include traditional measures such as both inpatient and outpatient settings, and since pulmonary function (as measured by pulmonary there are travel issues related to the catchment function tests), end-organ damage (diabetes, area, the team also plans to capture reports chronic kidney disease), resistant organism between visits, such that no more than two months colonization rates, hospitalization utilization, elapse between sessions of PRO data collection. symptom burden (including breathlessness, weight For this reason, the team prefers to use electronic change, worry, and fatigue) and quality of life methods, but the instrument it selected has only (QOL). The team plans to use a registry for this been psychometrically assessed via paper-based project because they do not feel that they can methods. The team collaborates with the reasonably test the effectiveness of these institutional expert on PROs to document paper- interventions through parallel or sequential electronic equivalence, and to perform usability randomized, controlled trials, but do wish to and feasibility testing for Web-based systematically capture outcomes of interest in a administration. This pilot study demonstrates that longitudinal manner as the interventions are it is reasonable to use a Web-based approach for introduced. PRO assessments. In considering the outcomes of interest, symptom From a health-care team standpoint, burden and health-related QOL merit closer implementation goes smoothly, since the entire CF inspection for inclusion of PROs. Certainly, team was involved in developing the registry and patients are better positioned than clinicians to PRO system. Missing data are minimal for report breathlessness, worry, fatigue, and QOL. In inpatient and clinic appointment collection, as the fact, most argue that patients are the only valid team heavily advertised the PRO collection system source of information on these issues, thus to the patients prior to implementation, provided inclusion of PROs in this registry is appropriate. in-clinic teaching, and used reports during the In considering which instruments to use, it is clinical visits; the instrument quickly becomes important for the team to consider the relationships viewed as a necessary component to the health between the symptoms under consideration and care encounter. However, as data accumulates, the QOL. Figure 5–2 illustrates some of the team identifies patterns of missing data for relationships that exist around health-related QOL between-visit administrations. It determines that in CF. Specifically, the influence of symptom at-home Internet access remains a problem for a burden on QOL must be weighed carefully, to help small but significant portion of patients. It receives determine if a series of single-item instruments is a grant from the local CF foundation to support most appropriate or if a multi-item, disease- Internet access for vulnerable patients, with specific instrument (of which there are several in subsequent reduction in missing data. CF) or another approach is most appropriate.

113 Section I. Creating Registries

Figure 5–2. Simplified concept map for cystic fibrosis

, ,

,

This example highlights several key points: (1) the importance of understanding the target population; (2) the need to identify outcomes of interest prior to selecting PRO instruments as the outcomes of interest should dictate the instrument, not vice versa; and (3) the benefit of incorporating PRO instruments into longitudinal, routine care.

114 Chapter 5. Use of Patient-Reported Outcomes in Registries

Case Examples for Chapter 5

Case Example 8. Developing and validating a Study, and a sample of men enrolled in the patient-administered questionnaire observational registry. Description The Benign Prostatic Using the data from the sample populations, the Hypertrophy (BPH) Registry and team conducted a series of analyses to develop the Patient Survey was a multicenter, scale. The team used to help select prospective, observational the items from the scale that had the highest registry examining the patient correlations with the principal factors. Using management practices conventional validation, the team examined of primary care providers and reliability (both internal consistency and test- urologists, and assessing patient retest repeatability). To assess validity, tests of outcomes, including symptom repeatability and discriminant/convergent validity amelioration and disease were used to determine that the short form progress. The registry collected successfully discriminated between men with no patient- to mild LUTS/BPH and those with moderate to reported and clinician-reported severe LUTS/BPH. Lastly, the team examined the data at multiple clinical visits. correlation between the 7-item ejaculation domain of the 25-item MSHQ and the new short-form Sponsor sanofi-aventis scale, using data from the observational registry. Year Started 2004 Results Year Ended 2007 Based on the results of these analyses, the team No. of Sites 403 selected three ejaculatory function items and one ejaculation bother item for inclusion in the new No. of Patients 6,928 MSHQ-EjD Short Form. The new scale Challenge demonstrates a high degree of internal consistency and reliability, and it provides information to Lower urinary tract symptoms associated with identify men with no to mild LUTS/BPH and benign prostatic hyperplasia (LUTS/BPH) have a those with moderate to severe LUTS/BPH. strong relationship to sexual dysfunction in aging males. Sexual dysfunction includes both erectile Key Point dysfunction (ED) and ejaculatory dysfunction Developing new instruments for collecting (EjD), and health care providers treating patients patient-reported outcomes requires careful testing with symptoms of BPH should evaluate men for of the new tool in representative populations to both types of dysfunction. Providers can use the ensure validity and reliability. Registries can Male Sexual Health Questionnaire (MSHQ), a provide a large sample population for validating validated, self-administered, sexual function scale, new instruments. to assess dysfunction, but the 25-item scale can be For More Information perceived as too long. To assess EjD more efficiently, it was necessary to develop a brief, Rosen RC, Catania JA, Althof SE. et al. patient-administered, validated questionnaire. Development and validation of four-item version of Male Sexual Health Questionnaire to assess Proposed Solution ejaculatory dysfunction. Urology. The team used representative, population-based 2007;69(5):805–9. samples to develop a short-form scale for Rosen R, Altwein J, Boyle P. et al. Lower urinary assessing EjD. The team administered the 25-item tract symptoms and male sexual dysfunction: the MSHQ to three populations: a sample of men Multinational Survey of the Aging Male. Eur from the Men’s Sexual Health Population Survey, Urol. 2003;44:637–49. a subsample of men from the Urban Men’s Health

115 Section I. Creating Registries

Case Example 9. Using validated measures to To achieve the study goals, the registry needed to collect patient-reported outcomes collect health-related data directly from Description The Study to Help Improve Early participants in such a way that the data would be evaluation and management of reliable, valid, and comparable across participant risk factors Leading to Diabetes groups and over time. (SHIELD) is a household panel Proposed Solution registry designed to assess the The investigators decided to use validated prevalence and incidence of patient-reported outcomes measures (PROs) to diabetes mellitus and collect information on health status and cardiovascular disease; disease behaviors. The PROs allowed the data from the burden and progression; risk registry to be compared with data collected in predictors; and knowledge, other registries to assess the generalizability of attitudes, and behaviors data on the study population. In addition, the regarding health in the U.S. PROs already took into account issues such as population. The study involves recall bias and interpretability of the questions, three distinct phases: an initial and self-administered instruments eliminated the screening survey, a baseline possibility of introducing interviewer bias. survey, and yearly followup surveys for 5 years. The registry included seven PROs: (1) the 12-item Short Form Health Survey (SF-12) and Sponsor AstraZeneca Pharmaceuticals LP European Quality of Life (EuroQoL) EQ-5D Year Started 2004 instrument, to assess health-related quality of Year Ended 2009 life; (2) the Sheehan Disability Scale, to assess the level of disruption in work, social life, and No. of Sites Not applicable family/home life; (3) the 9-item Patient Health No. of Patients More than 211,000 individuals Questionnaire, to assess depression; (4) the Work were included in the screening Productivity and Activity Impairment survey; approximately 15,000 Questionnaire: General Health, to assess work individuals were followed for 5 productivity and absenteeism; (5) the Diet and years. Health Knowledge Survey; (6) the Press-Ganey Satisfaction questionnaire; and (7) the Challenge International Physical Activity Questionnaire, to The SHIELD registry used survey methodologies assess health-related physical activity and to collect health information from a large sample sedentary behaviors. of adults. The goal of the study was to capture The investigators considered many factors, such participants’ perspectives and views on diabetes as length, ease of use, format, and scoring and cardiovascular disease, risk factors for the system, when selecting the PROs to include in diseases, and burden of the diseases. The study the survey. For example, a major reason for investigators, noting that treatment for diabetes selecting the SF-12 rather than the SF-36 as a and cardiovascular disease relies heavily on measure of quality of life was the length of the patient self-management, felt that it was forms (12 vs. 36 items). The survey is entirely particularly important to gather information on paper-based, with participants mailing back activities, weight control, health attitudes, quality completed forms. The validated scoring of life, and other topics directly from the algorithms are used to account for missing or participant, without a physician as an illegible values on the completed forms. All intermediary. The investigators also wanted to participants must be able to read and write follow participants over time to better understand English. disease progression and changes in health behaviors or activities.

116 Chapter 5. Use of Patient-Reported Outcomes in Registries

Case Example 9. Using validated measures to followup rate should be stated a priori so that collect patient-reported outcomes (continued) response rates can be better interpreted with Results respect to their potential for introducing bias. The registry had a generally high response rate For More Information for the surveys. The response rates were 63.7 Gavin JR III, Rodbard HW, Fox KM, et al. percent for the screening survey, 71.8 percent for Association of overweight and obesity with health the baseline survey, and between 71 and 75 status, weight management, and exercise percent for the annual surveys. In terms of behaviors among individuals with type 2 diabetes missing data, participants who return the survey mellitus or with cardiometabolic risk factors. forms tended to complete all of the questions in Risk Management and Healthcare Policy the appropriate manner. However, the registry is 2009;2:1−7. missing longitudinal data from some participants. Grandy S, Chapman RH, Fox KM, et al. Quality For example, a participant may have returned the of life and depression of people living with type 2 completed form in 2005, failed to return the form diabetes mellitus and those at low and high risk in 2006, and returned the form again in 2007. The for type 2 diabetes: findings from the Study to investigators must account for the missing 2006 Help Improve Early evaluation and management values when conducting longitudinal analyses. of risk factors Leading to Diabetes (SHIELD). The data from the survey have been sufficient to Int J Clin Pract 2008;62:562–8. support comparisons over time and across participant groups, leading to several Grandy S, Fox KM. EQ-5D visual analog scale publications. and utility index values in individuals with diabetes and at risk for diabetes: findings from Key Point the Study to Help Improve Early evaluation and Utilization of standardized, validated instruments management of risk factors Leading to Diabetes in a registry can offer many benefits, including (SHIELD). Health Qual Life Outcomes enhanced scientific rigor, the ability to compare 2008;6:18. patient views over time, and the ability to Grandy S, Fox KM, Bazata DD, et al. Association compare registry data with data from other of self-reported weight change and quality of life, sources to assess the representativeness of the and exercise and weight management behaviors registry population. It should be noted that among adults with type 2 diabetes mellitus. significant initial planning is necessary to identify Cardiol Res Pract 2012;2012:892564. appropriate PROs, obtain the necessary permissions, and include them in a registry. Rodbard HW, Bays HE, Gavin JR III, et al. Rate Issues with missing data must be considered in and risk predictors for development of self- the planning phases for a registry. This registry reported type 2 diabetes mellitus over a 5-year considered missing data within returned survey period: the SHIELD study. Int J Clin Pract questionnaires. In addition, an acceptable 2012;66:684-691.

117 Section I. Creating Registries

Case Example 10. Challenges in the collection minimizing incomplete or missing PRO of PROs in a longitudinal registry questionnaires. Description A longitudinal registry of men Proposed Solution with metastatic castrate-resistant Investigators are using a registry to evaluate prostate cancer is being longitudinal PRO data among men with conducted among men receiving metastatic castrate-resistant prostate cancer outpatient care at the Memorial (mCRPC). mCRPC is characterized by disease Sloan-Kettering Cancer Center, progression (continued elevated prostate-specific Oregon Health and Science antigen [PSA] or radiographic progression) University, and John Hopkins regardless of first-line androgen depletion School of Medicine. therapy. Many patients with bone metastases Sponsor U.S. Department of Defense experience debilitative symptoms, such as bone Year Started 2012 pain, in addition to treatment toxicities. The purpose of the registry is to collect PRO and Year Ended Ongoing clinical data to inform the development of pain No. of Sites 3 endpoints for future oncology clinical trials that conform to regulatory standards of the FDA and No. of Patients Planned enrollment is 400 men the European Medicines Agency. The registry with castrate-resistant prostate does not seek to develop a new PRO, but instead cancer to advance methods for administering and Challenge interpreting PRO results, as well as to validate a PRO measure of analgesic medication use. Regulatory and government agencies and cancer Specifically, the goals of the registry are to organizations, including the U.S. Food and Drug identify (1) the clinical significance of pain score Administration (FDA), National Cancer Institute, changes, (2) the average time to pain progression, and the American Cancer Society, recommend (3) the proportion of men with pain starting new collecting patient-reported outcomes (PRO) data lines of treatment, (4) appropriate recall periods to capture cancer patients’ perspectives on and for pain assessment, and (5) the comparison of experiences of their symptoms, disease status and methods for quantifying analgesic use. functioning, and health-related quality of life. Collecting PRO data is important for diseases Results such as cancer, in order to fully evaluate the Registry participants will include 400 patients benefit and risk profile of potentially toxic with metastatic castrate-resistant prostate cancer oncology treatments. To this end, industry who are receiving outpatient care between 2012 sponsors and the FDA wish to include symptom and 2014 at one of the three institutions. Clinical endpoints in clinical trials, but currently lack data, including diagnosis, treatment, and resource sufficient information about optimal methods to utilization, will be abstracted from medical design robust endpoints. Many challenges to records every 3 months. Patient-reported data, designing PRO endpoints exist, such as including pain, analgesic use, and other identifying appropriate and validated instruments, symptoms, will be collected every 6 weeks by an ensuring interpretability of the data’s clinical automated telephone survey. A key feature of this significance, and having information on the registry is the use of a single centralized survey variability of symptoms in order to accurately platform that includes a phone survey completed determine necessary sample sizes. The collection by patients and a Web interface through which of PRO data presents additional challenges, study staff at participating sites can enter patient including identifying the optimal mode(s) of medical record data on a quarterly basis. The administration, minimizing patient burden, and integrated system of data collection is intended to

118 Chapter 5. Use of Patient-Reported Outcomes in Registries

Case Example 10. Challenges in the collection evaluate PRO instrument characteristics and of PROs in a longitudinal registry (contiued) collect data necessary to develop PRO endpoints Results (continued) for use in clinical trials. Addressing operational barriers such as mode of administration, reduce the burden of data management. The instrument length and frequency, and missing registry is designed in a way which addresses the PRO data will improve patient-reported data for challenges of collecting longitudinal PRO data, use in endpoint development, clinical trials, taking into account (1) the importance of treatment decisionmaking, and routine patient electronic PRO assessment and choosing the best care. mode (e.g., phone, Web-based) of assessment, (2) the importance of choosing the optimal For More Information frequency and length of each assessment, and Basch E, Abernethy AP. Supporting clinical (3) the use of automated reminders, clear practice decisions with real-time patient-reported instruction sheets, and survey questions relevant outcomes. J Clin Oncol. 2011 Mar 10; 29(8): to patients to ensure high survey completion 954-6. rates. Bennett AV, Jensen RE, Basch EM. Electronic Key Point patient-reported outcome systems in oncology To ensure appropriate clinical interpretation and clinical practice. CA Cancer J Clin. 2012 Sep- quality of PRO data, registries can be used to Oct;62(5):337-47.

Case Example 11. Collecting PRO data in a The Department of Psychiatry and Behavioral sensitive patient population Neurosciences at Cedars Sinai Medical Center Description The Cedars-Sinai Psychiatric enrolled consecutive patients presenting for Treatment Outcome Registry psychiatric evaluation in a patient registry. (CS-PTR) is a single-site patient Demographic information, DSM-IV diagnosis, registry that tracks the outcomes and current psychiatric comorbidities were of psychiatric interventions in a obtained by the provider using structure naturalistic clinical setting using interviewing (the Mini International measurement-based care and Neuropsychiatric Interview). Patients completed patient-reported outcomes. “self-assessment questionnaires” during their baseline visit and during quarterly followup Sponsor Cedars Sinai Medical Center visits. Validated patient-reported outcome (PRO) Year Started 2005 tools included questionnaires that collected information on depressive symptom severity Year Ended 2012 (Quick Inventory of Depressive No. of Sites 1 Symptomatology), functioning (Work and Social Adjustment Scale), and QOL (Quality of Life No. of Patients 2,600 Enjoyment and Satisfaction Questionnaire). Challenge The registry often encountered significant Psychiatric disorders are strongly associated with barriers to obtaining self-reported data from grave impairments in functioning and quality of psychiatric patients. For example, the baseline life (QOL), but most previous research has and followup questionnaires took approximately focused on symptom improvement and has not 20 to 30 minutes for patients to complete, and specifically investigated the extent to which many patients were resistant to spending that treatment can improve functioning and QOL amount of time completing the questionnaires, as outcomes. they did not see the value in completing them.

119 Section I. Creating Registries

Case Example 11. Collecting PRO data in a ethnicity, or had been diagnosed with mood sensitive patient population (continued) disorders. Baseline analysis also showed that Proposed Solution although symptom severity and functional impairments are significantly correlated with Staff at the clinic educated the patients about the lower reported levels of QOL, they only value of the self-assessment questionnaires. They explained a moderate amount of the in explained to patients that the results of the QOL. The findings point to the critical need to go questionnaires would be used to inform their beyond symptom severity monitoring and include providers’ decisions about diagnoses, appropriate functioning and QOL measures during the course treatment, and treatment progress. The staff of assessment, treatment, and research of implemented an appointment scheduling system psychiatric disorders. Analysis of followup data is that built in a 30-minute block of time before ongoing. patients were seen by a provider, to allow time for them to complete the self-assessment Key Point questionnaires. For patients who had trouble PROs are important tools for informing treatment completing the written questionnaire decisions. In patient populations where it is independently, clinicians worked with them to particularly difficult to obtain PRO data, help them complete the questionnaire verbally operational steps, such as discussing the benefits and recorded the answers themselves. During of PROs, changing appointment time frames, and quarterly followup visits, clinicians were offering support in completing PRO tools from expected to review the answers and PRO scores clinicians or other trained professionals, can be with patients, including any trends in symptom taken to minimize the burden on patients. severity, functioning, or QOL changes following Regularly reviewing PRO data with patients and treatment initiation. informing patients of the value of PRO data to Results their treatment may increase patients’ participation. A total of 2,600 patients were enrolled in the registry over the course of seven years. At For More Information baseline, patients reported a wide range of IsHak WW, Balayan K, Bresee C, et al. A symptom severity, which is to be expected given descriptive analysis of quality of life using the consecutive enrollment of patients in the patient-reported measures in major depressive registry with no exclusion criteria. Psychiatric disorder in a naturalistic outpatient setting. Qual patients tended to report severely low QOL Life Res. 2013 Apr;22(3):585-96. levels, especially if they were older, of Hispanic

References for Chapter 5 4. Flynn KE, Lin L, Ellis SJ, et al. Outcomes, health policy, and managed care: relationships between 1. Fairclough DL. Design and Analysis of Quality of patient-reported outcome measures and clinical Life Studies in Clinical Trials. Chapman & Hall/ measures in outpatients with heart failure. Am CRC Interdisciplinary Statistics. 2nd ed: Chapman Heart J. 2009 Oct;158(4 Suppl):S64-71. and Hall/CRC; 2010. p. 424. PMID: 19782791. PMCID: 2805910. 2. Lipscomb J, Gotay CC, Snyder CF. Patient- 5. Cleeland CS, Sloan JA, Group AO. Assessing the reported outcomes in cancer: a review of recent Symptoms of Cancer Using Patient-Reported research and policy initiatives. CA Cancer J Clin. Outcomes (ASCPRO): searching for standards. 2007 Sep-Oct;57(5):278-300. PMID: 17855485. J Pain Symptom Manage. 2010 Jun;39(6): 1077-85. PMID: 20538189. 3. Hewlett SA. Patients and clinicians have different perspectives on outcomes in arthritis. J Rheumatol. 2003 Apr;30(4):877-9. PMID: 12672220.

120 Chapter 5. Use of Patient-Reported Outcomes in Registries

6. Bushmakin AG, Cappelleri JC, Taylor-Stokes G, et 14. Cunningham WE, Crystal S, Bozzette S, et al. The al. Relationship between patient-reported disease association of health-related quality of life with severity and other clinical outcomes in survival among persons with HIV infection in the osteoarthritis: a European perspective. J Med United States. J Gen Intern Med. 2005 Econ. 2011;14(4):381-9. PMID: 21574904. Jan;20(1):21-7. PMID: 15693923. PMCID: 1490035. 7. Pakhomov SV, Jacobsen SJ, Chute CG, et al. Agreement between patient-reported symptoms 15. Mapes DL, Lopes AA, Satayathum S, et al. and their documentation in the medical record. Health-related quality of life as a predictor of Am J Manag Care. 2008 Aug;14(8):530-9. mortality and hospitalization: the Dialysis PMID: 18690769. PMCID: 2581509. Outcomes and Practice Patterns Study (DOPPS). Kidney Int. 2003 Jul;64(1):339-49. 8. Basch E, Iasonos A, McDonough T, et al. Patient PMID: 12787427. versus clinician symptom reporting using the National Cancer Institute Common Terminology 16. Gotay CC, Kawamoto CT, Bottomley A, et al. The Criteria for Adverse Events: results of a prognostic significance of patient-reported questionnaire-based study. Lancet Oncol. 2006 outcomes in cancer clinical trials. J Clin Oncol. Nov;7(11):903-9. PMID: 17081915. 2008 Mar 10;26(8):1355-63. PMID: 18227528. 9. Stacy M, Bowron A, Guttman M, et al. 17. Victorson D, Soni M, Cella D. Metaanalysis of the Identification of motor and nonmotor wearing-off correlation between radiographic tumor response in Parkinson’s disease: comparison of a patient and patient-reported outcomes. Cancer. 2006 Feb questionnaire versus a clinician assessment. Mov 1;106(3):494-504. PMID: 16353212. Disord. 2005 Jun;20(6):726-33. PMID: 15719426. 18. Rolfson O, Karrholm J, Dahlberg LE, et al. 10. U.S. Food and Drug Administration. Guidance for Patient-reported outcomes in the Swedish Hip Industry: Patient Reported Outcome Measures: Arthroplasty Register: results of a nationwide Use in Medical Product Development and prospective observational study. J Bone Joint Surg Labeling Claims. December 2009. http://www.fda. Br. 2011 Jul;93(7):867-75. PMID: 21705555. gov/downloads/Drugs/GuidanceCompliance 19. Castellano D, del Muro XG, Perez-Gracia JL, et al. RegulatoryInformation/Guidances/UCM193282. Patient-reported outcomes in a phase III, pdf. Accessed August 15, 2012. randomized study of sunitinib versus interferon- 11. Norekval TM, Fridlund B, Rokne B, et al. Patient- {alpha} as first-line systemic therapy for patients reported outcomes as predictors of 10-year with metastatic renal cell carcinoma in a European survival in women after acute myocardial population. Ann Oncol. 2009 Nov;20(11):1803-12. infarction. Health Qual Life Outcomes. PMID: 19549706. PMCID: 2768734. 2010;8:140. PMID: 21108810. PMCID: 3004873. 20. Thornburg CD, Calatroni A, Panepinto JA. 12. Wang XS, Shi Q, Lu C, et al. Prognostic value of Differences in health-related quality of life in symptom burden for overall survival in patients children with sickle cell disease receiving receiving chemotherapy for advanced nonsmall hydroxyurea. J Pediatr Hematol Oncol. 2011 cell lung cancer. Cancer. 2010 Jan 1;116(1):137- May;33(4):251-4. PMID: 21516020. 45. PMID: 19852033. PMCID: 3148693. 21. Molnar-Varga M, Molnar MZ, Szeifert L, et al. 13. Singh JA, Nelson DB, Fink HA, et al. Health- Health-related quality of life and clinical related quality of life predicts future health care outcomes in kidney transplant recipients. Am J utilization and mortality in veterans with self- Kidney Dis. 2011 Sep;58(3):444-52. reported physician-diagnosed arthritis: the PMID: 21658828. veterans arthritis quality of life study. Semin 22. Pronzato P, Cortesi E, van der Rijt CC, et al. Arthritis Rheum. 2005 Apr;34(5):755-65. Epoetin alfa improves anemia and anemia-related, PMID: 15846592. patient-reported outcomes in patients with breast cancer receiving myelotoxic chemotherapy: results of a European, multicenter, randomized, controlled trial. Oncologist. 2010;15(9):935-43. PMID: 20798194. PMCID: 3228044.

121 Section I. Creating Registries

23. Abernethy AP, McDonald CF, Frith PA, et al. 33. Basch E, Jia X, Heller G, et al. Adverse symptom Effect of palliative oxygen versus room air in event reporting by patients vs clinicians: relief of breathlessness in patients with refractory relationships with clinical outcomes. J Natl dyspnoea: a double-blind, randomised controlled Cancer Inst. 2009 Dec 2;101(23):1624-32. trial. Lancet. 2010 Sep 4;376(9743):784-93. PMID: 19920223. PMCID: 2786917. PMID: 20816546. PMCID: 2962424. 34. Berry DL, Blumenstein BA, Halpenny B, et al. 24. Deshpande AD, Sefko JA, Jeffe DB, et al. The Enhancing patient-provider communication with association between chronic disease burden and the electronic self-report assessment for cancer: a quality of life among breast cancer survivors in randomized trial. J Clin Oncol. 2011 Mar Missouri. Breast Cancer Res Treat. 2011 10;29(8):1029-35. PMID: 21282548. Oct;129(3):877-86. PMID: 21519836. PMCID: 3068053. PMCID: 3250926. 35. Velikova G, Booth L, Smith AB, et al. Measuring 25. Sprangers MA. Disregarding clinical trial-based quality of life in routine oncology practice patient-reported outcomes is unwarranted: Five improves communication and patient well-being: a advances to substantiate the scientific stringency randomized controlled trial. J Clin Oncol. 2004 of quality-of-life measurement. Acta Oncol. Feb 15;22(4):714-24. PMID: 14966096. 2010;49(2):155-63. PMID: 20059312. 36. Detmar SB, Muller MJ, Schornagel JH, et al. Role 26. Lipscomb J, Gotay CC, Snyder C. Outcomes of health-related quality of life in palliative assessment in cancer. Cambridge Univ Pr; 2005. chemotherapy treatment decisions. J Clin Oncol. p. 662. 2002 Feb 15;20(4):1056-62. PMID: 11844830. 27. Streiner DL, Norman GR. Health Measurement 37. National Quality Forum. A National Framework Scales: A practical guide to their development and and Preferred Practices for Palliative and Hospice use. 4th ed: Oxford University Press; 2008. p. 428. Care Quality: A Consensus Report. Oxford University Press, USA; 2006:124. www. 28. Basch E, Iasonos A, Barz A, et al. Long-term qualityforum.org/Publications/2006/12/A_ toxicity monitoring via electronic patient-reported National_Framework_and_Preferred_Practices_ outcomes in patients receiving chemotherapy. for_Palliative_and_Hospice_Care_Quality.aspx. J Clin Oncol. 2007 Dec 1;25(34):5374-80. Accessed August 17, 2012. PMID: 18048818. 38. Curtis JR, Jain A, Askling J, et al. A comparison 29. Abernethy AP, Herndon JE, 2nd, Wheeler JL, et al. of patient characteristics and outcomes in selected Feasibility and acceptability to patients of a European and U.S. rheumatoid arthritis registries. longitudinal system for evaluating cancer-related Semin Arthritis Rheum. 2010 Aug;40(1):2-14 e1. symptoms and quality of life: pilot study of an e/ PMID: 20674669. PMCID: 2958101. Tablet data-collection system in academic oncology. J Pain Symptom Manage. 2009 39. Center for Medical Technology Policy (CMTP). Jun;37(6):1027-38. PMID: 19394793. Effectiveness Guidance Document: Recommendations for Incorporating Patient- 30. Basch E, Artz D, Dulko D, et al. Patient online Reported Outcomes into the Design of Post- self-reporting of toxicity symptoms during Marketing Clinical Trials in Adult Oncology. May chemotherapy. J Clin Oncol. 2005 May 2012. www.cmtpnet.org/wp-content/uploads/ 20;23(15):3552-61. PMID: 15908666. downloads/2012/05/PRO-EGD.pdf. Accessed 31. Snyder CF, Jensen R, Courtin SO, et al. August 20, 2012. PatientViewpoint: a website for patient-reported 40. Willke RJ, Burke LB, Erickson P. Measuring outcomes assessment. Qual Life Res. 2009 treatment impact: a review of patient-reported Sep;18(7):793-800. PMID: 19544089. outcomes and other efficacy endpoints in approved PMCID: 3073983. product labels. Control Clin Trials. 2004 32. Dudgeon DJ, Knott C, Chapman C, et al. Dec;25(6):535-52. PMID: 15588741. Development, implementation, and process 41. McClimans LM, Browne J. Choosing a patient- evaluation of a regional palliative care quality reported outcome measure. Theor Med Bioeth. improvement project. J Pain Symptom Manage. 2011 Feb;32(1):47-60. PMID: 21110121. 2009 Oct;38(4):483-95. PMID: 19699607.

122 Chapter 5. Use of Patient-Reported Outcomes in Registries

42. Frost MH, Reeve BB, Liepa AM, et al. What is 53. Kamal AH, Currow DC, Ritchie C, et al. The sufficient evidence for the reliability and validity value of data collection within a palliative care of patient-reported outcome measures? Value program. Curr Oncol Rep. 2011 Aug;13(4): Health. 2007 Nov-Dec;10 Suppl 2:S94-S105. 308-15. PMID: 21556703. PMID: 17995479. 54. Neuss MN, Desch CE, McNiff KK, et al. A 43. Revicki DA, Osoba D, Fairclough D, et al. process for measuring the quality of cancer care: Recommendations on health-related quality of life the Quality Oncology Practice Initiative. J Clin research to support labeling and promotional Oncol. 2005 Sep 1;23(25):6233-9. claims in the United States. Qual Life Res. PMID: 16087948. 2000;9(8):887-900. PMID: 11284208. 55. Campion FX, Larson LR, Kadlubek PJ, et al. 44. Flynn KE, Dombeck CB, DeWitt EM, et al. Using Advancing performance measurement in item banks to construct measures of patient oncology: quality oncology practice initiative reported outcomes in clinical trials: investigator participation and quality outcomes. J Oncol Pract. perceptions. Clin Trials. 2008;5(6):575-86. 2011 May;7(3 Suppl):31s-5s. PMID: 21886517. PMID: 19029206. PMCID: 2662709. PMCID: 3092462. 45. NISO. Understanding Metadata. Bethesda, MD: 56. Fromme EK, Eilers KM, Mori M, et al. How NISO Press; 2004. p. 1-20. www.niso.org/ accurate is clinician reporting of chemotherapy publications/press/UnderstandingMetadata.pdf. adverse effects? A comparison with patient- Accessed July 16, 2013. reported symptoms from the Quality-of-Life Questionnaire C30. J Clin Oncol. 2004 Sep 46. The World Health Organization Quality of Life 1;22(17):3485-90. PMID: 15337796. assessment (WHOQOL): position paper from the World Health Organization. Soc Sci Med. 1995 57. Abernethy AP, Etheredge LM, Ganz PA, et al. Nov;41(10):1403-9. PMID: 8560308. Rapid-learning system for cancer care. J Clin Oncol. 2010 Sep 20;28(27):4268-74. 47. Institute of Medicine. Committee on Comparative PMID: 20585094. PMCID: 2953977. Effective Research Prioritization. Initial National Priorities for Comparative Effectiveness Research. 58. Abernethy AP, Herndon JE, 2nd, Wheeler JL, et al. Washington DC: National Academy Press; 2009. Improving health care efficiency and quality using p. 227. tablet personal computers to collect research- quality, patient-reported data. Health Serv Res. 48. Wu AW, Snyder C, Clancy CM, et al. Adding the 2008 Dec;43(6):1975-91. PMID: 18761678. patient perspective to comparative effectiveness PMCID: 2613994. research. Health Aff (Millwood). 2010 Oct;29(10):1863-71. PMID: 20921487. 59. Etheredge LM. A rapid-learning health system. Health Aff (Millwood). 2007 Mar-Apr;26(2): 49. Hirsch BR, Giffin RB, Esmail LC, et al. w107-18. PMID: 17259191. Informatics in action: lessons learned in comparative effectiveness research. Cancer J. 2011 60. Stone AA, Broderick JE, Schwartz JE. Validity of Jul-Aug;17(4):235-8. PMID: 21799331. average, minimum, and maximum end-of-day recall assessments of pain and fatigue. Contemp 50. Patient-Centered Outcomes Research Institute. Clin Trials. 2010 Sep;31(5):483-90. www.pcori.org/patient-centered-outcomes- PMID: 20620239. research. Accessed August 15, 2012. 61. Stone AA, Broderick JE, Kaell AT. Single 51. Plimack ER, Hudes GR. Selecting targeted momentary assessments are not reliable outcomes therapies for patients with renal cell carcinoma. for clinical trials. Contemp Clin Trials. 2010 J Natl Compr Canc Netw. 2011 Sep 1;9(9):997- Sep;31(5):466-72. PMID: 20580945. 1006; quiz 7. PMID: 21917624. PMCID: 3014612. 52. Basch EM, Reeve BB, Mitchell SA, et al. 62. Bainbridge D, Seow H, Sussman J, et al. Electronic toxicity monitoring and patient- Multidisciplinary health care professionals’ reported outcomes. Cancer J. 2011 Jul- perceptions of the use and utility of a symptom Aug;17(4):231-4. PMID: 21799330. assessment system for oncology patients. J Oncol Pract. 2011 Jan;7(1):19-23. PMID: 21532805. PMCID: 3014504.

123 Section I. Creating Registries

63. Paley L, Zornitzki T, Cohen J, et al. Utility of 72. Dupont A, Wheeler J, Herndon JE, 2nd, et al. Use clinical examination in the diagnosis of emergency of tablet personal computers for sensitive patient- department patients admitted to the department of reported information. J Support Oncol. 2009 medicine of an academic hospital. Arch Intern May-Jun;7(3):91-7. PMID: 19507456. Med. 2011 Aug 8;171(15):1394-6. 73. Coons SJ, Gwaltney CJ, Hays RD, et al. PMID: 21824956. Recommendations on evidence needed to support 64. Day S, Fayers P, Harvey D. Double data entry: measurement equivalence between electronic and what value, what price? Control Clin Trials. 1998 paper-based patient-reported outcome (PRO) Feb;19(1):15-24. PMID: 9492966. measures: ISPOR ePRO Good Research Practices Task Force report. Value Health. 2009 65. Rand CS, Wise RA, Nides M, et al. Metered-dose Jun;12(4):419-29. PMID: 19900250. inhaler adherence in a clinical trial. Am Rev Respir Dis. 1992 Dec;146(6):1559-64. 74. Gwaltney CJ, Shields AL, Shiffman S. PMID: 1456575. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome 66. Rose M, Bezjak A. Logistics of collecting measures: a meta-analytic review. Value Health. patient-reported outcomes (PROs) in clinical 2008 Mar-Apr;11(2):322-33. PMID: 18380645. practice: an overview and practical examples. Qual Life Res. 2009 Feb;18(1):125-36. 75. Turner RR, Quittner AL, Parasuraman BM, et al. PMID: 19152119. Patient-reported outcomes: instrument development and selection issues. Value Health. 67. Salaffi F, Gasparini S, Grassi W. The use of 2007 Nov-Dec;10 Suppl 2:S86-93. computer touch-screen technology for the PMID: 17995478. collection of patient-reported outcome data in rheumatoid arthritis: comparison with 76. Fayers P, Machin D. Quality of Life: The standardized paper questionnaires. Clin Exp Assessment, Analysis and Interpretation of Rheumatol. 2009 May-Jun;27(3):459-68. Patient-reported Outcomes. 2nd ed: Wiley; 2007. PMID: 19604439. p. 566. 68. Cella D, Yount S, Rothrock N, et al. The Patient- 77. Snyder CF, Watson ME, Jackson JD, et al. Reported Outcomes Measurement Information Patient-reported outcome instrument selection: System (PROMIS): progress of an NIH Roadmap designing a measurement strategy. Value Health. cooperative group during its first two years. Med 2007 Nov-Dec;10 Suppl 2:S76-85. Care. 2007 May;45(5 Suppl 1):S3-S11. PMID: 17995477. PMID: 17443116. PMCID: 2829758. 78. Cella D, Riley W, Stone A, et al. The Patient- 69. Garcia SF, Cella D, Clauser SB, et al. Reported Outcomes Measurement Information Standardizing patient-reported outcomes System (PROMIS) developed and tested its first assessment in cancer clinical trials: a patient- wave of adult self-reported health outcome item reported outcomes measurement information banks: 2005-2008. J Clin Epidemiol. 2010 system initiative. J Clin Oncol. 2007 Nov Nov;63(11):1179-94. PMID: 20685078. 10;25(32):5106-12. PMID: 17991929. PMCID: 2965562. 70. Reeve BB, Burke LB, Chiang YP, et al. Enhancing 79. Wilson IB, Cleary PD. Linking clinical variables measurement in health outcomes research with health-related quality of life. A conceptual supported by Agencies within the US Department model of patient outcomes. JAMA. 1995 Jan of Health and Human Services. Qual Life Res. 4;273(1):59-65. PMID: 7996652. 2007;16 Suppl 1:175-86. PMID: 17530449. 80. Sloan JA, Halyard MY, Frost MH, et al. The Mayo 71. Abernethy AP, Zafar SY, Uronis H, et al. Clinic Manuscript Series Relative to the Validation of the Patient Care Monitor (Version Discussion, Dissemination, and Operationalization 2.0): a review of system assessment instrument for of the Food and Drug Administration Guidance on cancer patients. J Pain Symptom Manage. 2010 Patient-Reported Outcomes. Value in health : the Oct;40(4):545-58. PMID: 20579839. journal of the International Society for Pharmacoeconomics and Outcomes Research. 2007;10:S59-S63.

124 Chapter 5. Use of Patient-Reported Outcomes in Registries

81. Terwee CB, Bot SD, de Boer MR, et al. Quality 86. Kutner JS, Bryant LL, Beaty BL, et al. Symptom criteria were proposed for measurement properties distress and quality-of-life assessment at the end of health status questionnaires. J Clin Epidemiol. of life: the role of proxy response. J Pain 2007 Jan;60(1):34-42. PMID: 17161752. Symptom Manage. 2006 Oct;32(4):300-10. PMID: 17000347. 82. Snaith RP. The Hospital Anxiety And Depression Scale. Health Qual Life Outcomes. 2003;1:29. 87. Jones JM, McPherson CJ, Zimmermann C, et al. PMID: 12914662. PMCID: 183845. Assessing agreement between terminally ill cancer patients’ reports of their quality of life and family 83. Norman GR, Sloan JA, Wyrwich KW. caregiver and palliative care physician proxy Interpretation of changes in health-related quality ratings. J Pain Symptom Manage. 2011 of life: the remarkable universality of half a Sep;42(3):354-65. PMID: 21454041. . Med Care. 2003 May;41(5):582-92. PMID: 12719681. 88. Basch E, Abernethy AP. Supporting clinical practice decisions with real-time patient-reported 84. Sprangers MA, Schwartz CE. The challenge of outcomes. J Clin Oncol. 2011 Mar 10;29(8): response shift for quality-of-life-based clinical 954-6. PMID: 21282536. oncology research. Ann Oncol. 1999 Jul;10(7):747-9. PMID: 10470418. 89. Cleeland CS, Wang XS, Shi Q, et al. Automated symptom alerts reduce postoperative symptom 85. Varni JW, Thissen D, Stucky BD, et al. severity after cancer surgery: a randomized PROMIS(R) Parent Proxy Report Scales: an item controlled clinical trial. J Clin Oncol. 2011 Mar response theory analysis of the parent proxy report 10;29(8):994-1000. PMID: 21282546. item banks. Qual Life Res. 2012 Sep;21(7): PMCID: 3068055. 1223-40. PMID: 21971875.

125

Chapter 6. Data Sources for Registries

1. Introduction • Patient identifiers—Some registries may use patient identifiers to link data. In these Identification and evaluation of suitable data registries, data elements are linked to the sources should be completed within the context of specific patient through a unique patient the registry purpose and availability of the data of identifier or registry identification number. The interest. A single registry may have multiple use of patient identifiers may not be possible in purposes and integrate data from various sources. all registries due to the additional legal While some data in a registry are collected directly requirements that usually apply to the use and for registry purposes (primary data collection), disclosure of such data. (See Chapter 7.) important information also can be transferred into • Patient selection criteria—The eligibility the registry from existing databases. Examples criteria in a registry protocol or study plan include demographic information from a hospital determine the group that will be included in the admission, discharge, and transfer system; registry. These criteria may be very broad or medication use from a pharmacy database; and restrictive, depending on the purpose. Criteria disease and treatment information, such as details often include demographics (e.g., target age of the coronary anatomy and percutaneous group), a disease diagnosis, a treatment, or coronary intervention from a catheterization diagnostic procedures and laboratory tests. laboratory information system, electronic medical Health care provider, health care facility or record, or medical claims databases. In addition, system, and insurance criteria may also be observational studies can generate as many included in certain types of registries (e.g., hypotheses as they test, and secondary sources of following care patterns of specific conditions at data can be merged with the primary data large medical centers compared with small collection to allow for analyses of questions that private clinics). were unanticipated when the registry was conceived. • Treatments and tests—Treatments and tests are necessary to describe the natural history of This chapter will review the various sources of patients. Treatments can include both primary and secondary data, comment on pharmaceutical, biotechnology, or device their strengths and weaknesses, and provide some therapies, or procedures such as surgery or examples of how data collected from different radiation. Evaluation of the treatment itself is sources can be integrated to help answer important often a primary focus of registries (e.g., questions. treatment safety and effectiveness over 5 years). Results of laboratory testing or diagnostic 2. Types of Data procedures may be included as registry outcomes and may also be used in defining a The types of data to be collected are guided by the diagnosis or condition of interest. registry design and data collection methods. The form, organization, and timing of required data are • Confounders—Confounders are elements or important components in determining appropriate factors that have an independent association data sources. Data elements can be grouped into with the outcomes of interest. These are categories identifying the specific variable or particularly important because patients are construct they are intended to describe. One typically not randomized to therapies in framework for grouping data elements into registries. Confounders such as comorbidities categories follows: (disease diagnoses and conditions) can confuse analysis results and interpretation of causality. Information on the health care provider,

127 Section I. Creating Registries

treatment facility, concomitant therapies, or Patient identifiers—Depending on the data sources insurance may also be considered. Unknown required, some registries may use certain personal confounders, or those not recorded in the identifiers for patients in order to locate them in registry, pose particular challenges for the other databases and link the data. For example, analysis of patient outcomes. If external, or Social Security numbers (SSNs) in combination linked, data sources may provide values for with other personal identifiers can be used to these confounder variables otherwise not in the identify individuals in the National Death Index registry, they may ultimately help reduce bias (NDI). Patient contact information, such as in the analysis and interpretation of patient address and phone numbers, may be collected to outcomes. support tracking of participants over time. • Outcomes—The focus of this document is on Information for additional contacts (e.g., family patient outcomes. Outcomes are end results and members) may be collected to support followup in are defined for each condition. Outcomes may cases where the patient cannot be reached. In many include patient-reported outcomes (PROs). In cases, patient informed consent and appropriate some registries, surrogate markers, such as privacy authorizations are required so that personal biomarkers or other interim outcomes (e.g., identifiers can be used for registry purposes, and hemoglobin A1c levels in diabetes) that are the use of personal identifiers may not be possible highly reflective of the longer term end results in some registries; Chapter 7 discusses the legal are used. requirements for including patient identifiers. Systems and processes must be in place to manage Before considering the potential sources for security and confidentiality of these data. registry data, it is important to understand the Confidentiality can be enhanced by assigning a types of data that may be collected in a registry. registry-specific identifier via a crosswalk Several types of data that may be gathered from algorithm, as discussed below. Demographics, other sources in some registries are described such as date of birth (to calculate age at any time below. point), gender, and ethnicity, are typically collected Cost/resource utilization—Cost and/or resource and may be used to stratify the registry population. utilization data may be necessary to examine the Disease/condition—Disease or condition data cost-effectiveness of a treatment. Resource include those related to the disease or condition of utilization data reflect the resources consumed focus for the registry and may incorporate (both services and products), while cost data comorbidities. Elements of interest related to the reflect a monetary value assigned to those confirmation of a diagnosis or condition could be resources. Examples include the actual cost of the date of diagnosis and the specific diagnostic results treatment (e.g., medication, screening, procedure) that were used to make the diagnosis, depending and the associated costs of the intervention (e.g., on the purpose of the registry. Disease or condition treatment of side effects, expenses incurred is often a primary eligibility or outcome variable traveling to and from clinicians’ appointments). in registries, whether the intent is to answer Costs that are avoided due to the treatment (e.g., specified treatment questions (e.g., measure the cost to treat the avoided disease) and costs effectiveness or safety) or to describe the natural related to lost workdays may also be important to history. This information may also be collected in collect, depending on the objectives of the study. constructing a medical history for a patient. In Registries that collect cost data over long periods addition to “yes” or “no” to indicate presence or of time (i.e., many years) may need to adjust costs absence of the diagnosis, it may be important to for inflation during the analysis phase of the study. capture responses such as “missing” or The types of data elements included in this “unknown.” framework are further described in Chapter 4 and below with respect to their source or the utility of Treatment/therapy—Treatment or therapy data the data for linking to other sources. Many of these include specific identifying information for the may be available through data sources outside of primary treatment (e.g., drug name or code, the registry system. biologic, device product or component parts, or

128 Chapter 6. Data Sources for Registries

surgical intervention, such as organ transplant or Hospital/clinic/health plan—System interactions coronary artery bypass graft) and may include include office visits, outpatient clinic visits, information on concomitant treatments. Dosage emergency room visits, inpatient hospitalizations, (or parameters for devices), route of procedures, and pharmacy visits, as well as administration, and prescribed exposure time, such associated dates. Data on all procedures as defined as daily or 3 times weekly for 4 weeks, should be by the registry protocol or plan (e.g., physical collected. Pharmacy data may include dispensing exam, psychological evaluation, chest x-ray, CAT information, such as the primary date of scan), including measurements, results, and units dispensation and subsequent refill dates. Data in of measure where applicable, should be collected. device registries can include the initial date of Cost accounting data may also be available to dispensation or implantation and subsequent dates match these interactions and procedures. and specifics of required evaluations or Descriptive information related to the points of modifications. Compliance data may also be care may be useful in capturing differences in care collected if pharmacy representatives or clinic patterns and can also be used to track patterns of personnel are engaged to conduct and report pill referral of care (e.g., outpatient clinic, inpatient counts or volume measurements on refill visits or hospital, academic center, emergency room, return visits for device evaluations and pharmacy). modifications. Insurance—The insurance system or payer claims Laboratory/procedures—Laboratory data include data can provide useful information on interactions a broad range of testing, such as blood, tissue, with the health care systems, including visits, catheterization, and radiology. Specific test results, procedures, inpatient stays, and costs associated units of measure, and laboratory reference ranges with these events. When using these data, it is or parameters are typically collected. Laboratory important to understand what services were databases are becoming increasingly accessible for covered under the various insurance plans at the electronic transfer of data, whether through a time the data were collected, as this may affect system-wide institutional database or a private utilization patterns. laboratory database. Diagnostic testing or evaluation may include procedures such as 3. Data Sources psychological or behavioral assessments. Results of these procedures and clinician exam procedures Data sources are classified as primary or may be difficult to obtain through data sources secondary based on the relationship of the data to other than the patient medical record. the registry purpose. Primary data sources Biosamples—The increased collection, testing, incorporate data collected for direct purposes of and storage of biological specimens as part of a the registry (i.e., primarily for the registry). registry (or independently as a potential secondary Primary data sources are typically used when the data source such as those described further below) data of interest are not available elsewhere or, if provides another source of information that available, are unlikely to be of sufficient accuracy includes both information from genetic testing and reliability for the planned analyses and uses. (such as genetic markers) and actual specimens. Primary data collection increases the probability of completeness, validity, and reliability because Health care provider characteristics—Information the registry drives the methods of measurement on the health care provider (e.g., physician, nurse, and data collection. (See Chapter 4.) These data or pharmacist) may be collected, depending on the are prospectively planned and collected under the purpose of the registry. Training, education, or direction of a protocol or study plan, using specialization may account for differences in care common procedures and the same format across patterns. Geographic location has also been used all registry sites and patients. The data are readily as an indicator of differences in care or medical integrated for tracking and analyses. Since the data practice.

129 Section I. Creating Registries

entered can be traced to the individual who been mentioned (e.g., obligations for informed collected them, primary data sources are more consent, appropriate data privacy, and readily reviewed through automated checks or confidentiality procedures). followup queries from a data manager than is Some of the secondary data sources do not collect possible with many secondary data sources. information at a specific patient level but are Secondary data sources are comprised of data anonymous and intended to reflect group or originally collected for purposes other than the population estimates. For example, census tract or registry under consideration (e.g., standard ZIP-Code-level data are available from the Census medical care, insurance claims processing). Data Bureau and can be merged with registry data. that are collected as primary data for one registry These data can be used as “ecological variables” to are considered secondary data from the perspective support analyses of income or education when of a second registry if linking was done. These such socioeconomic data are missing from registry data are often stored in electronic format and may primary data collection. The intended use of the be available for use with appropriate permissions. data elements will determine whether patient-level Data from secondary sources may be used in two information is required. ways: (1) the data may be transferred and imported The potential for data completeness, variation, and into the registry, becoming part of the registry specificity must be evaluated in the context of the database, or (2) the secondary data and the registry registry and intended use of the data. It is data may be linked to create a new, larger data set advisable to have a solid understanding of the for analysis. This chapter primarily focuses on the original purpose of the secondary data collection, first use for secondary data, while Chapters 16, 17, including processes for collection and submission, and 18 discuss the complexities of linking and verification and validation practices. Questions registries with other databases. to ask include: Is data collection passive or active? When considering secondary data sources, it is Are standard definitions or codes used in reporting important to note that health professionals are data? Are standard measurement criteria or accustomed to entering the data for defined instruments used (e.g., diagnoses, symptoms, purposes, and additional training and support for quality of life)? The existence and completeness of data collection are not required. Often, these data claims data, for example, will depend on insurance are not constrained by a data collection protocol company coverage policies. One company may and they represent the diversity observed in cover many preventive services, whereas another real-world practice. However, there may be may have more restricted coverage. One company increased probability of errors and underreporting may cover a treatment without restriction, while because of inconsistencies in measurement, another may require prior authorization by the reporting, and collection. Staff changes can further physician or require that the patient must have first complicate data collection and may affect data failed on a previous, less expensive treatment. quality. There may also be increased costs for Also, coverage policies can change over time. linking the data from the secondary source to the These variations must be known and carefully primary source and dealing with any potential documented to prevent misinterpretation of use duplicate or unmatched patients. rates. Additionally, secondary data may not all be Sufficient identifiers are also necessary to collected in the format (e.g., units of measure) accurately match data between the secondary required for registry purposes and may require sources and registry patients. The potential for transformation for integration and analyses. mismatch errors and duplications must be An overview of some secondary data sources that managed. (See Case Example 40.) The complexity may be used for registries is given below. Table 6–1 and obligations inherent in the collection and identifies some key strengths and limitations of the handling of personal identifiers have previously identified data sources.

130 Chapter 6. Data Sources for Registries

Table 6–1. Key data sources—strengths and limitations

Data Source Strength and Uses Limitations

Patient-reported data • Patient and/or caregiver outcomes. • Literacy, language, or other barriers that • Unique perspective. may lead to underenrollment of some • Obtaining information on treatments subgroups. not necessarily prescribed by clinicians • Validated data collection instruments (e.g., over-the-counter drugs, herbal may need to be developed. medications). • Loss to followup or refusal to continue • Obtaining intended compliance participation. information. • Limited confidence in reporting clinical • Useful when timing of followup may information and utilization information. not be concordant with timing of clinical encounter. Clinician-reported data • More specific information than • Clinicians are highly sensitive to burden. available from coded data or medical • Consistency in capture of patient signs, record. symptoms, use of nonprescribed therapy varies. Medical chart • Information on routine medical care, • The underlying information is not abstraction with more clinical context than coded collected in a systematic way. For claims. example, a diagnosis of bacterial • Potential for comprehensive view of pneumonia by one physician may be patient medical and clinical history. based on a physical exam and patient • Use of abstraction and strict coding report of symptoms, while another standards (including handling missing physician may record the diagnosis data) increases the quality and only in the presence of a confirmed interpretation of data abstracted. laboratory test. • It is difficult to interpret missing data. For example, does absence of a specific symptom in the visit record indicate that the symptom was not present or that the physician did not actively inquire about this specific symptom or set of symptoms? • Data abstraction is resource intensive. • Complete medical and clinical history may not be available (e.g., new patient to clinic).

131 Section I. Creating Registries

Table 6–1. Key data sources—strengths and limitations (continued)

Data Source Strength and Uses Limitations

Electronic health • Information on routine medical care • Underlying information from clinicians records (EHRs) and practice, with more clinical is not collected using uniform decision context than coded claims. rules. (See example under “Medical • Potential for comprehensive view of chart abstraction.”) patient medical and clinical history. • Consistency of data quality and breadth • Efficient access to medical and clinical of data collected varies across sites. data. • Difficult to handle information uploaded • Use of data transfer and coding as text files into the EHRs (e.g., scanned standards (including handling of clinician reports) vs. direct entry into missing data) will increase the quality data fields. of data abstracted. • Historical data capture may require manual chart abstraction prior to implementation date of medical records system. • Complete medical and clinical history may not be available (e.g., new patient to clinic). • EHR systems vary widely. If data come from multiple systems, the registry should plan to work with each system individually to understand the requirements of the transfer. Institutional or • Diagnostic and treatment information • Important to be knowledgeable about organizational (e.g., pharmacy, laboratory, blood coding systems used in entering data databases bank, radiology). into the original systems. • Resource utilization (e.g., days in • Institutional or organizational databases hospital). vary widely. The registry should plan • May incorporate cost data (e.g., billed to work with each system individually and/or paid amounts from insurance to understand the requirements of the claims submissions). transfer.

132 Chapter 6. Data Sources for Registries

Table 6–1. Key data sources—strengths and limitations (continued)

Data Source Strength and Uses Limitations

Administrative • Useful for tracking health care • Represents clinical cost drivers vs. databases resource utilization and cost-related complete clinical diagnostic and information. treatment information. • Range of data includes anything that • Important to be knowledgeable about is reimbursed by health insurance, the process and standards used in generally including visits to physicians claims submission. For example, only and allied health providers, most primary diagnosis may be coded and prescription drugs, many devices, secondary diagnoses not captured. In hospitalization(s), if a lab test was other situations, value-laden claims may performed, and in some cases, actual not be used (e.g., an event may be coded lab test results for selected tests (e.g., as a “nonspecific gynecologic infection” blood test results for cholesterol, rather than a “sexually transmitted diabetes). disease”). • In some cases, demographic • Important to be knowledgeable about information (e.g., gender, date of birth data handling and coding systems used from billing files) can be uploaded. when incorporating the claims data into • Potential for efficient capture of large the administrative systems. populations. • Can be difficult to gain the cooperation of partner groups, particularly in regard to receiving the submissions in a timely manner. Death indexes • Completeness—death reporting is • Time delay—indexes depend on mandated by law in the United States. information from other data sources • Strong backup source for mortality (e.g., State vital statistics offices), with tracking (e.g., patient lost to followup). delays of 12 to 18 months or longer • National Death Index (NDI)— (NDI). It is important to understand the centralized database of death records frequency of updates of specific indexes from State vital statistics offices; that may be used. database updated annually. • Absence of information in death indexes • NDI causes of death relatively reliable does not necessarily indicate “alive” (93–96%) compared with State death status at a given point in time. certificates. • Most data sources are country specific • Social Security Administration’s (SSA) and thus do not include deaths that Death Master File—database of deaths occurred outside of the country. reported to SSA; database updated • As of November 2011, Death Master weekly. File no longer includes protected State records.

133 Section I. Creating Registries

Table 6–1. Key data sources—strengths and limitations (continued)

Data Source Strength and Uses Limitations

U.S. Census Bureau • Population data. • Targets participants via survey sampling databases • Core census survey conducted every methodology and estimates. decade. • Does not provide subject-level data. • Wide range in specificity of information from U.S. population down to neighborhood and household level. • Useful in determining population estimates (e.g., numbers, age, family size, education, employment status). Existing registries • Can be merged with another data • Important to understand the existing source to answer additional questions registry protocol or plan to evaluate data not considered in the original registry collected for element definitions, timing, protocol or plan. and format, as it may not be possible to • May include specific data not generally merge data unless many of these aspects collected in routine medical practice. are similar. • Can provide historical comparison • Creates a reliance on the other registry. data. Other registry may end. • Reduces data collection burden • Other registry may change data elements for sites, thereby encouraging (which highlights the need for regular participation. communication). • Some sites may not participate in both. Must rely on the data quality of the other registry.

Medical chart abstraction—Medical charts availability and reproducibility of the data of primarily contain information collected as a part of interest. It is important to recognize that physicians routine medical care. These data reflect the and other clinicians do not generally use practice of medicine or health care in general and standardized data definitions in entering at a specific level (e.g., geographical, by specialty information into medical charts, meaning that one care provider). Charts also reflect uncontrolled clinician’s documented diagnosis of “chronic patient behavior (e.g., noncompliance). Collection sinusitis” or “osteoarthritis” or description of of standard medical practice data is useful in “pedal edema” may differ from that of another looking at treatments and outcomes in the real clinician. world, including all of the confounders that affect Electronic health records—The use of electronic the measurement of effectiveness (as distinguished health records (EHRs), sometimes called from efficacy) and safety outside of the controlled electronic medical records (EMRs), is increasing. conditions of a clinical trial. Chart documentation EHRs have an advantage over paper medical is often much poorer than one might expect, and records because the data in some EHRs can be there may be more than one patient-specific readily searched and integrated with other medical record (e.g., hospital and clinical records). information (e.g., laboratory data). The ease with A pilot collection is recommended for this labor- which this is accomplished depends on whether intensive method of data collection to explore the

134 Chapter 6. Data Sources for Registries

the information is in a relational databasea or examples are computerized order entry systems, exists as scanned documents. An additional pharmacies, blood banks, and radiology challenge relates to terminology and relationships. departments. For example, including the term “fit” in a search Administrative databases—Private and public for patients with epilepsy can yield a record for medical insurers collect a wealth of information in someone who was noted as “fit,” meaning the process of tracking health care, evaluating “healthy.” Relationships can also be difficult to coverage, and managing billing and payment. identify through searches (e.g., “Patient had breast Information in the databases includes patient- cancer” vs. “Patient’s mother had breast cancer”). specific information (e.g., insurance coverage and The quality of the information has the same copays; identifiers such as name, demographics, limitations as described in the paragraph above. SSN or plan number, and date of birth) and health Both the availability and standardization of EHR care provider descriptive data (e.g., identifiers, data have grown significantly in recent years, and specialty characteristics, locations). Typically, this trend is expected to continue. As of 2009, private insurance companies organize health care some data suppliers cited individual data sets data by physician care (e.g., physician office visits) exceeding 10 million lives.1 More recently, data 2 3 and hospital care (e.g., emergency room visits, suppliers are reporting 20 million to 35 million hospital stays). Data include procedures and patients in their data sets. Further, it is anticipated associated dates, as well as costs charged by the that more significant standardization of EHR data provider and paid by the insurers. Amounts paid by will result from the “EHR certification” insurers are often considered proprietary and requirements being developed in phases under the unavailable. Standard coding conventions are used American Recovery and Reinvestment Act of 2009 in the reporting of diagnoses, procedures, and (ARRA). Such standardization should increase not other information. Coding conventions include the only the availability and utility of EHR records, Current Procedure Terminology (CPT) for but also the ability to aggregate them into larger physician services and International Classification data sources. of Diseases (ICD) for diagnoses and hospital Institutional or organizational databases— inpatient procedures. The databases serve the Institutional or organizational databases may be primary function of managing and implementing evaluated as potential sources of a wide variety of insurance coverage, processing, and payment. data. System-wide institutional or hospital (See Case Example 12.) databases are central data repositories, or data Medicare and Medicaid claims files are two warehouses, that are highly variable from examples of commonly used administrative institution to institution. They may include a databases. The Medicare program covers over 43 portion of everything from admission, discharge, million people in the United States, including and transfer information to data reflecting almost everyone over the age of 65, people under diagnoses and treatment, pharmacy prescriptions, the age of 65 who qualify for Social Security and specific laboratory tests. Laboratory test data Disability, and people with end-stage renal might be chemistry or histology laboratory data, disease.4 The Medicaid program covers low- including patient identifiers with associated dates income children and their mothers; pregnant of specimen collection and measurement, results, women; and blind, aged, or disabled people. As of and standard “normal” or reference ranges. 2007, approximately 40 million people were Catheterization laboratory data for cardiac covered by Medicaid.5 Medicare and Medicaid registries may be accessible and may include claims files, maintained by the Centers for details on the coronary anatomy and percutaneous Medicare & Medicaid Services (CMS), can be coronary intervention. Other organizational aIn a relational database, information is presented in tables with rows and columns. Data within a table may be related by a common concept, and the related data may be retrieved from the database. See: A Relational Database Overview. http://docs.oracle.com/javase/tutorial/jdbc/overview/database.html. Accessed July 16, 2013.

135 Section I. Creating Registries

obtained for inpatient, outpatient, physician, should be considered when using these sources, skilled nursing facility, durable medical and vital status should not be assumed to be equipment, and hospital services. As of 2006, “alive” by the absence of information at a recent Medicare claim files for prescription drugs can point in time. These indexes are valuable sources also be obtained. The claims files generally contain of data for death tracking. Of course, mortality person-specific data on providers, beneficiaries, data can be accessed directly through queries of and recipients, including individual identifiers that State vital statistics offices and health departments would permit the identity of a beneficiary or when targeting information on a specific patient or physician to be deduced. Data with personal within a State. Likewise, birth certificates are identifiers are clearly subject to privacy rules and available through State departments and may be regulations. As such, the information is useful in registries of children or births. confidential and to be used only for reasons Area-level databases—Two sources of area-level compatible with the purpose(s) for which the data data are the U.S. Census and the Area Health are collected. The Research Data Assistance Resources Files (AHRF). The U.S. Census Bureau Center (ResDAC), a CMS contractor at the databases12 provide population-level data utilizing University of Minnesota, provides assistance to survey sampling methodology. The Census Bureau academic, government, and nonprofit researchers conducts many different surveys, the main one interested in using Medicare and/or Medicaid data 6 being the population census. The primary use of for their research. the data is to determine the number of seats Death and birth records—Death indexes are assigned to each State in the House of national databases tracking population death data Representatives, although the data are used for (e.g., the NDI7 and the Death Master File [DMF] many other purposes. These surveys calculate of the Social Security Administration [SSA]8). estimates through statistical processing of the Data include patient identifiers, date of death, and sampled data. Estimates can be provided with a attributed causes of death. These indexes are broad range of granularity, from population populated through a variety of sources. For numbers for large regions (e.g., specific States), to example, the DMF includes death information on ZIP Codes, all the way down to a household level individuals who had an SSN and whose death was (e.g., neighborhoods identified by street reported to the SSA. Reports may come in to the addresses). Information collected includes SSA by different paths, including from survivors demographic, gender, age, education, economic, or family members requesting benefits or from housing, and work data. The data are not collected funeral homes. Because of the importance of at an individual level but may serve other registry tracking Social Security benefits, all States, purposes, such as understanding population nursing homes, and mortuaries are required to numbers in a specific region or by specific report all deaths to the SSA. Prior to 2011, the demographics. The AHRF is maintained by the DMF contained virtually 100-percent complete Health Resources and Services Administration, mortality ascertainment for those eligible for SSA which is part of the Department of Health and benefits. As of November 2011, however, the DMF Human Services. The AHRF includes county-level no longer includes protected State death records. data on health facilities, health professions, In practical terms, this means that approximately measures of resource scarcity, health status, 4.2 million records were removed from the economic activity, health training programs, and historical public DMF (which contained 89 million socioeconomic and environmental records), and some 1 million fewer records will be characteristics.13 9 added to the DMF each year. The NDI can be Provider-level databases—Data on medical used to provide both fact of death and cause of facilities and physicians may be important for death, as recorded on the death certificate. Cause- categorizing registry data or conducting of-death data in the NDI are relatively reliable subanalyses. Two sources of such data are the (93–96 percent) compared with death American Hospital Association’s Annual Survey certificates.10, 11 Time delays in death reporting

136 Chapter 6. Data Sources for Registries

Data and the American Medical Association’s identifiers and locators in addition to information Physician Masterfile Data Collection. The Annual on the transplants, such as samples from the donor Survey Data is a longitudinal database that collects and recipient, histocompatibility, and outcomes. 700 data elements, covering organizational NMDP actively encourages research and structure, personnel, hospital facilities and utilization of registry data through a data services, and financial performance, from more application process and submission of research than 6,000 hospitals in the United States.14 Each proposals. hospital in the database has a unique ID, allowing The Registry of Patient Registries (RoPR) may the data to be linked to other sources; however, become a useful resource for finding existing there is a data lag of about 2 years, and the data registries (https://patientregistry.ahrq.gov). RoPR may not provide enough nuanced detail to support is a database of registry-specific information some analyses of cost or quality of care. The intended to promote collaboration, reduce Physician Masterfile Data Collection contains redundancy, and improve transparency in registry- current and historic data on nearly one million based research. The database contains information physicians and residents in the United States. Data on existing registries, such as the registry on physician professional medical activities, description, classification, and purpose, as well as hospital and group affiliations, and practice the registry sponsor’s interest in collaboration specialties are collected each year. opportunities. Registry planners may be able to use Encounter-level databases—Databases of RoPR to identify relevant registries to contact individual patient encounters (e.g., physician office about data sharing or research collaborations. visits, emergency department visits, hospital In accessing data from one registry for the inpatient stays), generally do not contain purposes of another, it is important to recognize individual patient identifiers and thus may not be that data may have changed during the course of linkable to patient registries, but nevertheless the source registry, and this may or may not have provide valuable insight into the makeup of the been well documented by the providers of the data. registry’s target population. This is particularly For example, in the United States Renal Data true for data from nationally representative System (USRDS),16 a vital part of personal surveys, such as AHRQ’s Health Care Utilization identification is CMS 2728, an enrollment form Project (H-CUP) Nationwide Inpatient Sample that identifies the incident data for each patient as (NIS) and the suite of surveys by the Centers for well as other pertinent information, such as the Disease Control and Prevention (CDC) and the cause of renal failure, initial therapy, and comorbid National Center for Health Statistics (NCHS), conditions. Originally created in 1973, this form is including the National Ambulatory Medical Care in its third version, having been revised in 1995 Survey (NAMCS), the National Hospital and again in 2005. Consequently, there are data Ambulatory Medicare Care Survey (NHAMCS), elements that exist in some versions and not and the National Hospital Discharge Survey others. In addition, the coding for some variables (NHDS). has changed over time. For example, race has been Existing registry and other databases—There are redefined to correspond with Office of numerous national and regional registries and Management and Budget directives and Census other databases that may be leveraged for Bureau categories. Furthermore, form CMS 2728 incorporation into other registries (e.g., disease- was optional in the early years of the registry, so specific registries managed by nonprofit until 1983 it was filled out for only about one-half organizations, professional societies, or other of the subjects. Since 1995, it has been mandatory entities). An example is the National Marrow for all people with end-stage renal disease. These Donor Program (NMDP),15 a global database of changes in form content, data coding, and cord blood units and volunteers who have completeness would not be evident to most consented to donate marrow and blood cells. researchers trying to access the data. Databases maintained by the NMDP include

137 Section I. Creating Registries

4. Other Considerations for in Chapter 16, and a discussion of managing patient identity across systems is provided in Secondary Data Sources Chapter 17. The discussion below focuses on logistical and The best identifier is one that is not only unique data issues to consider when incorporating data but has no embedded personal identification, from other sources. Chapter 11 fully explores data unless that information is scrambled and the key collection, management, and quality assurance for for unscrambling it is stored remotely and securely. registries. The group operating the registry should have a Before incorporating a secondary data source into process by which each new entry to the registry is a registry, it is critical to consider the potential assigned a unique code and there is a crosswalk impact of the data quality of the secondary data file to enable the system to append this identifier to source on the overall data quality of the registry. all new data as they are accrued. The crosswalk file The potential impact of quality issues in the should not be accessible by people or entities secondary data sources depends on how the data outside the management group. are used in the primary registry. For example, In addition, consideration should be given to the quality would be significant for secondary data fact that a registry may need to accept and link that are intended to be populated throughout the data sets from more than one outside organization. registry (i.e., used to populate specific data Each institution contributing data to the registry elements in the entire registry over time), will have unique requirements for patient data, particularly if these populated data elements are access, privacy, and duration of use. While having critical to determining a primary outcome. Quality identical agreements with all institutions would be of the secondary data will have less effect on ideal, this may not always be possible from a overall registry quality if the secondary data are to practical perspective. Yet all registries have be linked to registry data only for a specific resource constraints, and decisions about including analytic study (see Chapter 18). For more certain institutions have to be determined based on information on data quality, see Chapter 11. the resources available in order to negotiate The importance of patient identifiers for linking to specialized agreements or to maintain specialized secondary data sources cannot be overstated. requirements. Agreements should be coordinated Multiple patient identifiers should be used, and as much as possible so that the function of the primary data for these identifiers should not be registry is not greatly impaired by variability entered into the registry unless the identifying among agreements. All organizations participating information is complete and clear. While an SSN in the registry should have a common is very useful, high-quality probabilistic linkages understanding of the rules regarding access to the can be made to secondary data sources using data. Although exceptions can be made, it should various combinations of such information as name be agreed that access to data will be based on (last, middle initial, and first), date of birth, and independent assessment of research protocols and gender. For example, the NDI will make possible that participating organizations will not have matches when at least one of seven matching individual veto power over access. conditions is met (e.g., one matching condition is When data from secondary sources are used, “exact month and day of birth, first name, and last agreements should specify ownership of the source name”). However, the degree of success in such data and clearly permit data use by the recipient probabilistic and deterministic matching generally registry. The agreements should also specify the is enhanced by having many identifiers to facilitate roles of each institution, its legal responsibilities, matching. As noted earlier, the various types of and any oversight issues. It is critical that these data (e.g., personal history, adverse events, issues and agreements be put in place before data hospitalization, and drug use) have to be linked are transferred so that there are no ambiguities or through a common identifier. A discussion of both unforeseen restrictions on the recipient registry statistical and privacy issues in linkage is provided later on.

138 Chapter 6. Data Sources for Registries

Some registries may wish to incorporate data from An example of the need for regular communication more than one country. In these cases, it is is a situation that arose with the United States important to ensure that the data are being Renal Data System a few years ago. The United collected in the same manner in each country or to Network for Organ Sharing (UNOS) changed the plan for any necessary conversion. For example, coding for donor type in their transplant records. height and weight data collected from sites in This resulted in an apparent 100-percent loss of Europe will likely be in different units than height living donors in a calendar year. The change was and weight data collected from sites in the United not conveyed to USRDS and was not detected by States. Laboratory test results may also be reported USRDS staff. After USRDS learned about the in different units, and there may be variations in change, standard analysis files that had been sent the types of pharmaceutical products and medical to researchers with the errors had to be replaced. devices that are approved for use in the Distributed data networks are another model for participating countries. Understanding these issues sharing data. In a distributed data network, data prior to incorporating secondary data sources from sharing may be limited to the results of analyses or other countries is extremely important to maintain aggregated data only. There is much interest in the the integrity and usefulness of the registry potential of distributed data networks, particularly database. for safety monitoring or public health surveillance When incorporating other data sources, (see Chapter 15, Section 11). However, the consideration should also be given to the registry complexities of data sharing within a distributed update schedule. A mature registry will usually data network are still being addressed, and it is have a mix of data update schedules. The registry premature to discuss good practice for this area. may receive an annual update of large amounts of data, or there could be monthly, weekly, or even 5. Summary daily transfers of data. Regardless of the schedule of data transfer, routine data checks should be in In summary, a registry is not a static enterprise. place to ensure proper transfer of data. These The management of registry data sources requires should include simple counts of records as well as attention to detail, constant feedback to all predefined distributions of key variables. participants, and a willingness to make Conference calls or even routine meetings to go adjustments to the operation as dictated by over recent transfers will help avoid mistakes that changing times and needs. might not otherwise be picked up until much later.

139 Section I. Creating Registries

Case Example for Chapter 6

Case Example 12. Using claims data along the role of environmental exposures in the with patient-reported data to identify patients etiology of ALS. Because ALS is a non- Description The National Amyotrophic reportable disease in the United States (except for Lateral Sclerosis (ALS) Registry the Commonwealth of Massachusetts), previous is a rare disease registry created attempts to estimate ALS incidence and by the Agency for Toxic prevalence using nonspecific mortality data have Substances and Disease Registry faced many challenges and at best overestimated (ATSDR) within the U.S. disease frequency. Identifying patients through Department of Health and site recruitment for research purposes poses Human Services (HHS). The additional challenges, as access to patient medical purpose of the registry is to records can be limited, costly, and time- quantify the incidence and consuming to obtain. Patient recruitment issues prevalence of ALS in the United are compounded by the complexities of this rare States, describe the disease, in which the average timeframe from demographics of people with diagnosis to death is 2–5 years. U.S. ALS, and examine potential risk governmental agencies acknowledged that a factors for the disease. national, structured data collection program for ALS was greatly needed, and that alternative data Sponsor U.S. Department of Health and sources and recruitment strategies would need to Human Services and Agency for be identified. Toxic Substances and Disease Registry, through funding from Proposed Solution the “ALS Registry Act” (U.S. In 2008, President Bush signed the ALS Registry Congress Public Law 110-373). Act into law, allowing ATSDR to create the Year Started 2010 National ALS Registry. The registry is the only Congressionally mandated population-based ALS Year Ended Ongoing registry in the United States. As a first step in No. of Sites All 50 States, including U.S. developing the registry, a workshop of territories; data from national international experts in neurological and administrative databases are autoimmune conditions was convened to discuss combined with patient self- approaches to creating a national database. Based enrollment data. on feedback from these experts, the registry uses a two-pronged approach to identify all U.S. cases No. of Patients The first registry report is of ALS. The first approach uses national anticipated for release in spring administrative databases, including those of 2014. Medicare, Medicaid, the Veterans Health Challenge Administration, and the Veterans Benefit Administration, to identify prevalent cases based Amyotrophic lateral sclerosis (ALS) is a on an algorithm developed through pilot projects. progressive, fatal neurodegenerative disorder of These administrative databases cover both the upper and lower motor neurons. Many approximately 90 million Americans, and the knowledge gaps exist in the understanding of algorithm identifies 80 to 85 percent of all true ALS, including uncertainty about the disease’s ALS cases when applied to these databases. The incidence and prevalence, misdiagnosis of ALS in second approach uses a secure Web portal to patients with other motor neuron disorders, and allow patients to self-enroll voluntarily. Data

140 Chapter 6. Data Sources for Registries

Case Example 12. Using claims data along inform people with ALS about new research (e.g., with patient-reported data to identify patients clinical trials, epidemiological studies) for which (continued) they may be eligible. Lastly, ATSDR is funding a Proposed Solution (continued) feasibility study for the creation of a national biospecimen repository that would be open to all from the two approaches are combined into the U.S. residents with ALS who are enrolled in the registry database, and duplicate patients are registry. This proposed biorepository will help identified and removed so that each person with researchers better understand the disease because ALS is counted only once in the registry. it will pair biospecimens (e.g., blood, brain tissue) Results with existing risk-factor data from patients. The registry data will support several research Key Point projects. The Web portal for self-enrolled Combining multiple data sources, such as participants contains brief surveys that collect administrative databases and patient-reported information on potential risk factors, such as information, is a novel approach and can be an socio-demographic characteristics, occupational effective way to successfully identify patients history, military history, cigarette smoking, with a rare disease and to better understand the alcohol consumption, physical activity, family prevalence, incidence, and etiology of the disease. history of neurodegenerative diseases, and However, using alternative approaches requires a disease progression. ATSDR is also currently strong understanding of the nuances of the implementing active surveillance projects that individual data sources; pilot testing is also will allow population-based case estimates of helpful to identify potential issues with data ALS in certain smaller geographic areas (i.e., at sources prior to registry launch. the State and metropolitan levels) to help ATSDR evaluate the completeness of the registry. In For More Information addition, ATSDR has developed a system to http://wwwn.cdc.gov/als

References for Chapter 6 4. Centers for Medicare and Medicaid Services. “Medicare Coverage – General Information.” 1. Federal Coordinating Council for Comparative http://www.cms.gov/Medicare/Coverage/ Effectiveness Research. Report to the President CoverageGenInfo/index.html. Accessed August and the Congress. June 30, 2009. U.S. Department 15, 2012. of Health and Human Services. http://www.med. 5. DeNavas-Walt C, Proctor BD, Smith JC. Current upenn.edu/sleepctr/documents/ Population Reports, P60-235, Income, Poverty, FederalCoordinatingCouncilforCER_2009.pdf. and Health Insurance Coverage in the United Accessed August 14, 2012. States: 2007. Washington, DC: U.S. Government 2. GE Healthcare. “Medical Quality Improvement Printing Office; 2008. http://www.census.gov/ Consortium.”. http://www3.gehealthcare.com/en/ prod/2008pubs/p60-235.pdf. Accessed July 16, Products/Categories/Healthcare_IT/Clinical_ 2013. Knowledge_Solutions/MQIC. Accessed August 6. Research Data Assistance Center. http://www. 15, 2012. resdac.org. Accessed August 15, 2012. 3. Practice Fusion. “Practice Fusion Releases EMR 7. National Center for Health Statistics. National Data set, Launches Health Data Challenge with Death Index. http://www.cdc.gov/nchs/ndi.htm. Kaggle.” http://www.practicefusion.com/pages/pr/ Accessed July 16, 2013. health-data-initiative-forum-challenge-2012.html. Accessed August 15, 2012. 8. Social Security Administration. Death Master File. National Technical Information Service. http:// www.ntis.gov/products/ssa-dmf.aspx. Accessed August 15, 2012.

141 Section I. Creating Registries

9. National Technical Information Service. Important 13. Health Resources and Services Administration. Notice: Change in Public Death Master File Area Health Resources Files (AHRF). http://arf. Records. http://www.ntis.gov/pdf/import-change- hrsa.gov. Accessed August 15, 2012. dmf.pdf. Accessed August 14, 2012. 14. American Hospital Association. AHA Data and 10. Doody MM, Hayes HM, Bilgrad R. Comparability Directories. http://www.aha.org/research/rc/ of national death index plus and standard stat-studies/data-and-directories.shtml. Accessed procedures for determining causes of death in August 15, 2012. epidemiologic studies. Ann Epidemiol. 2001 15. National Marrow Donor Program. Jan;11(1):46-50. PMID: 11164119. http://www.bethematch.org. Accessed August 15, 11. Sathiakumar N, Delzell E, Abdalla O. Using the 2012. National Death Index to obtain underlying cause 16. United States Renal Database. http://www.usrds.org. of death codes. J Occup Environ Med. 1998 Accessed August 15, 2012. Sep;40(9):808-13. PMID: 9777565. 12. U.S. Bureau of the Census. http://www.census.gov. Accessed August 15, 2012.

142 Section II Legal and Ethical Considerations for Registries

Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

1. Introduction about legal protections for the privacy of health information focuses solely on U.S. law. Health This chapter covers the ethical and legal information is also legally protected in European considerations that should accompany the and some other regions by distinctly different development and use of all health information rules, none of which are discussed in this chapter.1 registries, including patient registries as defined in If registry developers intend to obtain health this document, for the purposes of public health information from outside of the United States or activities, governmental health program oversight, transfer to or share their information with quality improvement/assurance (I/A), and research. registries outside the United States, they should These considerations apply generally accepted consult legal counsel early in the registry planning ethical principles for scientific research involving process for the necessary assistance. It also should human subjects to health information registries. be noted that the rules and regulations described Related topics include issues of transparency in the here are to protect patients and research operation of registries, oversight of registry participants, not to prevent legitimate research. activities, and property rights in health care While the requirements may seem daunting, they information and registries. are not insurmountable barriers to research. With careful planning and legal guidance, registries can Section 2.1 of this chapter discusses the ethical be designed and operated in compliance with concerns and considerations involved with applicable rules and regulations. obtaining and using confidential health information in registries. Section 2.2 describes the In the context of this chapter, health information is transformation of ethical concerns into the legal broadly construed to include any information regulation of human subjects research and the created or used by health care providers and privacy of individually identifiable health insurance plans that relates to an individual’s information. In Section 3, an overview is presented health condition, the provision of health care of these regulatory requirements and their services to an individual, or payment for health interactions as they specifically relate to registries. care services provided to an individual.2 As a Section 4 makes recommendations about registry result, health information may include a broad transparency and oversight, based on the need to range of demographic information and personal ensure the independence, integrity, and credibility characteristics, such as socioeconomic and marital of biomedical research, while preserving and status, the extent of formal education, improving the utility of registry data. Finally, developmental disability, cognitive capacities, property rights in health information and registries emotional stability, and gender, age, and race, all are briefly discussed. Table 7-1, at the end of this of which may affect health status or health risks. chapter, provides an overview of the applicable Typically, health information includes information regulatory requirements based on the type of about family members, so it also can have an registry developer and the extent to which registry impact on the privacy of third parties. Patients data are identifiable. widely regard health information as private and thus expect confidentiality to be maintained. The purpose of this chapter is solely to provide information that will help readers understand the Concerns about potential risks to individual issues, not to provide specific legal opinions or privacy have led to Federal legal requirements for regulatory advice. Legal advisors should always be prospective review of research projects and consulted to address specific issues and ensure that conditions on the use or disclosure of health all applicable Federal, State, and local laws and information for research and other purposes. The regulations are followed. The discussion below creation and use of patient registries for a research

145 Section II. Legal and Ethical Considerations for Registries

purpose ordinarily constitute “research involving produced an impetus to make financial and other human subjects” as defined by regulations arrangements more public. The discussion of applicable to research activities funded by the U.S. transparency in this chapter includes Department of Health and Human Services3 recommendations for the public disclosure of (HHS) and certain other Federal agencies. registry operations as a means of maintaining Moreover, Federal privacy regulations promulgated public trust and confidence in the use of health under the Health Insurance Portability and information for registry purposes. Reliance on a Accountability Act of 1996 (HIPAA)4 and standing advisory committee is recommended to modified by the Health Information Technology registry developers as a way to provide expert for Economic and Clinical Health Act (HITECH technical guidance for registry operations and to – part of the American Recovery and Reinvestment firmly establish the independence of the registry Act of 2009) specifically apply to the use and from committed or conflicted interests, as disclosure of certain individually identifiable described in Chapter 2. This discussion of health information for research and other transparency in methods is not intended to purposes. discourage private investments in registries that The term human subjects is used throughout this produce proprietary information in some chapter for consistency with applicable Federal circumstances. Neither the funding source nor the law. Some may prefer the term research generation of proprietary information from a participants. registry determines whether a registry exercises and adheres to the good practices described in this This chapter provides a general guide to Federal guide. legal requirements in the United States. (Legal requirements in other countries may also be Health care providers and health insurance plans relevant and may be different from those in this have plausible claims of ownership to health and country, but a discussion of any applicable claims information, although the public international rules is beyond the scope of this perspective on these claims has not been tested. document.) These legal requirements may Registry developers should anticipate negotiating influence registry decisions involving the selection access to health and claims information, especially of data elements and data verification procedures, when it is maintained in electronic form. Registry and may also affect subsequent uses of registry developers also are likely to encounter licensing data for secondary research purposes. State laws requirements, including processing and use fees, in also may apply to the use of health information for obtaining health and claims information. The research purposes. The purpose of a registry, the processes for use of registry data sets, especially in status of its developer, and the extent to which multiple analyses by different investigators, should registry data are identifiable largely determine be publicly disclosed to assure the public that applicable regulatory requirements. This chapter registries are appropriately protecting the reviews the most common of these arrangements. confidentiality of health information. The complexity and sophistication of registry structures and operations vary widely, with 2. Ethical Concerns Relating to considerable variability also observed in the Health Information Registries processes registry stewards use to obtain data. Nonetheless, common ethical and legal principles 2.1 Application of Ethical Principles are associated with the creation and use of registries. These commonalities are the focus of The Belmont Report5 is a summary of the basic this chapter. principles and guidelines developed to assist in resolving ethical problems in conducting research Ethical concerns about the conduct of biomedical using human subjects. It was the work product of research, especially research involving the the National Commission for the Protection of interaction of the clinical research community with Human Subjects of Biomedical and Behavioral its patients and commercial funding agencies, have

146 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

Research, which was created by the National operations to inform potential subjects’ decisions Research Act of 1974.6 about participation in a research registry. In some The Belmont Report identifies three fundamental defined circumstances, the principle ofrespect for principles for the ethical conduct of scientific persons may be subordinate to other ethical research that involves human subjects. These principles and values, with the result that an principles are respect for persons as autonomous explicit consent process for participation in the agents (self-determination), beneficence (do good, registry may not be necessary. A waiver of do no harm, protect from harm), and justice informed consent requirements may apply to the (fairness, equitable distribution of benefits and registry and be ethically acceptable. burdens, equal treatment). Together, they provide a (See discussion of waivers of informed consent foundation for the ethical analysis of human requirements in Section 3.3.5.) In these situations, subjects research, including the use of health alternatives to an explicit consent process for each information in registries developed for scientific individual contributing health information to the purposes with a prospect of producing social registry may be adequate. For example, the benefits. These principles are substantively the registry might provide readily accessible, publicly same as those identified by the Council for available information about its activities as an International Organizations of Medical Sciences in alternative to individual informed consent. its international guidelines for the ethical review of A general ethical requirement for consent clearly epidemiologic studies.7 implies that human subjects voluntarily permit the Nevertheless, the application of these principles to use of their health information in a registry, unless specific research activities can result in different a specific exception to voluntary participation conclusions about what comprises ethical design applies to the registry. One such exception is a and conduct of the research in question. These legally mandated, public health justification for the different conclusions frequently occur because the compilation of health information (e.g., certain principles are assigned different values and relative infectious disease reporting). Voluntary agreement importance when more than one person performs to the use of health information in a registry the ethical analysis. In most of these situations, necessarily allows a subsequent decision to however, a generally supported consensus position discontinue participation. Any limitation on an on the ethical design and conduct of the research is individual’s ability to withdraw information from a desired and achievable goal. This goal does not the registry (e.g., once incorporation into preclude re-analysis as social norms or concerns aggregated data has occurred) should be clearly about research activities change over time in communicated in the consent process as a response to new information, new technologies or condition of initial participation. The consent persistent ethical questioning. process should also include instructions about the procedures for withdrawal at any time from The ethical principle of respect for persons participation in the registry unless a waiver of supports the practice of obtaining individuals’ consent applies to the registry. Incentives for consent to the use of their health information for registry use of health information (e.g., insurance research purposes related or unrelated to the coverage of payments for health care services) clinical and insurance reasons for creating the should be carefully evaluated for undue influence information. In connection with research registries, both on the individuals whose health information consent may have multiple components: is sought for registry projects and on the health (1) consent to registry creation by the compilation care providers of those services.8, 9 of patient information; (2) consent to the initial research purpose and uses of registry data; and Conflicts of interest also may result in undue (3) consent to subsequent use of registry data by influence on patients and may compromise the registry developer or others for the same or voluntary participation. One potential source of different research purposes. The consent process conflict widely identified within clinical research should adequately describe registry purposes and is the use of recruitment incentives paid by

147 Section II. Legal and Ethical Considerations for Registries

funding agencies to health care providers.10 Some information is used in the registry. Exceptions to professional societies and research organizations this arise when a registry is designed to provide have established policy on the use of recruitment benefits to the human subjects as individuals, such incentives. Many entities have characterized as as longitudinal reports on treatment effects or unethical incentives that are significantly beyond health status or quality-of-care reports. Risks to fair market value for the work performed by the privacy and dignity are minimized by health care provider; others require disclosure to conscientious protection of the confidentiality of research subjects of any conflicting interest, the health information included in the registry15 financial or nonfinancial.11 Federal law now through the use of appropriate physical, technical, requires manufacturers of certain drugs, devices, and administrative safeguards for data in the or medical supplies to report, for public display, operations of the registry. These safeguards should the amounts of remuneration paid to physicians for include controls on access to registry data, research purposes.12 Some States, including including access to individual identifiers that may Massachusetts, have similar laws in effect.13 be included in registry data. Minimization of risks Research organizations, particularly grantees of also requires a precise determination of what Federal research funding, may have systematic information is necessary for the research purposes policies and procedures in place that registry of the registry and limiting the information developers can rely on for managing employee collected accordingly. conflicts of interest. Nonetheless, in their planning, Certain populations of patients may be vulnerable registry developers should specify and implement to social, economic, or psychological harms as a recruitment practices that protect patients against result of a stigmatizing health condition. inappropriate influences. Developers of registries compiling this health Applying the principle of respect for persons to the information must make special efforts to protect research use of health information generates the identities of the human subjects contributing additional ethical concerns about preserving the data to the registry. Additional legal protections privacy and dignity of patients, protecting the may apply if HHS-supported research is being confidentiality of health information, and conducted through or in connection with the minimizing potential harms. These concerns have registry. Additional protections also apply to intensified as health care services, third-party populations such as pregnant women, human payment systems, and health information systems fetuses, neonates, prisoners, and children, who are have become more complex. Legal standards for considered vulnerable to undue influence and the use and disclosure of health information create coercion during the consent process. In particular, a baseline of required privacy protections for data obtained from pediatric and adolescent individually identifiable health information. populations may lead to ethical concerns if there is However, depending on the particular health the potential for lifelong discrimination that may condition or population of interest, safeguards for effectively exclude them from educational the confidentiality of registry data beyond opportunities and other social benefits16 (e.g., applicable legal requirements may be ethically health insurance, although under the Affordable necessary to protect the privacy and dignity of Care Act health insurers may not discriminate those individuals contributing health information against individuals on the basis of pre-existing to the registry. conditions). The principle of beneficence ethically obligates In an analysis applying the principle of developers of health information registries for beneficence, research involving human subjects research purposes to minimize potential harms to that is unlikely to produce valid scientific the individuals or groups14 whose health information is unethical. This conclusion is based information is included in the registry. There are on the lack of social benefit to offset even minimal usually no apparent benefits to offset potential risks imposed by the research on participating harm to the individuals or groups whose health individuals. Health information registries should

148 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

incorporate an appropriate design (including, prospectively apply careful scrutiny to the where appropriate, calculation of the patient proposed purposes for and activities of a registry, sample as described in Chapter 3) and data in consultation with appropriate institutional elements, written operating procedures, and officials, to avoid both ethical and compliance documented methodologies, as necessary, to issues that may undermine achievement of the ensure the fulfillment of a valid scientific registry’s objectives. 17 purpose. Registry developers also must consider An ethical analysis employing the principle of confidentiality and/or proprietary concerns with justice also yields candid recognition of the regard to the identity of the health care providers, potential risks to those who contribute health at the level of both individual professionals and information to a registry, and the probable lack of institutions, and the health care insurance plans benefit to those individuals (except in the cases from which they obtain registry data. Information where registries are specifically constructed to about health care providers and insurance plans provide benefit to those individuals). The can also identify certain patient populations and, in imbalance of burden and benefit to individuals rare circumstances, individual patients. Moreover, reinforces the need to minimize the risks from the objectives of any registry, broadly speaking, registry use of health information. Precise and are to enhance the value of the health care services well-developed scientific reasons for inclusion received, not to undermine the credibility and thus (or exclusion) of defined health information in a the effectiveness of health care providers and registry help ensure that the burden placed on insurance plans in their communities. Developers individuals as a result of their participation is fair of registries created for public health and equitable. investigations, health system oversight activities, The above analysis refers to research activities. and quality I/A initiatives to monitor compliance However, the ethical concerns expressed may also with recognized clinical standards must consider apply to other activities involving the use or whether safeguards for the identity of service disclosure of individuals’ health information for professionals and institutions are appropriate. At nonresearch purposes. Public health, oversight of the same time, however, any confidentiality the delivery of health care services through safeguards should permit certain disclosures, as government programs, and quality I/A activities all permitted by applicable law and designated by the can evoke the same set of ethical concerns as service professionals and institutions, for the research activities about the protection of patient reporting of performance data, which are self-determination, privacy, and dignity; the increasingly associated with payment from payers. maintenance of the confidentiality of individually 2.2 Transformation of Ethical Concerns identifiable health information to avoid potential Into Legal Requirements harms; and the imposition of a risk of harm on some individuals to the benefit of others not at Important ethical concerns about the creation, risk. In the past, different assignments of social maintenance, and use of patient registries for value to these activities and different potential for research purposes include risks of harm to human the social benefits and harms they produce have subjects resulting from unauthorized access to created different levels of social acceptance and registry data and inappropriate use of the compiled formal oversight for these activities compared with health information. These concerns about harms research activities. Nonetheless, these activities arise from public expectations of confidentiality may include a research component in addition to for health information and the importance of that their primary stated objectives, a circumstance that confidentiality in preserving the privacy and implicates the ethical concerns discussed above dignity of individual patients as well as the and produces additional concerns about clinician/patient relationship. compliance with the legal requirements for Over the last decade, two rapid technological research activities. Registry developers should developments have intensified these ethical

149 Section II. Legal and Ethical Considerations for Registries

concerns. One of these advances was DNA on the institutional context of the registry sequencing, , recombination, and the developer, relevant institutional policies, and concomitant application of this technology to whether the health information contributed to the biomedical research activities in human genetics. registry maintains patient identifiers. Widespread anticipation of potential social benefits The Office for Human Research Protections produced by biomedical research as a result of (OHRP) administers the Common Rule as it these technologies was accompanied by ethical applies to human subjects research conducted or concern about the potential for affronts to personal supported by HHS. Guidance published by OHRP dignity and economic, social, or psychological discusses research use of identifiable private health harms to individuals or related third parties. information. This guidance makes clear that In addition to specific ethical concerns about the OHRP considers the creation of health information effect of technological advances in biomedical registries—containing individually identifiable, research, general social concerns about the privacy private information—for research purposes to be of patient information have accompanied the human subjects research for the institutions subject advance of health information systems technology to its jurisdiction.21 The applicability of the and electronic information processing, as applied Common Rule to research registries is discussed in to the management and communication of health more detail in Section 4. information. These social concerns produced legal OHRP regulations for human subjects protection privacy protections, first in Europe and later in the require prospective review and approval of the United States. The discussion below about legal research by an institutional review board (IRB) protections for the privacy of health information and the informed consent (usually written) of each focuses solely on U.S. law. of the human subjects involved in the research, 2.2.1 The Common Rule unless an IRB expressly grants a waiver of informed consent requirements.22 A research International and domestic concerns about the project must satisfy certain regulatory conditions protection, respect, and privacy of human subjects to obtain IRB approval of a waiver of the informed resulted in a uniform set of regulations from the consent requirements. (See Section 3.3.5 for Federal agencies that fund such research known as discussion of waivers of informed consent the “Common Rule.”18, 19 The legal requirements requirements.) A registry plan is the research of the Common Rule apply to research involving “protocol” reviewed by the IRB. At a minimum, human subjects conducted or supported by the 17 the protocol should identify (1) the research Federal departments and agencies that adopted the purpose of a health information registry, Rule. Some of these agencies may require (2) detailed arrangements for obtaining informed additional legal protections for human subjects. consent, or detailed justifications for not obtaining The HHS regulations will be used for all following informed consent, to collect health information, references to the Common Rule. and (3) appropriate safeguards for protecting the Among these requirements is a formal written confidentiality of registry data, in addition to any agreement, from each institution engaged in such other information required by the IRB on the risks research, to comply with the Common Rule. For and benefits of the research.23 human subjects research conducted or supported As noted previously, for human subjects research by most of the Federal entities that apply the conducted or supported by most Federal Common Rule, the required agreement is called a departments and agencies that have adopted the Federalwide Assurance (FWA).20 Research Common Rule, an FWA satisfies the requirement institutions may opt in their FWA to apply for an approved assurance of compliance. Some Common Rule requirements to all human subjects research organizations extend the application of research activities conducted within their facilities their FWA to all research, regardless of the or by their employees and agents, regardless of the funding source. Under these circumstances, any source of funding. The application of Common patient information registry created and Rule requirements to a particular registry depends

150 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

maintained within the organization may be subject discussion in this chapter assumes that the data to the Common Rule. In addition, some research sources for registries are HIPAA-covered entities organizations have explicit institutional policies or their business associates, which are subject to and procedures that require IRB review and provisions of the Privacy Rule. In the event that a approval of all human subjects research. registry developer intends to collect and use data from sources that are not covered entities or their 2.2.2 The Privacy Rule business associates under the Privacy Rule, such as In the United States, HIPAA and its implementing personal health record vendors not working on regulations4 (here collectively called the Privacy behalf of a covered entity or business associate, Rule) created legal protections for the privacy of these sources still may be subject to other laws, individually identifiable health information created such as State laws, that may govern the protection and maintained by “covered entities” and their of patient information. “business associates.” “Individually identifiable Although data sources are assumed to be subject to health information” is information, including the Privacy Rule, registry developers and the demographic data, created or received by a health associated institutions where the registry will care provider, health plan, employer, or health care reside may not be. Notably, the Privacy Rule does clearinghouse, that identifies an individual or not apply to registries that reside outside of a could be used to identify an individual, and relates covered entity or business associate. Within to (1) an individual’s past, present, or future academic medical centers, for example, registry physical or mental health condition; (2) the developers may be associated with units that are provision of health care to an individual; or (3) the outside of the institutional health care component past, present, or future payment for health care to to which the Privacy Rule applies, such as a an individual. With certain exceptions, biostatistics or economics department. But because “individually identifiable health information” is many, if not virtually all, data sources for registries “protected health information” (PHI) under the are covered entities or their business associates, Privacy Rule when it is transmitted or maintained registry developers are likely to find themselves by a covered entity or a business associate on deeply enmeshed in the Privacy Rule. In addition, behalf of a covered entity.24 Because registries may the formal agreements required by the Privacy exist over long periods of time, it is important to Rule in certain circumstances in order to access, note one exception that provides that the process, manage, and use certain forms of patient individually identifiable information of persons information impose legally enforceable continuing who have been deceased for more than 50 years is conditions upon data recipients under contract law. not considered PHI. Such conditions of use also may result in direct Covered entities are health care providers that liability under HIPAA if the registry is considered engage in certain standard financial or a business associate of a data source that is a administrative health care transactions covered entity (see the discussion in Section 3.4.3 electronically, health plans, and health care below of the HITECH Act, which extended direct clearinghouses.25 Business associates generally are liability for compliance with certain requirements persons or organizations, other than a member of a of the HIPAA Rules to business associates of covered entity’s workforce, that perform certain covered entities, where before business associates functions or services (e.g., claims processing, data were required and liable to protect the information analysis, data aggregation, patient safety activities) to which they had access only through their on the covered entity’s behalf that involve access to business associate contracts with covered entities). protected health information.26 For the purposes of Therefore, registry developers should be cognizant this chapter, the relevant entities are covered health of the patient privacy considerations confronting care providers, which may include individual their likely data sources—as well as themselves, if health care providers (e.g., a physician, they are performing functions or services on pharmacist, or physical therapist), health care behalf of their data sources as business insurance plans, and their business associates. The associates—and should consider following certain

151 Section II. Legal and Ethical Considerations for Registries

Privacy Rule procedures, required or not, developing a registry, they should expect to depending on their arrangements with those data interact with clinicians, the privacy officer, the sources. IRB or privacy board staff, health information In general, the Privacy Rule defines the system representatives, legal counsel, compliance circumstances under which covered entities officials, and contracting personnel. Registry (e.g., health plans and most health care providers) developers should also maintain awareness of and their business associates may use and disclose regulatory modifications or amendments to, or new patient information for a variety of purposes, guidance on how to comply with, the Privacy Rule, including research. The Privacy Rule establishes a which can be expected as the use of electronic Federal baseline of protections, which do not health information becomes more prevalent. For pre-empt more stringent (more protective) State example, on January 25, 2013, HHS issued laws.27 For example, the Privacy Rule requires significant modifications to the Privacy Rule, many of which implemented HITECH Act covered entities to include certain information in 30 patient authorizations for the use or disclosure of requirements. One of the most relevant individually identifiable information, including an modifications for registry developers, as expiration date or event that can be many years in mentioned above and discussed more fully below the future. The laws of the State of Maryland, in Section 3.4.3, is the extension of certain Privacy Rule requirements and liability directly to business however, specifically require that, absent certain 31 exceptions, a patient’s authorization may only be associates. valid for a maximum period of 1 year.28 In this Subsequent use and sharing of registry data may case, a covered entity located in Maryland can and be affected by the regulatory conditions that apply should satisfy both the Privacy Rule and State law to initial collection, as well as by other ethical requirements by complying with the State’s concerns and legal issues. The Privacy Rule one-year maximum expiration deadline on its created multiple pathways by which registries can patient authorization forms. compile and use patient information. For instance, The Privacy Rule regulates the use of identifiable a registry within a covered entity may obtain a patient information within health care providers’ HIPAA authorization from each patient organizations and health insurance plans (and contributing identifiable information to a registry within their business associates), and the for a particular research project, such as the disclosure of patient information to others outside relationship between hypertension and Alzheimer’s of the entity.29 As a result, the initial collection of disease. If the registry subsequently seeks to use registry data from covered entities or business the data for another research purpose, it may do so associates is subject to Privacy Rule requirements, if it obtains new authorizations or satisfies the which may apply in different ways depending on conditions of another permission in the Privacy the registry’s purpose, whether the registry resides Rule—for example, by de-identifying the data to within a covered entity or outside of a covered Privacy Rule standards. Alternatively, the registry entity, whether the registry is considered a can obtain authorizations for use and disclosure of business associate of the covered entity, and the individuals’ health information for future research extent to which the patient information identifies purposes at the same time that it obtains individuals. Health care providers or insurance authorization to place the information in the plans, as well as their business associates, that registry, as long as the authorization for future create, use, and disclose patient information for research use or disclosure adequately describes the clinical use or business purposes are subject to purposes of the future research such that it would civil—and in some cases criminal—liability for be reasonable for the individual to expect that his violations of the Privacy Rule. or her information could be used or disclosed for the future research. Registry developers should be sufficiently knowledgeable about the Privacy Rule to facilitate The authors recommend that registry developers the necessary processes for their data sources. In establish a detailed tracking system, based on the

152 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

extent to which registry data remain identifiable HHS have jointly published specific guidance on for individual patients, for the collection, uses, and the Privacy Rule for public health activities.37 disclosures of registry data. The tracking system Other Privacy Rule provisions permit the use or should produce comprehensive documentation of disclosure of patient health information as required compliance with both Privacy Rule requirements by other laws.38 and legally binding contractual obligations to data The protections for patient information created by sources. the Privacy Rule that are generally relevant to With regard to registries developed for research registries developed for research purposes include purposes, the Privacy Rule defines research as “a explicit individual patient authorization for the use systematic investigation, including research or disclosure of identifiable information,39 legally development, testing, and evaluation, designed to binding data use agreements for the release of develop or contribute to generalizable “limited data sets” between health information knowledge.”32 Commentary by HHS on the sources and users,40 the removal of specified Privacy Rule explicitly includes within this identifiers or statistical certification to achieve definition of research the development (building de-identification of health information,41 an and maintenance) of a repository or database for accounting of disclosures to be made available to future research purposes.33 The definition of patients at their request,42 and notification in the research in the Privacy Rule partially restates the event of a breach of unsecured protected health definition of research in the preexisting Common information to affected individuals who may be Rule for the protection of human subjects, adopted affected by the breach. In addition, if certain by HHS and certain other Federal agencies.34 criteria required by the Privacy Rule are satisfied, Some implications of this partial restatement of an IRB or privacy board may grant a waiver of the definition of research are discussed later in this individual patient authorization for the use or chapter. disclosure of health information in research.43 The National Institutes of Health (NIH) has 2.2.3 FDA Regulations published guidance, in collaboration with the U.S. Food and Drug Administration (FDA) Office for Civil Rights and other HHS offices and regulatory requirements for research supporting an agencies, on the impact of the Privacy Rule on application for FDA approval of a product also health services research and research databases include protections for human subjects, including and repositories. The NIH guidance identifies the specific criteria for protection of privacy and options available to investigators under the Privacy maintaining the confidentiality of research data.44 Rule to gain access to health information held by health care providers and insurance plans.35 For 2.2.4 Applicability of Regulations to Research; example, in addition to provisions for the use or Multiple-Purpose Registries disclosure of identifiable patient information for At many institutions, the IRB or the office that research, the Privacy Rule permits health care provides administrative support for the IRB is the providers and insurance plans (and business final arbiter of the activities that constitute human associates on their behalf) to use or disclose subjects research, and thus may itself determine patient information for certain defined public what activities require IRB review. A registry health activities.36 The Privacy Rule defines a developer is strongly encouraged to consult his or public health authority as “an agency or authority her organization’s IRB or a central IRB, as of the United States, a State, a territory, a political applicable, early in the registry planning process to subdivision of a State or territory, or an Indian avoid delays and lessen the need for multiple tribe, or a person or entity acting under a grant of revisions of documentation submitted to the IRB. authority from or contract with such public Distinctions between research and other activities agency… that is responsible for public health that apply scientific methodologies are frequently matters as part of its official mandate.”32 The unclear. Such other activities include both public Centers for Disease Control and Prevention and

153 Section II. Legal and Ethical Considerations for Registries

health practice45 and quality-related confidentiality concerns, including the Privacy investigations.46 Both the ostensible primary and Rule protections that may be applicable to secondary purposes of an activity are factors registries created by or maintained within covered considered in the determination of whether entities, such as health care providers and registry activities constitute research subject to the insurance plans, or business associates. For Common Rule. As interpreted by OHRP, if any example, information about certain conditions secondary purpose of an activity is research, then (such as alcoholism or HIV-positive status) and the activity should be considered research.47 This certain populations (such as children) may be OHRP interpretation of research purpose differs associated with a greater potential for harm from from that of the Privacy Rule with respect to social stigma and discrimination. Under these quality-related studies performed by health care circumstances, the IRB can make approval of a providers and insurance plans. Under the Privacy registry plan contingent on implementation of Rule, only if the primary purpose of a quality- additional safeguards that it determines are related activity is to obtain generalizable necessary to minimize the risks to the individuals knowledge do the research provisions of the contributing health information to the registry. Privacy Rule apply; otherwise, the Privacy Rule defines the activity as a “health care operation.”48 3. Applicable Regulations Registry developers should rely on their privacy officer’s and IRB’s experience and resources in This section discusses the specific applicability of 49 50 defining research and other activities for their the Common Rule and the Privacy Rule to the institutions and determining which activities creation and use of health information registries. require IRB review as research. In meeting Registry developers are strongly encouraged to accreditation standards, inpatient facilities consult with their organization’s privacy officer typically maintain standing departmental and IRB or privacy board early in the planning (e.g., pediatrics) or service (e.g., pharmacy or process to clarify applicable regulatory nursing) committees to direct, review, and analyze requirements and the probable effect of those quality-related activities. Some physician groups requirements on registry design and development. also establish and maintain quality-related This discussion assumes three general models for programs, because good clinical practice includes health information registries. One model is the ongoing evaluation of any substantive changes to creation of a registry containing the contact, the standard of care. These institutional quality demographic, and diagnostic or exposure committees can provide guidance on the activities information of potential research subjects who will that usually fall within their purview. Similarly, be individually notified about projects in which public health agencies typically maintain they may be eligible to participate. The notification systematic review processes for identifying the process permits the registry to shield registry activities that fit within their legal authority. participants from an inordinate number of As mentioned previously, use of registry data for invitations to participate in research projects, as multiple research purposes may entail obtaining well as to protect privacy and confidentiality. This additional permissions from patients or satisfying model is particularly applicable to patients with different regulatory requirements for each research unusual conditions, patients who constitute a 51 purpose. Standard confidentiality protections for vulnerable population, or both (e.g., children registry data include requirements for physical, with a rare condition). A second model is the technical, and administrative safeguards to be creation of a registry and the conduct of all incorporated into plans for a registry. In some subsequent research using registry data by the instances, an IRB may not consider legally same group of investigators. No disclosures of required protections for the research use of patient registry data will occur and all research activities information sufficient to address relevant have the same scientific purpose. This model

154 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

applies, in general, to quality improvement maintaining the confidentiality of patient registries and other quality-related investigations information used by public health authorities are of a clinical procedure or service. Note, however, similar to those for research use, but they generally that some quality improvement registries may are explicitly balanced against potential social involve confidential feedback to providers as well benefits during the legislative process. as public reporting of provider performance in a Nonetheless, if the registry supports human patient de-identified format. These activities may subjects research activities as well as its public or may not constitute research as defined by the health purposes, Common Rule requirements for Common Rule. Under the Privacy Rule, these IRB review may apply to the creation and activities may be regulated as the health care maintenance of the registry. operations of the covered entity that provides the Cancer registries performing public health data to the registry, rather than research, provided surveillance activities mandated by State law are the obtaining of generalizable knowledge is not the well-known exceptions to Common Rule primary purpose of the activities. A third model is regulation. However, secondary uses of public the creation of a registry for an initial, specific health registry data for research and the creation of purpose by a group of investigators with the registries funded by public health agencies, such as express intent to use registry data themselves, as the Centers for Disease Control and Prevention well as to disclose registry data to other and the Agency for Healthcare Research and investigators for additional related or unrelated Quality, may be subject to the Common Rule as scientific purposes. An example of this last model sponsored research activities. The Common Rule’s is a registry of health information from patients definitions of human subjects research34 may diagnosed with a condition that has multiple encompass these activities, which are discussed in known comorbidities to which registry data can be the next subsections of this chapter. Not all cancer applied. This third model is most directly registries support public health practice alone, applicable to industry-sponsored registries. The even though the registries are the result of American College of Epidemiology encourages governmental programs. For example, the the data sharing contemplated in this last registry 52 Surveillance Epidemiology and End Results model. Data sharing enhances the scientific (SEER) program, funded by the National Cancer utility of registry data and diminishes the costs of Institute, operates and maintains a population- compilation. based cancer reporting system of multiple The extent to which the regulations will apply to registries, including public use data sets with each of these registry models will depend on public domain software. SEER program data are factors such as the registry developer, purpose of used for many research purposes in addition to the registry, potential for individual patient aiding public health practices. These latter research identification, consent process, and inclusion of activities may be subject to the Common Rule.53 genetic information. These factors are discussed Disclosures of health information by health care further below. providers and insurance plans and their business associates for certain defined public health 3.1 Public Health, FDA-Regulated activities are expressly permitted by the Privacy Products, Health Oversight Rule without patient authorization.36 An example When Federal, State, or municipal public health of a public health activity is the practice of agencies create registries in the course of public surveillance, in which the distributions and trends health practice, specific legislation typically of designated risk factors, injuries, or diseases in authorizes the creation of the registries and populations are monitored and disseminated.54 regulates data acquisition, maintenance, security, Health care providers or insurance plans are likely use, and disclosures of registry data for research. to insist upon documentation of public health Ethical considerations and concerns about authority for legal review before making any

155 Section II. Legal and Ethical Considerations for Registries

disclosures of health information. Registry meet this definition constitute research for developers should obtain this documentation from purposes of this policy, whether or not they are the agency that funds or enters into a contract for conducted or supported under a program the registry, and present it to the health care which is considered research for other provider or insurance plan well in advance of data purposes. For example, some demonstration collection efforts. and service programs may include research 34 The Privacy Rule characterizes responsibilities activities. related to the quality, safety, or effectiveness of a OHRP interprets this Common Rule definition of product or activity regulated by FDA as public research to include activities having any research health activities. This public health exception purpose, no matter what the stated objectives of allows uses and disclosures of patient information the activity may be. Compliance with Common to a person subject to FDA jurisdiction with Rule requirements depends on the nature of the respect to FDA-regulated products or activities for organization where the registry resides. If an which the person has responsibility, such as for organization receives Federal funding for research, adverse event reporting; product tracking; product then it is likely that Common Rule requirements recalls, repairs, replacement, or look-back; and apply. postmarketing surveillance (e.g., as part of a risk The Privacy Rule’s definition of research32 restates management program that is a condition for 55 the first sentence of the Common Rule definition. approval of an FDA-regulated product). However, the Privacy Rule distinguishes between The Privacy Rule also permits uses and disclosures research and quality I/A conducted by covered by health care providers and insurance plans (and entities or their business associates,59 which are their business associates on their behalf) for defined as “health care operations.”32 Under the “health oversight activities” authorized by law.56 Privacy Rule, if the primary purpose of a quality- These activities include audits and investigations related registry maintained by a covered entity is necessary for oversight of the “health care system” to support a research activity (i.e., to create and other entities subject to government regulatory generalizable knowledge), Privacy Rule programs for which health information is relevant requirements for research apply to the use or to determining compliance with program disclosure of the patient information to create the standards.57 The collection of patient information, registry and to subsequent research use of registry such as occurrences of decubitus ulceration, from data. If, however, the primary purpose is other than nursing homes that are operating under a to create generalizable knowledge, the study is compliance or corporate integrity agreement with considered a health care operation of the covered a Federal or State health care program, is an entity and is not subject to Privacy Rule example of a health oversight activity. requirements for research activities, including the requirement to obtain patient authorization or a 3.2 Research Purpose of a Registry waiver of authorization from an IRB or privacy The Common Rule defines research, and its board for the uses or disclosures. definition is partially restated in the Privacy Rule, As noted earlier, both public health practice and as described above. These regulatory definitions quality I/A can be difficult to distinguish from affect how the regulatory requirements of each research activities.60 The determination of whether rule are applied to registry activities.58 a particular registry should be considered as or In the Common Rule: include a research activity depends on a number of different factors, including the nature of the Research means a systematic investigation, organization where the registry will reside; the including research development, testing, and employment duties of the individuals performing evaluation, designed to develop or contribute the activities associated with the registry; the to generalizable knowledge. Activities which source of funding for the registry; the original,

156 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

intended purpose of the registry; the sources of and a limited data set of information (specified registry data; whether subsequent uses or direct identifiers removed).62 The latter two disclosures of registry data are likely; and other categories of information may include certain circumstances of registry development. codes linked to identifiers, provided certain Quality I/A activities entail many of the same conditions are met. ethical concerns about protecting the If applicable, Common Rule requirements affect confidentiality of health information as research all research involving patient information that is activities do. Obtaining express patient consent to individually identifiable and obtained by the participate in quality I/A activities is not the usual investigator conducting the research. The practice; instead, the professional and cultural definition of “human subject” in the Common Rule norms of health care providers, both individual and is “a living individual about whom an investigator institutional, regulate these activities. Registry (whether professional or student) conducting developers should consider whether the ethical research obtains (1) data through intervention or concerns associated with a proposed quality I/A or interaction with the individual, or (2) identifiable patient safety registry require independent review private information.” and the use of special procedures such as notice to This regulatory definition further explains that: patients or providers. Registry advisory committee members, quality I/A and patient safety Private information includes…information literature,61 hospital ethics committees, IRB which has been provided for specific purposes members, and clinical ethicists can make valuable by an individual and which the individual can contributions to these decisions. reasonably expect will not be made public (for example, a medical record). Private To avoid surprises and delays, the decision about information must be individually identifiable the nature of the activity that the registry is (i.e., the identity of the subject is or may intended to support should be made prospectively, readily be ascertained by the investigator or in consultation with appropriate officials of the associated with the information) in order for funding agency and officials of the organization obtaining the information to constitute where the registry will reside. Some research research involving human subjects.63 institutions may have policies that either require IRB review for quality I/A, especially if In short, the Common Rule definition ofhuman publication of the activity is likely, or exclude subject makes all research use of identifiable them from IRB review. Frequently, IRBs make this patient information subject to its requirements; if determination on a case-by-case basis. the identity of the patients whose information is used for research purposes is not readily 3.3 Potential for Individual Patient ascertainable to the investigator, the research is not Identification human subjects research to which the Common The specific regulatory requirements applicable to Rule applies. Moreover, research involving the the use or disclosure of patient information for the collection of information from existing records is creation of a registry to support human subjects exempt from the Common Rule if the information research depend in part on the extent to which is recorded by the investigator in such a manner patient information received and maintained by the that subjects cannot be identified, directly or registry can be attributed to a particular person. through a coded link to identifiers. Registry Various categories of information, each with a developers should consult the IRB early in the variable potential for identifying individuals, are process of selecting data elements to obtain distinguished in the Privacy Rule: individually guidance about whether registry activities identifiable health information, de-identified constitute human subjects research or may be information (all identifying elements removed), exempt from Common Rule requirements.

157 Section II. Legal and Ethical Considerations for Registries

Also among the criteria specified by the Common standard, a limited data set of patient information Rule for IRB approval of research involving does not include specified direct identifiers of the human subjects are provisions to protect the patient, or the patient’s relatives, employer, or privacy of subjects and to maintain the household members.67 64 confidentiality of data. In addition, the consent In an electronic environment, masking of process for research subjects should include individual identities is a complex task. Data explicit information about the confidentiality suppression limits the utility of the information protections in place when records containing 65 from the registry. Linkage or triangulation of identifiers are going to be used. information can enable the re-identification of Data collection frequently requires patient individuals. A technical assessment of electronic identifiers, especially in prospective registries with records for their uniqueness within any data set is ongoing data collection, revision, and updates. necessary to minimize the potential for re- Secondary or subsequent research use by outside identification. In aggregated published data, investigators (i.e., those not involved in the standard practice assumes that a subgroup size of original data collection) of patient information less than six may also be identifiable, depending containing direct identifiers is complicated, on the nature of the data. An evaluation for however, because ethical principles for the conduct uniqueness should be performed to ensure that the of human subjects research require that risks, electronic format does not produce a potential for including risks to confidentiality of patient identification greater than this standard practice, identifiable information, be minimized. In including when the information is triangulated addition, the Privacy Rule requires patient within a record or linked with other data files. authorization to describe the purpose of the use or If a registry for research, public health, or other disclosure of patient information. In order for a purposes will use any of the categories of health patient authorization to allow for secondary information discussed below, a registry developer research uses of the patient information, the should consult the IRB, the privacy officer, and the authorization must adequately describe the institutional policies developed specifically in purpose of the future research, such that it would response to the Privacy Rule early in his or her be reasonable for the patient to expect that his or planning. These consultations should establish the her information could be used or disclosed for purpose of the registry, the applicability of the such purpose. Thus, unless the registry developer Common Rule requirements to registry activities, has anticipated and adequately described the and the applicability of the Privacy Rule to the purposes of the secondary research in the initial collection and use of registry data. In addition, the authorization received from a patient, that initial registry developer should consult a representative authorization may not constitute authorization for of the information technology or health the use of identifiable registry data for secondary information system office of each health care research purposes. In cases where there is no provider or insurance plan that will be a source of authorization for the secondary research, there data for the registry, as well as a representative of may be other options under the Privacy Rule for the IRB or privacy board for each data source, so secondary use of the data collected, such as as to obtain feasibility estimates of data de-identification of the information or the creation availability and formats. of a limited data set.66 Chapter 16 provides a discussion of the technical and legal considerations 3.3.1 De-Identified Patient Information related to linking registry data for secondary The Privacy Rule describes two methods for research purposes. de-identifying health information.68 One method Direct identifiers, as specified by the Privacy Rule, requires the removal of certain data elements. The include, for example, a patient’s name, contact other method requires that a qualified expert information, medical record number, and Social certify that the potential for identifying an Security number. As stated in the Privacy Rule individual from the data elements is very small.

158 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

A qualified expert should have “appropriate information as a result of monitoring patients over knowledge of and experience with generally time or information from multiple sources in order accepted statistical and scientific principles and to compile information on a health event methods for rendering information not individually (e.g., cancer incidence). Dates of service and identifiable” in order to make this determination.69 geographic location may be crucial to achieving De-identified information may include a code the purposes of the registry or to the integrity and permitting re-identification of the original record use of the data. Health information provided to the by the data source (covered entity), provided registry that is not fully de-identified but excludes certain conditions are met.70 The code may not be direct identifiers may constitute alimited data set derived from information about an individual, as defined by the Privacy Rule.67 A health care including hash codes,71 and should resist provider or insurance plan (or business associate translation. In addition, the decoding key must on behalf of a provider or plan if permitted by the remain solely with the health care provider or plan terms of the business associate agreement) may that is the source of the patient information, and disclose a limited data set of health information for the covered entity cannot use or disclose the code research, public health, or health care operations for any other purpose.70 purposes, provided it enters into a data use Research using existing data in which individual agreement (DUA) with the recipient. The terms of the DUA must satisfy specific Privacy Rule patients cannot be identified directly or indirectly 72 through linked identifiers does not involve human requirements. Institutional officials for both the subjects as defined by the Common Rule, and thus data source and the registry developer should sign is not subject to the requirements of the Rule.63 the DUA so that a legal contract results. The DUA Refer to the discussion later in this chapter. establishes the permitted uses of the limited data set by the registry developer (i.e., the creation of As a prudent business practice, each health care the registry and subsequent use of registry data for provider or insurance plan or their business specified research purposes). The DUA may not associate that is a source of de-identified authorize the registry developer to use or disclose information is likely to require an enforceable the information in a way that would result in a legal agreement with the registry developer. It violation of the Privacy Rule if done by the data should be signed by an appropriate institutional source.73 Furthermore, the DUA for a limited data official on behalf of the registry developer. At a set of health information must provide that the minimum, this agreement will likely contain the data recipient will appropriately safeguard the following terms, some of which may be negotiable: information and not attempt to identify individual the identification of the content of the data and the patients or to contact those patients.74 Certain medium for the data; a requirement that the data other requirements also apply. recipient, and perhaps the health care provider or insurance plan or their business associate An investigator who works for a health care providing the data, make no attempt to identify provider or insurance plan to which the Privacy individual patients; the setting of fees for data Rule applies and that is the source of the health processing and data use; limitations on disclosure information for a registry may use a limited data or further use of the data, if any; and an allocation set to develop a registry for its own research of the risks of legal liability for any improper use purpose. In these circumstances, the Privacy Rule of the data. still requires a DUA that satisfies the requirements of the Privacy Rule between the health care 3.3.2 Limited Data Sets of Health Information provider or insurance plan and the investigator. De-identified health information may not suffice to This agreement may be in the form of a written 75 carry out the purposes of a registry, especially if confidentiality agreement. the registry is designed to receive followup

159 Section II. Legal and Ethical Considerations for Registries

A registry developer may assist a health care registry involves human subjects; frequently, a provider or insurance plan or their business special form for this purpose is available from the associate by creating the limited data set.67 In IRB. The IRB (or institutional official) should some situations, this assistance may be crucial to provide the registry developer with documentation ensuring that data are accessible and available to of its decision. the registry. In order for the registry developer to 3.3.3 Direct Identifiers: Authorization and create a limited data set on behalf of a data source, Consent the Privacy Rule requires that the data source (the covered entity or their business associate) and the As discussed above, the Privacy Rule permits the registry developer (in this instance acting as a use or disclosure of patient information for business associate) enter into a business associate research with a valid, written authorization from agreement that satisfies certain regulatory each patient whose information is used or criteria.76 The business associate agreement is a disclosed.78 The Privacy Rule specifies the content binding legal arrangement that should be signed by of this authorization, which gives permission for a appropriate institutional officials on behalf of the specified use or disclosure of the health data source and registry developer. This agreement information.79 Health care providers and insurance contains terms for managing health information as plans frequently insist on using the specific required by the Privacy Rule.76 Most health care authorization forms that they have developed in providers have developed a standard business order to avoid additional legal review and associate agreement in response to the Privacy minimize any potential liability that they believe Rule and will likely insist on using it, although might be associated with use of other forms. some modifications may need to be negotiated in One exception to the requirement for an order to produce registry data. A registry developer authorization occurs when a health care provider hired to create a limited data set must return or or insurance plan creates a registry to support its destroy the direct identifiers once the business “health care operations.”80 The Privacy Rule’s associate relationship formed for purposes of definition of “health care operations” explicitly creating the limited data set terminates and the includes quality I/A, outcomes evaluation, and the registry developer now wants to use the limited development of clinical guidelines, provided that data set (subject to the required data use the obtaining of generalizable knowledge is not the agreement) for the registry purposes. primary purpose of any studies resulting from such The registry populated with a limited data set may activities.32 For example, a hospital registry include a coded link that connects the data back to created to track its patient outcomes against a patient records, provided the link does not recognized clinical care standard as a quality replicate part of a direct identifier.67 The key to the improvement initiative has a health care operations code (e.g., encryption key) may allow health purpose. The hospital would not be required to information obtained from patients over time to obtain authorizations from its patients for use or supplement existing registry data or allow the disclosure of the health information it tracks in this combination of information from multiple sources. registry. If the registry data obtained by investigators Research use of health information containing constitute a limited data set, then the research does identifiable information constitutes human subjects not involve human subjects, as defined by HHS research as defined by the Common Rule.63 In regulations at 45 Code of Federal Regulations general, the Common Rule requires documented, (CFR) 46.102(f), and the Common Rule legally effective, voluntary, and informed consent requirements would not apply to the registry.77 of each research subject.81 An IRB or an institutional official knowledgeable Documentation of the consent process required by about the Common Rule requirements should the Common Rule may be combined with the make the determination of whether a research authorization required by the Privacy Rule for

160 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

disclosure and use of health information.82 IRBs may approve waivers or alterations of both However, registry developers should be aware that authorization (for use or disclosure of patient a health care provider or insurance plan may not information for registry purposes) and consent (to immediately accept the combined form as a valid registry participation), provided the research use of authorization and may insist on legal review of the health information satisfies certain regulatory combined form before disclosing any health conditions. In addition, the Privacy Rule provides information. for the ability of privacy boards to approve waivers Authorizations for the use or disclosure of health of authorization for the research use of health information where an organization does not have information under the Privacy Rule and informed 83 consent to participate in research under the an IRB. Waivers are discussed in detail below. Common Rule must be legally effective In certain limited circumstances, research subjects (i.e., obtained from a legally competent subject or can consent to future unspecified research using the subject’s legally authorized representative and their identifiable patient information. The documented in a manner consistent with the Common Rule permits an IRB-approved consent regulations and applicable laws of the jurisdiction). process to be broader than a specific research Adults, defined in most States as persons who are project81 and to include information about research at least 18 years old, are generally presumed that may be done in the future. In its review of legally competent in the absence of a judicially such future research, an IRB can subsequently approved guardianship. Minors commonly are determine that the previously obtained consent defined as individuals less than 18 years old and (1) satisfies or (2) does not satisfy the regulatory are presumed legally incompetent; therefore, a requirements for informed consent. If the biological, adoptive, or custodial parent or previously obtained consent is not satisfactory, an guardian generally must provide consent and additional consent process may be required; authorization on the minor child’s behalf, unless alternatively, the IRB may grant a waiver of the child has been legally emancipated or another consent, provided the regulatory criteria for a exception applies. Registry developers should waiver are satisfied. consult legal counsel about situations in which As such, an IRB-approved consent process for the these presumptions seem inapplicable, such as creation of a research registry should include a when a registry is created to investigate description of the specific types of research to be contraceptive drug and device use by adolescents, conducted using registry data. For any future or other situations in which State or Federal law research that involves private identifiable exceptions may apply. information maintained by the registry, the IRB In addition to being voluntary and legally may determine that the original consent process effective, an individual’s consent should be (for the creation of the research registry) satisfies informed about the research, including what the applicable regulatory requirements because the activities are involved, as well as the expected risks prospect of future research and future research and potential benefits from participation. The projects were adequately described. The specific Common Rule requires the consent process to details of that future research using registry data include specific elements of information.81 may not have been known when data were Registry developers should provide non–English- collected to create the registry, but that research speaking patients with appropriate resources to may have been sufficiently anticipated and ensure that the communication of these elements described to satisfy the regulatory requirements for during the consent process is comprehensible. All informed consent. For consent to be informed as written information for patients should be demanded by the ethical principle of respect for translated, or else arrangements should be made persons, however, any description of the nature and for qualified translators to assist in the consent purposes of the research should be as specific as process. possible.

161 Section II. Legal and Ethical Considerations for Registries

If a registry developer anticipates subsequent legally bound by the limited purpose of any research use of identifiable private registry data, he original authorization that was obtained to permit or she should request an assessment by the IRB of data sources to disclose identifiable patient the description of the research that will be used in information to the registry. However, data sources the consent process for potential subjects at the or their business associates that are subject to the time the data are initially collected. Nonetheless, Privacy Rule are unlikely to be willing to provide in its review of any subsequent research, an IRB patient information without a written agreement may require an additional consent process for each with the registry developer that includes legally research subject or may grant a waiver for enforceable protections against redisclosure of obtaining further consent. identifiable patient information. Regardless of Historically, HHS rejected broadening the whether such a written agreement is in place, a description of purpose in authorizations under the valid authorization must contain a warning to Privacy Rule to allow for future unspecified patients that their health information may not be 84 protected by Privacy Rule protections once research. As a result, an authorization for the use 88 or disclosure of health information to create a disclosed to recipient organizations. research registry could not also authorize the In addition to (or in lieu of) obtaining health future research uses of the information if the information from covered entities or business specific details of the future studies were not associates, registry developers can request that known when the authorization was obtained.85 patients obtain and share with the registry copies However, under the modified HIPAA Privacy Rule of their own records from their health care released on January 25, 2013, HHS modified its providers or insurance plans. This strategy can be prior interpretation and guidance that research useful for collecting data on mobile populations, authorizations must be research study specific.86 such as elderly retirees who occupy different While this modification does not make any residences in winter and summer, and for changes to the authorization requirements at 45 collecting the health records of school children. A CFR § 164.508, HHS no longer interprets the Federal privacy law89 protects the health records of “purpose” provision for authorizations as children that are held by schools from disclosure permitting only study-specific descriptions. This without explicit parental consent; thus, parents can change now allows future research to be authorized often obtain copies of these records more easily provided the authorization adequately describes than investigators. Alternatively, individuals can the purposes of any future research such that it simply be asked to volunteer health information in would be reasonable for the individual to expect response to an interview or survey. These that his or her health information could be used or information collection strategies do not require disclosed for such future research.87 Where an obtaining a Privacy Rule authorization from each authorization for the use or disclosure of registry subject; however, IRB review and other data for the future research does not exist, a health requirements of the Common Rule, including care provider or health insurance plan maintaining careful protections of the confidentiality of registry the registry may need to obtain an additional data still may apply to a registry project with a authorization for the research from individuals or research purpose. Moreover, a registry developer seek a waiver of authorization from an IRB or may encounter Privacy Rule requirements for the privacy board. Alternatively, the use or disclosure use or disclosure of patient information by a health of a limited data set or de-identified registry data care provider or insurance plan for purposes of can occur, provided regulatory criteria are recruiting registry participants. For example, a satisfied. Registries maintained by organizations to patient authorization or waiver of authorization which the Privacy Rule does not apply (e.g., (discussed below) may be necessary for the funding agencies for research that are not health disclosure of patient contact information by a care providers or insurance plans, professional health care provider or insurance plan (or their societies, or non-health care components of hybrid business associate) to a registry developer. entities such as in many universities) are not

162 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

3.3.4 Certificates of Confidentiality and Other Registry developers should also be aware that Privacy Protections Federal law provides specific confidentiality Certificates of confidentiality granted by the NIH protections for the identifiable information of patients in drug abuse and alcoholism treatment permanently protect identifiable information about 96 research subjects from legally compelled programs that receive Federal funding. These disclosure. For the purposes of certificates of programs may disclose identifiable information confidentiality, identifiable information is broadly about their patients for research activities only with the documented approval of the program defined to include any item, or combination of 97 items, in research data that could directly or director or authorization of the patient. The basis indirectly lead to the identification of a research for the director’s approval is receipt of written participant.90 Federal law authorizes the Secretary assurances about the qualifications of the of HHS (whose authority is delegated to NIH) to investigator to conduct the research and the provide this privacy protection to subjects of confidentiality safeguards incorporated into the biomedical, behavioral, clinical, and other research protocol, and an assurance that there will research.91 Federal funding for the research is not be no further disclosure of identifying information a precondition for obtaining a certificate of by the investigator. Moreover, an independent confidentiality.92 An investigator whose research review of the research project should determine project has been granted a certificate of and verify in writing that the protocol provides confidentiality may refuse to disclose identifying adequate protection of the rights and welfare of the patients and that the benefits of the research information collected for that research even though 97 a valid subpoena demands that information for a outweigh any risks to patients. Prior to civil, criminal, administrative, or legislative submitting proposed consent documentation to an proceeding at the Federal, State, or local level. The IRB, registry developers should consult legal protection provided by a certificate of counsel about the limitations of these confidentiality is intended to prevent the disclosure confidentiality protections. of personal information that could result in adverse As a condition of approval, IRBs frequently effects on the social, economic, employment, or require investigators to obtain a certificate of insurance status of a research subject.93 Detailed confidentiality for research involving information information about certificates of confidentiality is about substance abuse or other illegal activities available on the NIH Web site.94 (e.g., underage purchase of tobacco products), The grant of a certificate of confidentiality to a sexual attitudes and practices, and genetic research project, however, is not intended to affect information. Registry developers should consult State laws requiring health care and other legal counsel to determine if and how the professionals to report certain conditions to State limitations of a certificate of confidentiality may officials; for example, designated communicable affect privacy protection planning for registry data. diseases, neglect and abuse of children and the In all circumstances, the consent process should elderly, or threats of violent harm. If investigators ensure that clear notice is given to research are mandatory reporters under State law, in subjects about the extent of privacy protections general, they continue to have a legal obligation to they may expect for their health information when make these reports.95 In addition, other limitations it is incorporated into a registry. to the privacy protection provided by certificates of In the absence of a certificate of confidentiality, a confidentiality exist and may be relevant to valid subpoena or court order for registry data will particular research projects. Information on the usually compel disclosure of the data unless State NIH Web site describes some of these other legal law specifically protects the confidentiality of data. limitations.94 For example, Louisiana’s laws specifically protect the collection of information related to tobacco use

163 Section II. Legal and Ethical Considerations for Registries

from subpoena.98 On the other hand, a subpoena or The Privacy Rule and the Common Rule each court order may supersede State law confidentiality specify the criteria under which waivers or protections. These legal instruments can be alterations of authorization and the consent challenged in the court having jurisdiction for the process are permitted.100 There are potential risks underlying legal proceeding. In some to patients participating in the registry resulting circumstances, research institutions may be willing from these waivers of permission. A waiver of to pursue such a challenge. The remote yet definite authorization potentially imposes the risk of a loss possibility of this sort of disclosure should be of confidentiality and consequent invasion of clearly communicated to research subjects as a privacy. A waiver of consent potentially imposes limitation on confidentiality protections, both risks of harm from the loss of self-determination, during the consent process and in an authorization dignity, and privacy expected under the ethical for use or disclosure of patient information. principles of respect for persons and beneficence. State law may assure the confidentiality of certain Acknowledging these potential risks, regulatory quality I/A activities performed by health care criteria for waiver and alterations require an IRB providers as peer review activities.99 When State or privacy board to determine that risks are law protects the confidentiality of peer review minimal, in addition to other criteria. This activities, generally, it is implementing public determination is a necessary condition for approval policy that encourages internal activities and of an investigator’s request for a waiver or initiatives by health care providers to improve alteration of these permissions. health care services by reducing the risks of The following discussion refers only to waivers; medical errors and systematic failures. Protection registry developers should note that privacy boards by peer review statutes may limit the use of data and IRBs may approve alterations to authorizations generated by quality I/A activities for any other or the consent process, provided a requested purposes. alteration satisfies all of the criteria required for a waiver by the Privacy Rule or Common Rule. 3.3.5 Waivers and Alterations of Authorization Alterations are generally preferable to waivers in and Consent an ethical analysis based on the principle of As mentioned above, the Privacy Rule provides for respect for persons, because they acknowledge the the ability of privacy boards and IRBs to importance of self-determination. In requesting sometimes waive or alter authorizations by alterations to an authorization or to the consent individual patients for the disclosure or use of process, registry developers should be prepared to health information for research purposes. (See justify each proposed change or elimination of Case Example 13.) In addition, the Common Rule required elements (such as description of authorizes IRBs to waive or alter the consent alternative procedures, courses of treatment, or process. It is important for registry developers to benefits). Plausible justifications include a registry keep distinct the terms “consent” and to which a specific element does not apply or a “authorization,” as they are not interchangeable registry in which one element contradicts other with respect to the Privacy Rule and Common required information in the authorization or Rule. As described above, authorization is the term consent documentation. The justifications for used to describe individuals’ written agreement to alterations should relate as specifically and directly the use or disclosure of their health information as possible to the regulatory criteria for IRB or under the Privacy Rule, while consent is the term privacy board approval of waivers and alterations. used to describe research subjects’ agreement to The Privacy Rule permits a covered entity to participate in research, as required by the Common obtain the approval of an IRB or privacy board for Rule. There are separate requirements for each of a waiver of authorization if the following criteria these permissions. are met: (1) the use or disclosure involves no more than minimal risk to the privacy of individuals;

164 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

(2) the research cannot practicably be conducted justification of avoiding selection bias. A waiver without the waiver; and (3) the research cannot be permits the registry to include the health practicably conducted without access to, and use information of all patients who are eligible. An of, the health information. The determination of IRB may also agree to consider requests for a minimal risk to privacy includes several elements: limited waiver of consent that applies only to those an adequate plan to protect identifiers from individuals who decline use of their health improper use or disclosure; an adequate plan to information in a registry project. This limited destroy identifiers at the earliest opportunity, waiver of consent most often permits the collection unless a health or research justification exists to of de-identified and specified information retain them; and adequate written assurances that sufficient to characterize this particular population. the health information will not be reused or An important difference between the Common disclosed to others, except as required by law, as Rule and FDA regulations for the protection of necessary for oversight of the research, or as human subjects involves consent to research permitted by the Privacy Rule for other 101 participation. The FDA regulations require research. The registry developer should provide consent, except for emergency treatment or detailed documentation of the IRB or privacy research, and do not permit the waiver or alteration board’s decision to the health care provider or of informed consent.105 If registry data are insurance plan (covered entity) that is the source of 43 intended to support the labeling of an FDA- the health information for registry data. The regulated product, a registry developer should plan documentation should clearly communicate that to obtain the documented, legally effective, each of the criteria for a waiver required by the 102 voluntary, and informed consent of each individual Privacy Rule has been satisfied. The privacy whose health information is included in the board or IRB documentation should also provide a registry. description of the health information it determined to be necessary to the conduct of the research and The Common Rule also permits an IRB to waive the procedure it used to approve the waiver.103 A documentation of the consent process under two health care provider or insurance plan might insist different sets of regulatory criteria. The first set of on legal review of this documentation before conditions for approval of this limited waiver disclosing any health information. requires that the only record linking an individual subject to the research is the consent document, The criteria for a waiver of consent in the and that the principal risk to subjects is the Common Rule are similar to those for a waiver of potential harm from a breach of confidentiality. authorization under the Privacy Rule. An IRB Each subject individually determines whether his should determine that: (1) the research involves no or her consent should be documented.106 more than minimal risk to subjects; (2) the waiver Alternatively, an IRB can waive documentation of will not adversely affect the rights and welfare of consent if the research involves no more than subjects; (3) the research cannot practicably be minimal risk of harm to subjects and entails no carried out without a waiver; and (4) whenever procedures for which written consent is normally appropriate, subjects will be provided with 107 104 obtained outside of a research context. For additional information after participation. The either set of regulatory criteria, the IRB may criterion for additional information can be satisfied require the investigator to provide subjects with at least in part by public disclosure of the written information about the research activities in purposes, procedures, and operations of a registry, which they participate.108 The written information as discussed in Section 4.1. may be as simple as a statement of research Some IRBs produce guidance about what purposes and activities, or it may be more constitutes “not practicable” justifications and the elaborate, such as a Web site for regularly updated circumstances in which justifications are information describing the progress of the research applicable. For population-based research projects, project. registry developers may also present the scientific

165 Section II. Legal and Ethical Considerations for Registries

The Privacy Rule creates a legal right for patients include any data, reports, records, memoranda, to receive, upon request, an accounting of certain analyses, and statements that can improve patient disclosures of their health information that are safety, health care quality, or health care outcomes, made by health care providers, insurance plans, provided that all such data must be developed for and their business associates.109 The accounting the purpose of reporting it to a PSO. Certain must include disclosures that occur with a waiver categories of information are expressly excluded of authorization approved by a privacy board or from being PSWP. These include “a patient’s IRB. The Privacy Rule specifies the information medical record, billing and discharge information, that an accounting should contain110 and requires or any other original patient or provider it to cover a six-year period or any requested information...[and] information that is collected, shorter period of time.111 If multiple disclosures maintained, or developed separately, or exists are made to the same recipient for a single separately, from a patient safety evaluation purpose, including a research purpose, a summary system.”116 of these disclosures may be made. In addition, Once PSWP is received by a PSO, it may be because most waivers of authorization cover aggregated and analyzed by the PSO to assist a records of many individuals, and thus an provider in determining, among other things, individualized accounting in such circumstances certain quality benchmarks and underlying causes may be burdensome or impossible, the Privacy of patient risks. Under the PSQIA, PSWP is Rule provides that if the covered entity has considered privileged and confidential, and it may disclosed the records of 50 or more individuals for not be disclosed unless certain requirements are a particular research purpose, the covered entity met. In addition, the Privacy Rule’s limitations on may provide to the requestor a more general uses and disclosures may apply where PSWP accounting, which lists the research protocols for includes PHI. Civil money penalties may be which the requestor’s information may have been imposed for violations.117 disclosed, among other items.112 The Patient Safety Rule permits PSOs to disclose 3.3.6 Patient Safety Organizations PSWP—that is, they may release, transfer, provide The final rule (the “Patient Safety Rule”) access to, or otherwise divulge PSWP to another implementing the Patient Safety and Quality person—in certain circumstances, including where Improvement Act of 2005 (PSQIA) became disclosures are authorized by the identified health effective on January 19, 2009.113 The PSQIA care providers, limited to nonidentifiable PSWP, or statute was enacted in response to a 1999 report by made to FDA, among other specified the Institute of Medicine that identified medical circumstances.118 With respect to disclosure of errors as a leading cause of hospital deaths in the PSWP for purposes of research, the regulations United States, with many such errors being provide a narrow exception. The Rule allows for preventable.114 In accordance with the statute, the disclosure of identifiable PSWP to entities Patient Safety Rule allows health care providers to carrying out “research, evaluation, or voluntarily report patient safety data, known as demonstration projects authorized, funded, patient safety work product (PSWP), to certified, or otherwise sanctioned by rule or other independent patient safety organizations (PSOs). means by the Secretary [of Health and Human In general, PSWP falls into three general Services], for the purpose of conducting categories: (1) information collected or developed research.”119 Notably, the disclosure of PSWP for by a provider for reporting to a PSO and actually general research activities is not permitted under reported; (2) information developed by the PSO the PSQIA statute or the Patient Safety Rule. itself as part of patient safety activities; and An organization desiring to become a PSO must (3) information that identifies or constitutes the complete and submit a certification form to the deliberations or analysis of, or identifies the fact of Agency for Healthcare Research and Quality to 115 reporting to, a patient safety evaluation system. become “listed” as a PSO.120 A registry may The Patient Safety Rule broadly defines PSWP to choose to become listed as a PSO; however, the

166 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

registry should consider whether the obligations ability to obtain informed consent for future imposed on it in its capacities as a PSO would research studies, among other differences. limit or otherwise restrict its attainment of its Citing more uniform regulations in other countries, original objectives and whether it can fully meet 121 the IOM Report affirmed that “a new direction is the regulatory requirements that apply to a PSO. needed, with a more uniform approach to patient In evaluating whether or not to be listed as a PSO, protections, including privacy, in health the registry developer should carefully review the research.”123 As its primary recommendation, the registry’s organizational structure and data IOM Committee held that Congress should collection processes to help ensure that there is a authorize HHS and other Federal agencies to clear distinction between the collection of registry- develop a new approach to protecting privacy that related data and PSWP. For example, certain would apply uniformly to all health research and to registries may publish certain information and exempt health research from the Privacy Rule results related to the data collected in the registry. when this new approach was implemented. Until As described above, if that registry is a PSO, then such an overhaul could be accomplished, the IOM it must ensure that publication of such data does Committee called upon HHS to revise the Privacy not constitute unauthorized disclosure for purposes Rule and associated guidance to address certain of the PSQIA (as well as HIPAA, if applicable). issues. HHS addressed some of these issues in the Finally, it is important for registry developers to January 25, 2013, modifications to the Privacy know that, instead of becoming a PSO itself, a Rule, such as by allowing HIPAA authorizations to registry may elect to form a separate division or encompass future research and removing legal organization that it controls and segregate prohibitions on combining certain HIPAA PSWP into that component. These types of PSOs authorizations for multiple research studies, are referred to as “Component PSOs.” By keeping thereby harmonizing these HIPAA Privacy Rule registry data and PSWP separate, the registry can requirements with the Common Rule. reduce the possibility of an impermissible Nevertheless, registry operators should be aware disclosure of PSWP. that additional clarifications or modifications to the Privacy Rule as it relates to research activities may 3.4 Developments Affecting the HIPAA continue to be made in the future. Privacy Rule 3.4.2 The Genetic Information 3.4.1 The Institute of Medicine Report Nondiscrimination Act of 2008 On February 4, 2009, the Institute of Medicine The Genetic Information Nondiscrimination Act of (IOM) published a report that examined how 2008 (GINA) was signed into law on May 21, research was being conducted within the 2008. In general, GINA prohibits discrimination in framework of the Privacy Rule. The IOM Report health insurance coverage (Title I) and presented findings and recommendations of an employment (Title II) based on genetic IOM Committee tasked with assessing the impact information. GINA defines genetic information as, of the HIPAA Privacy Rule on health research. with respect to any individual, information about This group had proposed recommendations to the individual’s genetic tests, the genetic tests of ensure that important health research might be the individual’s family members, and the conducted while maintaining or strengthening manifestation of a disease or disorder in the privacy protections for research subjects’ health individual’s family members (e.g., family health information.122 The IOM Report stated that certain history). Title I of GINA took effect for most Privacy Rule requirements were difficult to health insurance plans on May 22, 2009, and Title reconcile with other regulations governing the II became effective for employers on November conduct of research, including the Common Rule 21, 2009. GINA also specifies that the definition of and the FDA regulations, and it noted a number of genetic information includes the genetic inconsistencies among applicable regulations information of a fetus carried by a pregnant related to the de-identification of data and the woman and an embryo legally held by an

167 Section II. Legal and Ethical Considerations for Registries

individual or family member utilizing an assisted and the business associate agreement. While many reproductive technology. Pursuant to GINA, health business associate agreements previously insurers are prohibited from using the genetic contained general safeguarding requirements information of individuals for underwriting (e.g., requiring the business associate to maintain purposes (e.g., determining health insurance appropriate technical safeguards), these eligibility and coverage, or premium setting), and agreements often had not imposed specific security employers are prohibited from using genetic requirements (e.g., a requirement that the business information in making employment-related associate implement procedures to terminate an decisions. electronic session after a predetermined time of In addition to its nondiscrimination requirements, inactivity). The HITECH provisions now subject GINA also required related amendments to the business associates to civil and criminal penalties Privacy Rule to clarify that genetic information is once reserved only for covered entities under the health information for purposes of the Privacy HIPAA Rules. The HITECH Act obligations Rule, and to prohibit certain health plans from imposed on business associates were finalized using or disclosing genetic information for through HHS rulemaking on January 25, 2013, underwriting purposes.124 with compliance required by September 23, 2013.31 3.4.3 The HITECH Act The HITECH Act also created new breach The American Recovery and Reinvestment Act of notification requirements for covered entities and 2009 (ARRA) was signed into law on February 17, business associates. Following a breach of 2009. Funds appropriated as a result of passage of unsecured protected health information, covered ARRA are supporting new registries developed to entities must notify the affected individuals, HHS, study comparative effectiveness of treatments and and in some cases, the media. Notification must be protocols. It should be noted that there are no provided without unreasonable delay but in no specific exceptions to regulatory or ethical case later than 60 calendar days after the breach is requirements for such comparative effectiveness discovered (except in cases of reports to HHS of registries. Title XIII of division A and Title IV of breaches affecting less than 500 individuals, in division B of ARRA, the Health Information which case, notification to HHS is required not Technology for Economic and Clinical Health Act later than 60 days after the end of the calendar year (HITECH Act) significantly modified the in which the breach was discovered). Depending obligations of HIPAA covered entities and the on the circumstances, individual notification may business associates who perform certain services include both direct, written notification to affected on behalf of covered entities. individuals via first-class mail or email, as well as Perhaps most significantly, the HITECH Act substitute notice via conspicuous posting on the extends to business associates direct liability for entity’s Web site or in major print or broadcast compliance with many of the key privacy and media. security obligations contained in the HIPAA Rules, If a business associate experiences a breach of any where before business associates were required to unsecured protected health information it protect (and liable for failing to protect) the maintains, the business associate must provide information to which they had access only through notification to the applicable covered entity their business associate contracts with covered without unreasonable delay, and in no case later entities. Specifically, the HITECH Act imposed than 60 calendar days after the breach is direct liability on business associates for discovered, so that the covered entity can provide compliance with the administrative, physical, and the notifications described above with respect to technical safeguard requirements for electronic the breach or delegate that responsibility to the protected health information under the HIPAA business associate. Any notification by a business Security Rule, as well as for compliance with the associate must include the identification of any use and disclosure provisions of the Privacy Rule individual(s) whose information was accessed,

168 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

acquired, or disclosed during the breach. The strongly encouraged to consult the IRB, not only Department issued an interim final rule to about the applicability of the Common Rule, but implement the breach notification requirements on also about the selection of data elements, the August 24, 2009, which became effective on content of the consent process or the regulatory September 23, 2009. On January 25, 2013, the criteria for waiver, and any anticipated future Department published modifications to and made research involving identifiable registry data. permanent these breach notification 125 State laws regulate public health activities and may requirements. also apply in various ways to the research use of 3.4.4 Summary of Regulatory Requirements health information. NIH can issue certificates of confidentiality to particular research projects for The use and disclosure of health information by the protection of identifiable personal information health care providers and insurance plans for from most legally compelled disclosures. Federal research purposes, including for registries, are law provides specific privacy protections to the assumed by the authors of this chapter to be health information of patients in substance abuse subject to regulation under the Privacy Rule and programs that receive Federal funding. The may be subject to the Common Rule. institutional policies of health care providers and In general, the Privacy Rule permits a covered insurance plans may also affect the use and entity (or business associate on its behalf) to use or disclosure of the health information of their patient disclose patient information for registry purposes, or insured populations. subject to specific conditions, where the use or Legal requirements applying to use or disclosure disclosure is: (1) for a registry supporting certain of health information for research are evolving and public health activities, including registries can significantly influence the planning decisions developed in connection with FDA-regulated of registry developers and investigators. It is products; (2) for a registry supporting the health prudent to obtain early and frequent consultation, care operations of a health care provider or as necessary, with institutional privacy officers, insurance plan (covered entities), such as for privacy board or IRB staff and members, quality I/A; (3) for a registry created by health information system representatives of health care oversight authorities for health system oversight providers and insurance plans, plus technology activities authorized by law; (4) limited to de- transfer representatives and legal counsel. identified health information; (5) limited to a “limited data set” of patient information that lacks specified direct identifiers, and a data use 4. Registry Transparency, agreement is in place with the recipient; Oversight, and Data (6) pursuant to patient authorizations; or Ownership (7) consistent with a waiver or alteration of authorization by an IRB or privacy board. 4.1 Registry Transparency The Common Rule will apply to the creation and use of registry data if (1) the registry is funded by Efforts to make registry operations transparent HHS or the organization responsible for the (i.e., to make information about registry operations registry has an FWA that encompasses the registry public and readily accessible to anyone who is project, regardless of funding, and (2) the creation interested) are desirable. Such efforts may be of the registry and subsequent research use of the crucial to realizing the potential benefits of registry data constitute nonexempt human subject research using health information. Registry research as defined by the Common Rule. As transparency can also educate about scientific interpreted by OHRP, human subject research processes. Transparency contributes to public and includes registry activities which have a research professional confidence in the scientific integrity purpose, which may be in addition to the main and validity of registry processes, and therefore in purpose of the registry. Registry developers are the conclusions reached as a result of registry activities. Public information about registry

169 Section II. Legal and Ethical Considerations for Registries

operations may also increase the scientific utility committee(s) comprised of representatives of all of registry data by promoting inquiries from stakeholders in the registry, including scientists with interests to which registry data may investigators, the funding agency, patients, apply. clinicians, biostatisticians, information technology Registry developers can promote transparency by specialists, and government agencies. making the registry’s scientific objectives, Registry developers should also consider governance, eligibility criteria, sampling and appointing an independent advisory board to recruitment strategies, general operating protocol, provide oversight of registry operations. An and sources of data available to anyone who is advisory board can assist registry operations in interested. Proprietary interests of funding two important ways: (1) providing guidance for the agencies, contractual obligations, and licensing technical aspects of the registry operations and terms for the use of patient or claims information (2) establishing the scientific independence of the may limit, to some extent, the information registry. The latter function can be valuable when available to the public about the registry. It is controversies arise, especially those related to important to stress that, while transparency and patient safety and treatment, or resulting from access to information are to be encouraged, the actions by a regulatory agency. Advisory boards intent is not to discourage or criticize investments collectively should have relevant technical in patient registries that produce proprietary expertise, but should also include representatives information. Neither the funding source nor the of other registry stakeholders, including patients. generation of proprietary information from a Advisory board actions should be limited to registry determines whether a registry adheres to making recommendations to the ultimate the good practices described in this handbook. decisionmaker, whether an executive committee or Funding agencies, health care providers, and the registry developer. insurance plans do, however, have an important Registry developers may also appoint other types stake in maintaining public confidence in how of oversight committees to resolve specific health information is managed. The extent of recurring problems, such as verifying diagnoses of registry transparency should be prospectively patient conditions or adjudicating data negotiated with these entities. inconsistencies. Creating a Web site of information about registry objectives and operations is one method of 4.3 Data Ownership achieving transparency; ideally, registry 4.3.1 Health Information Ownership in General information should be available in various media. An IRB may require registry transparency as a The Privacy Rule was not intended to affect condition of approval to satisfy one of the existing laws governing the ownership of health regulatory criteria for granting a waiver of consent, records.128 However, multiple entities are often in which is to provide “additional pertinent a position to assert ownership claims to health information after participation.”126 For those information in various forms. For example, certain interested, a useful example of registry States have enacted laws that assign ownership of transparency can currently be found on an health records. At the current time, such claims of international transplant registry Web site.127 ownership are plausible, but none are known to be legally tested or recognized, with the exception of 4.2 Registry Oversight copyright claims. Entities that could claim ownership include health care providers and Registry governance must reflect the nature and insurance plans, funding agencies for registry extent of registry operations. As described in projects, research institutions, and government Chapter 2, governing structures can vary widely, agencies. Notably, State law requires health care from one in which the registry developer is the providers to maintain documentation of the sole decisionmaker to a system of governance by

170 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

services they provide. This documentation is the necessary, in business associate agreements medical-legal record compiled on each patient who required by the Privacy Rule . Subsequent research receives health care services from an individual or use of the registry data will likely be based on the institutional provider. Individuals, including terms of the original license. patients (who may have a potential liberty interest Among the changes ARRA has made in the in maintaining control of its use), registry regulation of health care information is a developers, and investigators, may also assert prohibition on its sale, subject to certain ownership claims to health information. The basis exceptions, including a limited one for the for these claims is control of the tangible disclosure of health information for research expression of and access to the health information. purposes. This exception permits covered entities There is no legal basis for assertions of ownership to recover a reasonable cost-based fee to cover the of facts or ideas; in fact, established public policy cost for the preparation and transmittal of the supports the free exchange of ideas and wide health information for the research purpose.131 dissemination of facts as fundamental to 129 For academic institutions, publication rights are an innovation and social progress. However, as a important component of intellectual property rights tangible expression of health information moves in data. Formal institutional policies may address from its creation to various derived forms under publication rights resulting from faculty the control of successive entities, rights of educational and research activities. Moreover, the ownership may be transferred (assigned), shared, social utility and benefit of any registry is evaluated or maintained, with use of the information under a on the basis of its publicly known findings and any license (i.e., a limited transfer of rights for use conclusions based on them. The authors strongly under specific terms and conditions). Currently, in encourage registry developers to maximize public each of these transactions, the rights of ownership communication of registry findings through the are negotiated on a case-by-case basis and customary channels of scientific conferences and formalized in written private agreements. The peer-reviewed journals. The goal of public funding agency for a registry may also assert communication for scientific findings and claims to ownership as a matter of contract law in conclusions applies equally to registries operated their sponsorship agreements with research outside of academic institutions (i.e., directly by organizations. industry or professional societies). For further Many health care providers are installing systems discussion of developing data access and for electronic health records at great expense. publication policies for registries, see Chapter 2. Many also are contemplating an assertion of The concept of ownership does not fit comfortably ownership in their health records, which may in the context of health information, because it include ownership of copyright. The claim to largely fails to acknowledge individual patient ownership by health care providers may be an privacy interests in health information. An overture to commercialization of their health care 130 inescapable personal nexus exists between information in aggregate form. Public individuals and information about their health. A knowledge of and response to such assertions of recent failure that illustrates this relationship, with ownership are uncertain at this time. A licensing regard to patient interests in residual tissue from program for the use of health information may clinical procedures, resulted in widely publicized enable health care providers to recoup some of litigation to determine who owned the residual their investment costs of electronic health records tissue and how it could be used for future including the expenses associated with the research.132 The legal concept of custody may be a technicians engaged to maintain them. In the near useful alternative to that of ownership. Custodians future, research use of health information for a have legal rights and responsibilities like those that registry may require licensing, in addition to the a guardian has for a ward or parents have for their terms and conditions in data use agreements and, if

171 Section II. Legal and Ethical Considerations for Registries

children. Custody also has a protective function, information registry existing in any medium, consistent with public expectations of including electronic digital media. The “facts” confidentiality practices that preserve the privacy compiled in a health information registry, however, and dignity of individual patients. Custody and its do not correlate closely to other compilations associated legal rights and responsibilities are protected by copyright, such as telephone books or transferable from one custodian to another. The even genetic databases.137 Instead, registry data concept of custody can support health care constitute legally protected, confidential provider investments in information systems and information about individual patients to which the licensed use of health information for multiple, independent and varied legal protections apply. socially beneficial purposes without denying Copyright protections may marginally enhance, patient interests in their health information. but do not diminish, other legal restrictions on The sharing of registry data subsequent to their access to and use of health information and collection presents special ethical challenges and registry data. For more information on copyright legal issues.133 The criteria used to determine the law, see Appendix B. conditions for shared use include applicable Federal or State law as well as the regulatory 5. Conclusions requirements in place when the health information was originally obtained. These legal and regulatory Ethical considerations arise in many of the requirements, as well as processing and licensing essential aspects of planning and operating a fees, claims of property rights, and concerns about registry. These considerations can affect the legal liability, are likely to result in formal written scientific, logistical, and regulatory components of agreements for each use of registry data. registry development, as well as claims of property Moreover, to educate patients and establish the rights in health information. The guiding ethical scientific independence of the registry, registry principles for these considerations are beneficence, developers should make known the criteria under justice, and respect for persons and avoidance of which data is used. harm. Currently, there are no widely accepted social or At the most fundamental level, investigations that legal standards that govern property rights in involve human subjects and that are not capable of health information, with the possible exception of achieving their scientific purpose are unethical. copyright, which is discussed below. At the time of The risk-benefit ratio of such studies is this writing, health information sources and other unacceptable in an analysis based on the principle users privately reach agreement to manage access of beneficence, which obligates investigators to and control. The Privacy Rule regulates the use avoid harming subjects, as well as maximize the and disclosure of health information by covered benefits and minimize the harms of research entities (including health plans and most health projects. Ethical scientific design must be robust, care providers), plus business associates working must be based on an important question, and must on behalf of covered entities, without regard to ensure sufficient statistical power, precise ownership interests over the data; the Privacy Rule eligibility criteria, appropriately selected data does not affect existing laws, if any, regarding elements, and adequately documented operating property rights in health information.134 procedures and methodologies. In addition, an ethical obligation to minimize 4.3.2 Copyright Protection for Health harms requires planning for and establishing Information Registries adequate protections to ensure the confidentiality In terms of copyright theory, a health information of the health information disclosed to a registry. registry is likely to satisfy the statutory definition Such planning should include developing policies of a compilation135 and reflect independent and procedures for the appropriate use and creativity by its developer.136 Thus, copyright law disclosure of registry data, and implementing may provide certain protections for a health physical, technical, and administrative safeguards

172 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

to limit access to and use of registry data balance of individual patient interests in health accordingly. Reducing the potential harms information against the potential social benefits associated with the use of health information in a from various uses of that information, including in registry is particularly important, because generally research. Consultation with legal counsel is no directly offsetting benefit from participation in a strongly recommended to determine the possible registry accrues to individuals whose health effect of these laws on a particular registry project. information is used in the registry. According to an Additional ethical considerations also affect the analysis applying the principle of justice, research operational aspects of registries, including activities that produce a significant imbalance of governance, transparency, and data ownership. potential risks and benefits to participating Registry governance, discussed in Chapter 2, individuals are unethical. should reflect both appropriate expertise and Protection of the confidentiality of the health representation of stakeholders, including patients. information used to populate a registry reflects the An independent advisory committee can provide ethical principle of respect for persons and useful guidance to registry developers and avoidance of harm. Health information intimately managers, especially on controversial issues. engages the privacy and dignity of patients. Transparency involves making information about Registry developers should acknowledge public registry governance and operations publicly expectations of protection for patient privacy and available. Registry transparency improves the dignity with clear and consistent communications credibility of the scientific endeavors of a registry, to patients about protections in place to prevent the use of health information for scientific inappropriate access to and use of registry data. purposes, and the results based on analyses of The regulatory requirements of the Privacy Rule registry data. In short, registry transparency and Common Rule address past ethical concerns promotes public trust. about research involving human subjects, as well as Claims of “ownership” of health information and general social anxiety about potential loss of registries are plausible, but have not yet been privacy associated with rapid advances in health legally tested. In addition, how the public would information systems technology and respond to such claims is uncertain, particularly as communications and biomedical developments in many such claims do not seem to acknowledge human genetics. Compliance with these regulatory patient interests in health information. Nonetheless, requirements not only is a cost of doing business in theory, copyright protections for compilations for a registry project, but also demonstrates may be applied to the patient information held by recognition of the ethical considerations health care providers and insurance plans, as well accompanying use of health information for as to registries. In general, property rights related scientific purposes. Compliance efforts by registry to health information are likely to be negotiated developers also acknowledge the important public privately under the terms and conditions of formal relations and liability concerns of health care agreements between registry developers, funding providers and insurance plans, public health agencies, and health care providers or insurance agencies, health oversight agencies, and research plans. As a practical matter, “ownership” implies organizations. Regulatory compliance contributes operational control of registry data and publication to, and generally supports, the credibility of rights. scientific research activities and research In summary, careful attention to the ethical organizations, as well as that of particular projects. considerations associated with the design and These and other Federal and State privacy laws operation of a registry, and fulfillment of the may affect registry development, especially applicable legal requirements, are critical to the registries created for public health purposes. Such success of registry projects and to the realization of laws express an explicit, legislatively determined their social and scientific benefits.

173 Section II. Legal and Ethical Considerations for Registries

6. Summary of Privacy Rule Note that the information in the table is a simplified summary of material; there may be and Common Rule other laws and individual institutional policies that Requirements also apply. Each research project is unique. Therefore, this table is not intended to provide Table 7-1 summarizes Privacy Rule and Common answers to specific questions that arise in the Rule requirements. The table generally assumes context of a given project. This table is no that the Privacy Rule applies to the data source— substitute for consultation with institutional i.e., that the data source is a “covered entity” or officials and others about the regulatory their “business associate.” The exception is requirements that apply to a particular registry Category 8, registry developers that use data not project. subject to the Privacy Rule.

174 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy Waivers are not applicable. Waivers Rule, respect to the Privacy With of board or IRB approval privacy of authorization depends a waiver on satisfaction of specific regulatory criteria. If the Common Rule applies,** of of a waiver IRB approval consent documentation or process depends on satisfaction of specific regulatory criteria. with are not applicable Waivers Rule. respect to the Privacy If the Common Rule applies,** of of a waiver IRB approval consent documentation or process depends on satisfaction of specific regulatory criteria. Waiver of Authorization, Waiver or Documentation of Consent, Consent Process The Privacy Rule permits use The Privacy health or disclosure to a public health authority for public activities. The Common Rule is not applicable. Rule permits The Privacy use or disclosure with patient authorization or IRB privacy of authorization. board waiver If the Common Rule applies,** and documented IRB review unless an IRB consent are required, of documentation grants a waiver for the consent process. or waiver Rule permits use or The Privacy disclosure to a person responsible product. for an FDA-regulated and Common regulations, FDA require Rule, if applicable,** a documented IRB review, consent process, and protection of confidentiality of research data. Health Information Includes Health Information Identifiers Direct The Privacy Rule permits use The Privacy health or disclosure to a public health authority for public activities. The Common Rule is not applicable. Rule permits the use The Privacy or disclosure of a limited data the data source and set, provided enter into a registry developer DUA. apply.** The Common Rule may Rule permits The Privacy use or disclosure to a person for an FDA- responsible product. regulated apply.** The Common Rule may Health Information Excludes Health Information Identifiers Direct No requirements. No requirements. No requirements. Health Information Health Information Is De-identified* . FDA- involving involving research research not . . public health practice health practice public Registry Developer or Registry Developer of Registry Purpose 1A. Federal or State public or State public 1A. Federal Registry health agency: for legal within agency’s authority research or State public Federal 1B. Registry health agency: is an agency project 2. Registry producing in support of evidence labeling for an product regulated Table 7-1. Summary of Privacy Rule and Common requirements Summary of Privacy 7-1. Table

175 Section II. Legal and Ethical Considerations for Registries Waiver of authorization is not Waiver under the Privacy applicable Rule. applies the If institutional policy of a Common Rule, IRB approval of consent documentation waiver or process depends on satisfaction of specific regulatory criteria. of authorization is not Waiver under the Privacy applicable Rule. If the Common Rule applies,** of of a waiver IRB approval consent documentation or process depends on satisfaction of specific regulatory criteria. are not applicable. Waivers Waiver of Authorization, Waiver or Documentation of Consent, Consent Process The Privacy Rule permits use or The Privacy disclosure for health oversight law. authorized by activities apply may Institutional policy the Common Rule or require IRB review. Rule permits use or The Privacy other law. disclosure required by apply, The Common Rule may qualify for the study may however Institutional policy exemption. the Common Rule or apply may or not whether require IRB review a research purpose is involved. Rule permits use or The Privacy disclosure for the “health care operations” of the data source in certain circumstances, of and, entity. another covered The Common Rule is not applicable. Health Information Includes Health Information Identifiers Direct

The Privacy Rule permits use or The Privacy disclosure for health oversight law. authorized by activities The Common Rule is not applicable. Rule permits use or The Privacy other law. disclosure required by If the Common Rule applies,** it permits an IRB grant of if the data is existing exemption unless a re- available, or publicly identification code is used. apply, The Common Rule may qualify the study may however for exemption. Rule permits the use The Privacy or disclosure of a limited data set for health care operations, the data source and provided enter into a registry developer data use agreement. The Common Rule is not applicable. Health Information Excludes Health Information Identifiers Direct No requirements. No requirements. No requirements. Health Information Health Information Is De-identified* . a . ; Common Rule may ; Common Rule may involving research involving Registry required by required Registry not Quality I/A registry Registry Developer or Registry Developer of Registry Purpose 3. Health oversight agency agency 3. Health oversight registry to perform activity health oversight not 4. law if registry involves apply research. 5. research involving Table 7-1. Summary of Privacy Rule and Common Rule requirements (continued) Rule and Common requirements Summary of Privacy 7-1. Table

176 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy IRB or privacy board approval board approval IRB or privacy depends on satisfaction of specific regulatory criteria. of a board approval Privacy of authorization depends waiver on satisfaction of specific regulatory criteria. Waiver of Authorization, Waiver or Documentation of Consent, Consent Process The Privacy Rule permits use The Privacy or disclosure for research with patient authorization or individual of board waiver an IRB or privacy authorization. The Common Rule requires IRB and documented consent review unless the IRB grants a waiver of documentation consent or a for the consent process. waiver Rule permits use The Privacy or disclosure for research with patient authorization or individual of authorization. waiver Health Information Includes Health Information Identifiers Direct

The Privacy Rule permits the The Privacy use or disclosure of a limited data set for research, provided the data source and registry enter into a DUA. developer The Common Rule permits an from IRB grant of exemption or if the data is existing review unless a available, publicly re-identification code is used. Rule permits the The Privacy disclosure of a limited data set, the data source and provided enter into a registry developer DUA. Health Information Excludes Health Information Identifiers Direct No requirements. No requirements. Health Information Health Information Is De-identified*

Research registry Research Registry Developer or Registry Developer of Registry Purpose 6. Research registry residing in organization Common Rule to which applies.** 7. organization by developed that is not a health care or insurance provider plan and is not subject to the Common Rule, using health information obtained from a health or insurance care provider plan. Table 7-1. Summary of Privacy Rule and Common Rule requirements (continued) Rule and Common requirements Summary of Privacy 7-1. Table

177 Section II. Legal and Ethical Considerations for Registries Waivers are not applicable. Waivers Waiver of Authorization, Waiver or Documentation of Consent, Consent Process

No requirements. Health Information Includes Health Information Identifiers Direct No requirements. Health Information Excludes Health Information Identifiers Direct No requirements. Health Information Health Information Is De-identified*

Registry Developer or Registry Developer of Registry Purpose 8. Research registry organization by developed that is not a health care or insurance provider plan and is not subject to the Common Rule, using health information collected from entities not subject to the Privacy Rule. Table 7-1. Summary of Privacy Rule and Common Rule requirements (continued) Rule and Common requirements Summary of Privacy 7-1. Table board. Administration; IRB = institutional review and Drug Food = U.S. = data use agreement; FDA DUA to a apply is not a substitute for consultation with appropriate institutional officials about the regulatory requirements that may Note: Reference to this table particular registry project. Rule standard for de-identification. *Information lacks the data elements specified in Privacy the registry will reside within which with the registry project, (2) the organization funding is involved applies if: (1) Federal **The Common Rule likely Note that its employees. conducted in its facilities or by the Common Rule to all research activities to apply Assurance (FWA) has agreed in its Federalwide the Common Rule. also apply may institutional policy

178 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

Case Example for Chapter 7

Case Example 13. Obtaining a waiver of recently, surgical aortic valve replacement has informed consent been the only effective treatment for adults with Description The STS/ACC TVT RegistryTM severe symptoms. The TAVR procedure is a new tracks patient safety and real- option for patients who are considered to be world outcomes for patients inoperable for conventional aortic valve undergoing a transcatheter aortic replacement surgery, but evidence is lacking on valve replacement (TAVR) long-term patient outcomes. procedure for treatment of aortic In 2012, CMS issued a Medicare National stenosis. The registry collects Coverage Decision for TAVR. Under the decision, data on patient demographics, CMS permits Medicare coverage for TAVR only procedure details, and facility when (1) the procedure is performed with a and physician information to device approved by the U.S. Food and Drug support analyses of patient Administration (FDA) consistent with labeled outcomes and clinical practice indications and any other FDA requirement; patterns. The Centers for (2) the procedure is performed in facilities Medicare & Medicaid Services meeting certain requirements; and (3) when all (CMS) approved the registry as patients undergoing the TAVR procedure are meeting the requirements included in a national TAVR registry or outlined in the Medicare participate in an approved clinical study. The National Coverage Decision on national TAVR registry must consecutively enroll TAVR. The participant TAVR patients, accept all manufactured devices, agreement for the registry follow patients for at least one year, and comply permits the registry sponsors to with all relevant regulations related to the conduct cardiovascular research protection of human research subjects. The using a limited data set. The National Coverage Decision specifically requires registry sponsors have signed a that the registry collect data on the following Federalwide Assurance that outcomes: stroke, all-cause mortality, transient requires that all research be ischemic attacks, major vascular events, acute conducted consistent with the kidney injury, repeat aortic valve procedures, and Common Rule. quality of life. Sponsor The Society of Thoracic The development of a national registry to meet Surgeons (STS) and the these requirements was challenging, particularly American College of Cardiology due to the need to collect at least one year of (ACC) followup and quality of life data. The registry was Year Started 2012 expected to enroll hundreds of sites and thousands of patients, making it time consuming, Year Ended Ongoing administratively cumbersome, and expensive to No. of Sites 247 hospitals obtain local institutional review board (IRB) approval for each site and informed consent for No. of Patients 9,051 patients each patient. Challenge Proposed Solution Aortic stenosis is the most common valvular The registry developers determined that the abnormality in the United States, and the national TAVR registry was most likely to be prevalence of this condition is expected to successful if it collected data that was already increase as the U.S. population ages. Until routinely documented as part of the standard of

179 Section II. Legal and Ethical Considerations for Registries

Case Example 13. Obtaining a waiver of Results informed consent (continued) The registry began collecting data in 2012 and Proposed Solution (continued) has been approved by CMS as meeting the care and was able to obtain a waiver of informed requirements of the Medicare National Coverage consent from a central IRB. To obtain a waiver of Decision. The ACC and STS, the institutions informed consent, the registry must meet all of operating the registry, are considered the only the following four criteria, as documented in 45 entities engaged in research, while the CFR 46.116(d): participating sites are considered to be providing data only. The registry was approved only by the 1. The research involves no more than minimal single IRB designated under the ACC/STS’s risk to the subjects; Federalwide Assurance. Based on the registry 2. The waiver or alteration will not adversely protocol, the IRB determined that the ACC/STS affect the rights and welfare of the subjects; are engaged in research on human subjects and 3. The research could not practicably be carried granted a waiver of informed consent. The waiver out without the waiver or alteration; and, of informed consent was awarded primarily because the participating sites are collecting and 4. Whenever appropriate, the subjects will be submitting information that is already provided with additional pertinent information documented in the medical record as part of the after participation. standard of care. As the registry operators, the When designing the data collection instruments ACC and STS submit data, including patient for the registry, the developers worked closely identifiers, to CMS as required by the National with surgeons and interventional cardiologists to Coverage Decision. However, the ACC and STS understand which data are already collected. The only conduct research on a limited data set, and developers were able to limit the registry data any research studies not covered by the protocol collection effort to data already collected are submitted to the IRB for review. routinely, thereby allowing registry data to be Because the ACC and STS have IRB approval abstracted from the medical record with no data and a waiver of informed consent, and because collected solely for the purposes of the registry. the data are already collected as part of the In particular, the registry developers carefully standard of care, the individual sites participating considered how to collect followup data and in the registry do not necessarily need to go quality of life data without requiring the through an IRB prior to enrolling in the registry. collection of information solely for the purposes Some individual sites elect to submit the registry of the registry. to their local IRB, in many cases because they Based on discussions with surgeons and intend to use the data they collect for the registry interventional cardiologists, the developers in additional research studies. The local IRBs determined that patients are seen for followup generally have reached the same conclusion as care routinely at 30 days and 1 year following the the central IRB. However, a local IRB may reach procedure. Published guidelines have established a different conclusion, perhaps due to differences the use of the Kansas City Cardiomyopathy in the data collection process at an individual site. Questionnaire (KCCQ) to assess quality of life as For example, the data collection process at an a standard of care for TAVR patients at these individual site may provide an opportunity to follow-up visits. The registry was designed to use consent the patient, in which case the IRB may the data collected at these followup visits, not grant a waiver of informed consent. In these including the KCCQ, to meet the requirements cases, the individual site will follow the advice of for collecting long-term outcomes and quality of the local IRB. life information.

180 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

Case Example 13. Obtaining a waiver of registry identify a strategy that protects patients’ informed consent (continued) privacy without overburdening the participating Key Point sites. Protecting the subjects whose data will be used is For More Information of the utmost importance when developing a STS/ACC TVT Registry. https://www.ncdr.com/ registry. When developing a registry, sponsors TVT/Home/Default.aspx. should consider the planned data collection effort in the context of requirements for IRB review and informed consent. This approach may help the

References for Chapter 7 9. Council for International Organizations of Medical Sciences. International Guidelines for 1. See, for example, Section III, Article 8, of Ethical Review of Epidemiological Studies. 1991. Directive 95/46/EC of the European Parliament Note 6, at paragraphs 11 and 12. http://www. and of the Council of 24 October 1995 on the cioms.ch/publications/guidelines/1991_texts_of_ protection of individuals with regard to the guidelines.htm. Accessed March 4, 2014. processing of personal data and on the free 10. U.S. Department of Health and Human Services. movement of such data. Office of the Inspector General. Recruiting 2. Based to a certain extent on 45 CFR 160.103: Human Subjects: Sample Guidelines for Practice. definition of health information; and 45 CFR OEI-01-97-00196. June 2000. p. 5. http://oig.hhs. 46.102(f): definition of human subject. gov/oei/reports/oei-01-97-00196.pdf. 3. 45 CFR Part 46. 11. U.S. Department of Health and Human Services. Office of the Inspector General. Recruiting 4. Part C of Title XI of the Social Security Act, 42 Human Subjects: Sample Guidelines for Practice. USC §§ 1320d to 1320d-9 , and section 264 of the Appendix A. OEI-01-97-00196. June 2000. Health Insurance Portability and Accountability http://oig.hhs.gov/oei/reports/oei-01-97-00196. Act of 1996, 42 USC §1320d-2 note ; 45 CFR pdf. Parts 160 and 164. 12. Patient Protection and Affordable Care Act. 5. The National Commission for the Protection of Section 6002, Transparency Reports and Human Subjects of Biomedical and Behavioral Reporting of Physician Ownership or Investment Research. April 18, 1979. http://www.hhs.gov/ Interests. Public Law 111-148. March 23, 2010. ohrp/humansubjects/guidance/belmont.html. http://www.gpo.gov/fdsys/pkg/ Accessed March 4, 2014. PLAW-111publ148/pdf/PLAW-111publ148.pdf. 6. Public Law 93-348 (1974), Title II. 13. Massachusetts regulation 105 CMR 970.000 7. Council for International Organizations of implementing Massachusetts General Law, Medical Sciences. International Guidelines for Chapter 111N, Pharmaceutical and Medical Ethical Review of Epidemiological Studies. 1991. Device Manufacturer Conduct, as enacted under Noted to be under revision. http://www.cioms.ch/ Chapter 305 of the Acts of 2008, An Act To publications/guidelines/1991_texts_of_guidelines. Promote Cost Containment, Transparency and htm. Accessed March 4, 2014. See especially Efficiency in the Delivery of Quality Health Care. sections entitled General Ethical Principles and http://www.lawlib.state.ma.us/source/mass/cmr/ Informed Consent.. cmrtext/105CMR970.pdf. 8. Grant RW, Sugarman J. Ethics in human subjects 14. Council for International Organizations of research: do incentives matter? J Med Philos. Medical Sciences. International Guidelines for 2004 Dec;29(6):717-38. PMID: 15590518. Ethical Review of Epidemiological Studies. 1991. Note 6, at paragraphs 18–21. http://www.cioms. ch/publications/guidelines/1991_texts_of_ guidelines.htm. Accessed March 4, 2014.

181 Section II. Legal and Ethical Considerations for Registries

15. Council for International Organizations of 22. 45 CFR Part 46, Subpart A. Medical Sciences. International Guidelines for 23. See, for example, on the essential elements of a Ethical Review of Epidemiological Studies. 1991. protocol: Epstein M, International Society for Note 6, at paragraph 26. http://www.cioms.ch/ Pharmacoepidemiology. Guidelines for Good publications/guidelines/1991_texts_of_guidelines. Pharmacoepidemiology Practices (GPP). htm. Accessed March 4, 2014. Pharmacoepidemiol Drug Safety. 2005 16. See generally Council for International August;14(8):589–95. PMID: 15918159. Organizations of Medical Sciences. International 24. See 45 CFR 160.103. Guidelines for Ethical Review of Epidemiological Studies. 1991. Note 6, at paragraph 43. http:// 25. 45 CFR 160.102, Applicability, and 160.103, www.cioms.ch/publications/guidelines/1991_ definitions of covered entity, health care provider, texts_of_guidelines.htm. Accessed March 4, 2014. health plan, health care clearinghouse, and See also the Patient Protection and Affordable transaction. Care Act of 2010, Sec. 1101. http://www.gpo.gov/ 26. 45 CFR 160.103, definition of business associate. fdsys/pkg/PLAW-111publ148/pdf/ PLAW-111publ148.pdf. 27. 45 CFR 160.203. 17. Council for International Organizations of 28. Maryland Health General Statute § 4–303(b)(4). Medical Sciences. International Guidelines for 29. 45 CFR 160.103 defines both “disclosure” and Ethical Review of Epidemiological Studies. 1991. “use” for the purposes of the HIPAA Rules. Note 6, at paragraph 40. http://www.cioms.ch/ publications/guidelines/1991_texts_of_guidelines. 30. Modifications to the HIPAA Privacy, Security, htm. Accessed March 4, 2014. Enforcement, and Breach Notification Rules Under the Health Information Technology for 18. See, for example, U.S. Department of Health and Economic and Clinical Health Act and the Genetic Human Services regulations at 45 CFR Part 46; Information Nondiscrimination Act; Other 21 CFR Parts 50 and 56 for research conducted in Modifications to the HIPAA Rules; Final Rule, 78 support of products regulated by the U.S. Food Fed. Reg. 5566 (January 25, 2013) (codified at 45 and Drug Administration (FDA); and Williams CFR pts 160 and 164). ED. Federal Protection for Human Research Subjects: An Analysis of the Common Rule and 31. HITECH Act §13404(a); 45 CFR 164.104(b); Its Interactions with FDA Regulations and the 78 Fed. Reg. at 5591. HIPAA Privacy Rule. Congressional Research 32. 45 CFR 164.501. Service, Library of Congress. Updated June 2, 2005. http://www.fas.org/sgp/crs/misc/RL32909. 33. 67 Fed Reg 53231, August 14, 2002. pdf. 34. 45 CFR 46.102(d). 19. Regulations identical to 45 CFR 46 Subpart A 35. National Institutes of Health. Health Services apply to research funded or conducted by a total of Research and the HIPAA Privacy Rule. NIH 17 Federal agencies, some of which may also Publication Number 05-5308. May 2005. require additional legal protections for human http://privacyruleandresearch.nih.gov/ subjects. healthservicesprivacy.asp. See also National 20. U.S. Department of Health and Human Services. Institutes of Health. Research Repositories, Office for Human Research Protections. Terms of Databases, and the HIPAA Privacy Rule. NIH the Model Federalwide Assurance (FWA) for the Publication Number 04-5489, January 2004. Protection of Human Subjects. http://www.hhs. http://privacyruleandresearch.nih.gov/research_ gov/ohrp/assurances/assurances/filasurt.html. repositories.asp. Accessed March 4, 2014. 36. 45 CFR 164.512(b). 21. U.S. Department of Health and Human Services. 37. Centers for Disease Control and Prevention. Office for Human Research Protections. Guidance HIPAA Privacy Rule and Public Health: Guidance on Research Involving Coded Private Information from CDC and the U.S. Department of Health and or Biological Specimens. Oct 16, 2008. Human Services. MMWR. 2003:52. http://www.hhs.gov/ohrp/policy/cdebiol.html. PMID: 12741579.

182 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

38. 45 CFR 164.512(a). 49. A copy of the HHS version of the “Common Rule,” 45 CFR Part 46, subpart A, and additional 39. 45 CFR 164.508(a). subparts B, C, and D regarding vulnerable 40. 45 CFR 164.514(e). populations may be obtained on the Web site of 41. 45 CFR 164.514(a)–(c). the Office for Human Research Protection (OHRP) in the U.S. Department of Health and 42. 45 CFR 164.528. Human Services. http://www.hhs.gov/ohrp/ 43. 45 CFR 164.512(i)(1)(i). humansubjects/guidance/45cfr46.html. Accessed March 4, 2014. 44. 21 CFR 65.111(a)(7). 50. A copy of the “Privacy Rule,” 45 CFR Parts 160 45. Centers for Disease Control and Prevention. and 164, may be obtained on the Web site of the Guidelines for Defining Public Health Research Office for Civil Rights (OCR) in the U.S. and Public Health Non-Research. Revised October Department of Health and Human Services. 4, 1999. http://www.cdc.gov/od/science/integrity/ http://www.hhs.gov/ocr/hipaa/finalreg.html. docs/defining-public-health-research-non- Accessed March 4, 2014. research-1999.pdf. Accessed March 4, 2014. Gostin LO. Public Health Law: Power, Duty, 51. The Common Rule as adopted by the U.S. Restraint. Berkeley and Los Angeles, CA: Department of Health and Human Services University of California Press; New York: The contains special protections for certain defined Milbank Memorial Fund; 2000. pp. 126–127. See “vulnerable” populations, i.e., women, human also Council for International Organizations of fetuses, neonates, prisoners, and children. See 45 Medical Sciences. International Guidelines for CFR Part 46, Subparts B, C, D. Ethical Review of Epidemiological Studies. 1991. 52. American College of Epidemiology: Policy Note 6, Introduction, noting that epidemiological Statement on Sharing Data from Epidemiologic practice and research may overlap. http://www. Studies. May 2002. http://www.acepidemiology. cioms.ch/publications/guidelines/1991_texts_of_ org/sites/default/files/DataSharing.pdf. guidelines.htm. Accessed March 4, 2014. Accessed March 4, 2014. 46. Bellin E, Dubler NN. The quality improvement- 53. National Cancer Institute. Surveillance research divide and the need for external Epidemiology and End Results. http://seer.cancer. oversight. Am J Public Health. 2001;91(9): gov. Accessed August 27, 2012. 1512–1517. PMID: 11527790. Lindenauer PK, Benjamin EM, et al. The role of the institutional 54. Gostin LO, Lillienfeld DE, Stolley PD review board in quality improvement: a survey of Foundations of Epidemiology (revised). Oxford quality officers, institutional review board chairs, University Press; 1994. p. 104. See also Gostin and journal editors. Am J Med. 2002;113(7): LO. Public Health Law: Power, Duty, Restraint. 575–9. PMID: 12459404. Lo B, Groman M. Berkeley and Los Angeles, CA: University of Oversight of quality improvement: focusing on California Press; New York: The Milbank benefits and risks. Arch Intern Med. Memorial Fund; 2000. Note 43, at 114, Table 5. 2003;163(12):1481–6. PMID: 12824099. 55. 45 CFR 164.512(b)(1)(iii). 47. U.S. Department of Health and Human Services. 56. 45 CFR 164.512(d). Office for Human Research Protections. 57. 45 CFR 164.512(d)(1). http://www.hhs.gov/ohrp. Accessed March 4, 2014. 58. 45 CFR 46.102(d) and 45 CFR 164.501, respectively. 48. National Institutes of Health. Health Services Research and the HIPAA Privacy Rule. NIH 59. National Institutes of Health. Health Services Publication Number 05-5308, pp. 2–3. See also 45 Research and the HIPAA Privacy Rule. NIH CFR 164.501 for the definition of health care Publication No. 05-5308. May 2005. pp. 2–3. operations. http://privacyruleandresearch.nih.gov/ http://privacyruleandresearch.nih.gov/ healthservicesprivacy.asp. healthservicesprivacy.asp. Accessed March 13, 2014.

183 Section II. Legal and Ethical Considerations for Registries

60. Centers for Disease Control and Prevention. 71. 67 FR 53182, 53233, August 14, 2002. Guidelines for Defining Public Health Research 72. 45 CFR 164.514(e)(4). and Public Health Non-Research. October 4, 1999. pp. 126-7. http://www.cdc.gov/od/science/ 73. 45 CFR 164.514(e)(4)(ii)(A). integrity/docs/defining-public-health-research- 74. 45 CFR 164.514(e)(4)(ii)(C)(2) and (5), non-research-1999.pdf. Accessed March 4, 2014. respectively. Gostin LO. Public Health Law: Power, Duty, Restraint. Berkeley and Los Angeles, CA: 75. 67 Fed Reg 53182, 53236, August 14, 2002. University of California Press; New York: The 76. 45 CFR 164.504(e). Milbank Memorial Fund; 2000. Note 43. See also Council for International Organizations of 77. 45 CFR 46.101(b)(4). Medical Sciences. International Guidelines for 78. 45 CFR 164.508. Ethical Review of Epidemiological Studies. 1991. Note 6, Introduction, noting that epidemiological 79. 45 CFR 164.508(c). practice and research may overlap. http://www. 80. 45 CFR 164.502(a)(1). cioms.ch/publications/guidelines/1991_texts_of_ 81. 45 CFR 46.116. guidelines.htm. Accessed March 4, 2014. And Bellin E, Dubler NN. The quality improvement- 82. 45 CFR 164.508(b)(3). research divide and the need for external 83. 45 CFR 164.512(i)(1)(i)(B). oversight. Am J Public Health. 2001;91(9): 1512–1517. Note 44. PMID: 11527790. 84. 67 Fed Reg 53182, 53226, August 14, 2002. 61. Bellin E, Dubler NN. The quality improvement- 85. National Institutes of Health. Institutional Review research divide and the need for external Boards and the HIPAA Privacy Rule. NIH oversight. Am J Public Health. 2001;91(9): Publication Number 03-5428. August 2003. pp. 1512–1517. Note 44. PMID: 11527790. 15–16. http://privacyruleandresearch.nih.gov/ Lindenauer PK, Benjamin EM, et al. The role of irbandprivacyrule.asp. Accessed March 4, 2014. the institutional review board in quality 86. 45 CFR 164.508(b)(3); 78 Fed. Reg. at 5612. improvement: a survey of quality officers, institutional review board chairs, and journal 87. 45 CFR 164.508(c) and 164.508(c)(1)(iv). editors. Am J Med. 2002;113(7):575–9. 88. 45 CFR 164.508(c)(2)(iii). PMID: 12459404. Lo B, Groman M Oversight of quality improvement: focusing on benefits and 89. Family Educational Rights and Privacy Act risks. Arch Intern Med. 2003;163(12):1481–6. (FERPA), 20 USC 1232g, 34 CFR Part 99. PMID: 12824099. 90. National Institutes of Health. Office of Extramural 62. See 45 CFR 160.103 for the definition of Research. Grants & Funding. "Frequently Asked individually identifiable health information and 45 Questions, Certificates of Confidentiality. CFR 164.514(a)–(c) and (e) on the de- Definitions – What does identifying characteristic identification of health information and limited mean?" http://grants1.nih.gov/grants/policy/coc/ data sets, respectively. faqs.htm#369. Accessed March 4, 2014. 63. 45 CFR 46.102(f). 91. Public Health Services Act Section 301(d), 42 USC 241(d) as amended. See also 42 CFR Part 2a 64. 45 CFR 46.111(a)(7). about research activities on mental health, 65. 45 CFR 46.116(a)(5). including the use and effect of alcohol and other psychoactive drugs. 66. See 45 CFR 164.514(a)–(c) and (e) on the deidentification of health information and limited 92. U.S. Department of Health and Human Services. data sets, respectively. Office for Human Research Protection. Guidance on Certificates of Confidentiality. Background. 67. 45 CFR 164.514(e)(2). February 25, 2003. http://www.hhs.gov/ohrp/ 68. 45 CFR 164.514(b). policy/certconf.pdf. Accessed March 4, 2014. 69. 45 CFR 164.514(b)(1). 70. 45 CFR 164.514(c).

184 Chapter 7. Principles of Registry Ethics, Data Ownership, and Privacy

93. National Institutes of Health. "NIH Announces 115. See 73 Fed. Reg. 70,739. Statement on Certificates of Confidentiality." 116. 42 C.F.R. § 3.20. Notice NOTOD-02-037. March 15, 2002. http://grants1.nih.gov/grants/guide/notice-files/ 117. 42 CFR 3.402. NOT-OD-02-037.html. Accessed March 4, 2014. 118. 42 CFR 3.206. 94. Information about obtaining a certificate of 119. 42 CFR 3.206(b)(6). confidentiality is available at the “Certificates of Confidentiality Kiosk” on the National Institutes 120. 42 CFR 3.102. of Health Web site. http://grants.nih.gov/grants/ 121. 42 USC 299b–22(i). 45 CFR 164.501; 78 Fed. policy/coc/index.htm. Accessed March 4, 2014. Reg. at 5592. 95. National Institutes of Health. Office of Extramural 122. Nass SJ, Levit LA, Gostin LO, eds. Committee on Research. Certificates of Confidentiality: Health Research and the Privacy of Health Background Information. Febrary 14, 2006. Information. Beyond the HIPAA Privacy Rule: http://grants.nih.gov/grants/policy/coc/ Enhancing Privacy, Improving Health Through background.htm. Accessed March 4, 2014. Research. National Academies Press; The HIPAA 96. 42 USCS 290dd-2 and 290ee-3; 42 CFR Part 2. Privacy Rule; Institute of Medicine. http://www.nap.edu/catalog/12458.html. 97. 42 CFR 2.52(a). Accessed March 4, 2014. 98. In re Philip Morris, 706 So. 2d 665 (La.App. 4 123. Nass SJ, Levit LA, Gostin LO, eds. Committee on Cir. Jan. 28, 1998) (holding that raw data from Health Research and the Privacy of Health research on tobacco use is protected under Information. Beyond the HIPAA Privacy Rule: Louisiana statutes that govern the confidentiality Enhancing Privacy, Improving Health Through of public health data). Research. National Academies Press; The HIPAA 99. See, for example, Wis. Stat. 146.38. Privacy Rule; Institute of Medicine. http://www.nap.edu/catalog/12458.html. 100. See 45 CFR 164.512(i) and 46.116(d), Accessed March 4, 2014. pp. 26. respectively. 124. 45 CFR § 160.103, definition of health 101. 45 CFR 164.512(i)(2)(ii). information; 78 Fed. Reg. at 5658-61. 102. 45 CFR 164.512(i)(2). 125. 74 FR 42740 (August 24, 2009); 78 Fed. Reg. at 103. 45 CFR 164.512(i)(2)(iii) and (iv). 5638-58 (January 25, 2013); 45 CFR 164.400- 164.414. 104. 45 CFR 46.116(d). 126. 45 CFR 46.116(d)(4). 105. 21 CFR 50.20 and 50.23. 127. Center for International Blood & Marrow 106. 45 CFR 46.117(c)(1). Transplant Research. http://www.cibmtr.org/ 107. 45 CFR 46.117(c)(2). pages/index.aspx. Accessed March 4, 2014. 108. 45 CFR 46.117(c). 128. 67 F.R. 53213, August 14, 2002. 109. 45 CFR 164.528(b)(1). 129. Joyce C, Patry W, Leaffer M, et al. Introduction: 110. 45 CFR 164.528(b). the landscape of copyright. In: Copyright Law. 3rd ed., New York and San Francisco: Matthew 111. 45 CFR 164.528(a)(1). Bender & Co., Inc; 1994 (reprinted 1997):chapter 112. 45 CFR 164.528(b)(3) and (4). 1, pp. 1–41. 113. 42 CFR 3. 114. Kohn LT, Corrigan JM, Donaldson MS, eds. To Err is Human: Building a Safer Health System. Washington, D.C.: National Academies Press; 1999.

185 Section II. Legal and Ethical Considerations for Registries

130. American Medical Association House of 133. American College of Epidemiology. Policy Delegates, Connecticut Delegation. Guiding Statement on Sharing Data from Epidemiologic Principles, Collection and Warehousing of Studies. May 2002. http://www.acepidemiology. Electronic Medical Record Information. org/sites/default/files/DataSharing.pdf. Accessed Resolution No. 802. September 16, 2005. March 4, 2014. National Institutes of Health. Final http://www.ama-assn.org/meetings/public/ NIH Statement on Sharing Research Data. Notice interim05/802i05.pdf. Accessed March 4, 2014. NOT-OD-03-032. February 26, 2003. http://grants. Bailey S. Your data for sale? Boston Globe. March nih.gov/grants/guide/notice-files/not-od-03-032. 24, 2006. http://www.boston.com/business/ html. Accessed March 4, 2014. healthcare/articles/2006/03/24/your_data_for_ 134. 78 FR 5566 at 5606 (January 25, 2013). sale/. Accessed March 4, 2014. 135. 17 USC § 101. 131. American Recovery and Reinvestment Act, section 13405(d)(2)(B); 45 CFR § 164.502(a)(5) 136. Feist Publications, Inc. v. Rural Telephone Service, (ii). Co., Inc., 499 U.S. 340, 345, 348 (1991). 132. Washington University v. Catalona, Case No. 137. Id., 340 et seq; Harris RK, Rosenfield SS. 4:03CV1065SNL (E.D. Mo., filed Mar. 31, 2006). Copyright Protection for Genetic Databases, 2005. Washington University v. William J. Catalona, J. 45:225–250. M.D., No. 06-2286 and No. 06-2301 (8th Cir. June 20, 2007).

186 Chapter 8. Informed Consent for Registries

1. Introduction information of patients using a particular treatment to facilitate notifications or recalls). In fact, This chapter identifies the best practices for registries are increasingly being used for research obtaining informed consent and permission for purposes even when initially developed for clinical registry participation. It builds on some of the purposes, and thus it is suggested that in all cases, general ethical and legal principles discussed in consideration should be given to the informed Chapter 7, focusing specifically on the application consent issues, as well as HIPAA privacy of the regulations governing human subjects requirements, discussed in this chapter. The research and the requirements of the Health HIPAA Privacy Rule governs the use and Insurance Portability and Accountability Act disclosure of most individually identifiable health (HIPAA) Privacy Rule. information (called protected health information or “PHI”) held by covered entities (health plans, The purpose of this chapter is to provide an ethical health care clearinghouses, and most health care framework for obtaining informed consent and providers). permission for registry participation and to distinguish registries from clinical research The Federal research regulations promulgated by protocols with respect to these issues. It is not the U.S. Department of Health and Human designed to provide specific legal guidance, nor Services (HHS) focus on research involving can it substitute for Institutional Review Board human subjects. The U.S. Food and Drug (IRB) review. Moreover, legal discussion is limited Administration (FDA) regulations apply to “all to U.S. law, and more specifically, to Federal as clinical investigations” regulated by the FDA— opposed to State statutes and regulations. Some defined as “any experiment that involves a test States have guidelines governing the conduct of article and one or more human subjects”.1 The research involving human subjects or statutes HHS regulations apply only to “human subjects addressing privacy, and an exploration of either research,” where “research” is defined as a area is beyond the scope of this chapter. Likewise, “systematic investigation, including research analysis of the relevant international standards and development, testing and evaluation, designed to laws is left to others. Case examples 14, 15, and 16 develop or contribute to generalizable knowledge” provide descriptions of practical issues that and “human subject” is defined as a living person registries have encountered in this area. about whom the investigator obtains either data through intervention or interaction, or identifiable 2. Registries, Research, and private information.2 Thus, investigations that involve non-living individuals, or that do not Other Activities collect data through intervention/interaction with the individual, or do not collect identifiable private The purpose of this volume is to provide guidance information are not governed by the HHS research for registries used to evaluate patient outcomes, regulations. Despite the apparent limitations of the such as efforts to describe the natural history of regulatory language, institutions may choose to disease, determine clinical and/or cost apply the frameworks more broadly (sometimes effectiveness, assess safety or harm, and measure under an “assurance,” i.e., an agreement with HHS or improve quality of care. As a result, the focus of that the institution will apply the regulations to all this chapter is on informed consent and research at the institution regardless of funding authorization issues that arise in registries used for source). Even when the activity in question meets research. Some registries now used for research the HHS definition of involving human subjects, a may have been developed initially for clinical series of exemptions may apply.3 purposes (e.g., a registry of names and contact

187 Section II. Legal and Ethical Considerations for Registries

2.1 Registry Research vs. Clinical Questions about adapting the regulatory Research requirements to research that does not fit the typical clinical model are not unusual. There are Some differences between registry research and two other areas that have raised questions about clinical research are worth noting. In particular, the how the Federal regulations apply and that are the use of a control group in a registry setting is particularly relevant to registry evaluations: public often substantively different from the concept of a health activities and quality improvement/ control group in a clinical research setting. assurance (QI/QA). Registry controls may be pulled from a general population—in some cases a population that may 2.2 Public Health Activities not have interacted with health professionals or a health institution. Unlike clinical controls, who The HIPAA Privacy Rule expressly permits the may be exposed to placebos (and thus need to disclosure of Protected Health Information (PHI) consent) or exposed to a standard treatment (and to a public health authority that is authorized by thus will already be involved in the treatment law to collect or receive such information for the system), registry controls may be identified from purpose of preventing or controlling disease, an unaffected population. This raises ethical injury, or disability, including for activities related questions about the initial contact with an to disease, injury, or vital event reporting. Thus, a individual who may have no link to the registry covered entity may disclose PHI, without topic area and who may view the contact as an individual authorization, for a registry maintained unwelcome intrusion or perhaps even an incorrect by a public health authority (or by an entity acting indication of problematic health status.4 under a grant of authority from or contract with Furthermore, since a clinical research trial may such public agency) for authorized public health involve double-blind procedures, “controls” may purposes; such as, for example, immunization agree to participate because of the potential for registries, State cancer registries, birth and death direct therapeutic benefits or even the indirect registries, and general disease reporting (although therapeutic benefits that come from better this last type of reporting is often anonymous). The attendant care. In other situations, controls may HIPAA Privacy Rule also allows the disclosure of participate because they hope to help others PHI to a person subject to the jurisdiction of FDA suffering from their ailments (altruistic reasoning) for FDA-regulated–product reporting. or perhaps because they seek monetary Public health activities may not be considered compensation. In contrast, controls in a registry “human subjects research” under HHS or FDA trial have no similar potential therapeutic (direct or research regulations. However, differentiating indirect) or monetary benefits. While altruism may between public health practice and public health play a role in this context, its effects may be less research activities can be challenging. According than ideal. There is a great concern about the to the Belmont Report, the document on which the potential for selection bias in the creation of a Federal research regulations are based, if any control group for registry trials—as there is also aspect of an activity constitutes “research,” then significant concern about the effect of bias in the entire activity should undergo regulatory clinical trials. Those who may agree to participate review. The Office for Human Research in a registry may be qualitatively different from Protections (OHRP) interpretation of the HHS those who do not agree, and this disparity can regulations implies that if any part of the activity threaten the external validity of research findings. falls under the regulations the entire activity is Concerns about selection bias will be heightened covered.5 By contrast, the Centers for Disease for diseases with a low prevalence in the general Control and Prevention (CDC) only consider an population, since there will be a greater possibility activity research if the primary intent is to that the bias will affect the data. Developing contribute to or generate generalizable consent requirements in such a way as to avoid knowledge.6 The CDC, however, does not provide selection bias will be extremely important in this official interpretations of the HHS research setting. regulations, as the authority for this rests with

188 Chapter 8. Informed Consent for Registries

OHRP. Local IRB policies in this area vary; some obtaining of generalizable knowledge is not the focus on whether the primary intent of an activity primary purpose of any studies resulting from such is to gain generalizable knowledge, and others activities.”10 Individual authorization for use or categorically exclude normal public health disclosure of PHI in this context is not required, department activities. but a covered entity can choose to obtain To address confusion regarding what is considered individual consent for such uses and disclosures. a public health activity versus a research activity, The Federal research regulations do not have an the Council of State and Territorial explicit exemption for QI/QA activities. However, Epidemiologists (CSTE) issued a report clarifying many of the efforts in this area will: (a) not meet that public health practice activities are those for the regulatory definitions of “research,” (b) not which: there is a specific or general legal involve “human subjects,” (c) fall under a authorization to conduct (e.g., State statutory delineated exemption, or (d) not be supported by cancer registries, or reports of newborn hearing HHS, involve an FDA regulated product, or screening to the State health department); the otherwise be covered by an assurance of specific intent of the authority conducting the compliance. Some local IRBs appear to consider activity is to promote the health of, or prevent all QI/QA activities outside the scope of the harm to, the individuals or communities involved regulations, as they would public health (as opposed to research where the intent is to activities.11 generate generalizable knowledge); and there are, The application of the human subjects research in fact, health benefits to the individuals involved 7 regulations does not rest on whether or not a or to the target community. Moreover, public procedure is considered “standard” or part of the health activities, unlike research, are not likely to “standard of care;” rather, it rests on the purpose involve experimental procedures, to have one (or of the activity. Intent to publish the results of a QI/ more) individuals, such as a principal investigator QA activity is not determinative of whether the (PI), responsible for the development and conduct human subjects regulations apply. Registries of the activity, or to entail individual developed within an institution to implement a randomization for access to interventions. practice to improve the quality of patient care or to Alternatively, a public health activity may fit the collect data regarding the implementation of such definition of human subjects research, but fall into a practice are not considered “research” under the one of the various HHS exemptions. For example, regulations. Nor are registries designed to collect there are exemptions for research involving provider performance data for clinical, practical, surveys, interviews, or observations of public or administrative uses. Registries that involve behavior, provided certain requirements are met.8 existing data that are not individually identifiable There is also an exemption for the collection or may entail “research,” but do not involve “human examination of existing data, if publicly available subjects” as defined by the HHS research and information is recorded “in such a manner that regulations; therefore the HHS research subjects cannot be identified, directly or through regulations do not apply. However, a QI/QA identifiers.”9 project that involves an untested clinical intervention (whether or not part of the standard of 2.3 Quality Improvement/Quality care) for purposes of gathering scientific evidence Assurance Activities of efficacy (i.e., a systematic investigation As with certain public health activities, the HIPAA designed to contribute to generalizable knowledge) Privacy Rule provides an explicit permission to would be governed by the regulations, although a use or disclose PHI for “health care operations,” specific exemption may apply (e.g., if it is part of which are defined as certain activities of a covered the evaluation of a public benefit program).12 Even entity, including “conducting quality assessment if the regulations apply, waivers or alterations to and improvement activities…, provided that the the consent process may be approved as noted below.

189 Section II. Legal and Ethical Considerations for Registries

3. Current Challenges for Despite apparent public unease with a system of open access to EHRs, the Institute of Medicine Registries (IOM) in 2009 released a statement that informed consent for research using EHRs should not be 3.1 Electronic Health Records required, with the justification that obtaining The development of large-scale data registries permission from patients is too burdensome for raises a variety of regulatory questions, and this is researchers and should be eliminated entirely.17 nowhere more evident than in the discussions This proposal generated widespread concern that about electronic health records (EHRs). These its implementation would undermine the trust that issues are explored in detail in Chapter 15. This forms the basis of the patient-physician chapter focuses only on the relevant consent relationship and also more broadly increased issues. There are currently few, if any, efforts to concerns about patients’ privacy and obtain individual consent for the creation of an confidentiality protections. Given the strong EHR (or, for that matter, the creation of any health arguments on both sides, establishing consensus record). Yet, these databases have enormous on the topic has been slow. research potential. For example, Kaiser In the effort to resolve the debate, additional work Permanente, a leader in the use of health in this area should focus on striking the information technology, created and maintains one appropriate balance between providing patients of largest private-sector EHR systems, collecting with control over information and facilitating health information from more than 8.7 million necessary research. Commentators have suggested Kaiser members nationwide. Moreover, there are a a variety of different approaches, including number of efforts to develop (sometimes via State recognition of public ownership of large electronic legislation) multi-payer claims registries to support databases18 or the creation of licensed data centers comparative effectiveness research (CER). Various that would control access to information without steps have been discussed to ensure the privacy individual consent.19 It is not clear from the and confidentiality of the individual health empirical evidence that patients want full consent information gathered into these registries (e.g., the protections in this context. One study, for example, use of coded identifiers). Application of traditional found that patients were more likely to be consent models for the secondary use of these comfortable with the research uses of their EHR databanks for research may prove inefficient and information when they were asked about the use may result in selection bias, impacting the by a specific entity (e.g., universities, hospitals, or usefulness of downstream analyses. disease foundations) rather than when asked in the As the development of EHRs, claims registries, abstract, and that they fully supported public health information exchanges, and linkages health uses of their data.20 Public education about between innumerable health databases moves the scope of research uses may alleviate some forward, keeping records private becomes more patient concerns about the use of EHR data difficult to manage. Personal health information without consent.21 Similarly, addressing may be accessed and shared in ways patients never underlying fears about unauthorized access to imagined, often for the purpose of secondary identifiable data or discriminatory uses of the analysis and often without patient consent. information can also be helpful in increasing Although studies consistently indicate that support for this type of research. Given the vast Americans are generally supportive of EHRs and potential for using EHRs to conduct large-scale even secondary uses of the data, they want to be observational studies, development of an informed about how and to what extent their alternative to specific individual consent may be information will be shared and disclosed to useful. On July 26, 2011, HHS and the Office of others.13-16 Science and Technology Policy (OSTP) published an advance notice of proposed rulemaking

190 Chapter 8. Informed Consent for Registries

(ANPRM) entitled, “Human Subjects Research of clinical specimens, the issues raised are also Protections: Enhancing Protections for Research applicable to the use of clinical data. Similarly, Subjects and Reducing Burden, Delay, and SACHRP suggests that an IRB determine whether Ambiguity for Investigators.”22 A number of a transfer of specimens to a new bank or institution changes have been proposed to enhance is permissible under the initial consent—a relevant protections of research participants, while point for information transfers as well. facilitating valuable research and reducing the As with EHRs, there have been a variety of burden of research investigators. Included in the challenges to the use of biobanks for research proposal are suggestions that specifically address without specific individual consent. Many long- EHRs and large-scale electronic databanks. standing biobanks were established either for non-research purposes (e.g., newborn blood spot 3.2 Biobanks banks) or under a general consent allowing the use The increasing availability of biobanks, which are of leftover tissue in hospitals. While more recent defined as facilities that store biological material banks and repositories have been set up with a (e.g., serum, genomic material, pathology variety of consent protections, it is unclear what to specimens) from humans, raises additional do with existing repositories created without these concerns. In addition to the Federal regulations protections, or how to manage access to archived described below, there are also guidelines data within the repositories where initial consent governing the creation of a data repository or was either silent on the matter or significantly biobank (see Chapter 7). In particular, IRBs are limits future research. At least one author has charged with reviewing protocols for obtaining, suggested the creation of a new regulatory storing and sharing information; verifying oversight framework that would substitute for the informed consent; and protecting privacy and necessary individual informed consent for the use confidentiality. of existing data or tissue samples.24 Another The Secretary’s Advisory Committee on Human suggests using broad initial consents to cover a Research Protection (SACHRP) advises the variety of future uses.25 Litigation in Texas and Secretary of HHS on issues related to the Minnesota regarding the use of newborn blood protection of human subjects. SACHRP developed spots has highlighted this issue in the national Frequently Asked Questions (FAQs) to provide a dialogue, and development of additional framework for IRBs, institutions, and investigators regulations at the State level is likely. The ANPRM to consider relating to the collection, use, and cited above includes among its proposed changes storage of biospecimens.23 One of the FAQs states mechanisms to improve informed consent, that generally consent is necessary before moving including consent for the secondary use of pre- excess identifiable clinical specimens to a existing biospecimens and data. centralized databank. In rare circumstances, an Key unresolved issues relevant to both biobanks IRB may determine that the conditions for a and large information data repositories include: waiver of consent have been met. Relevant factors obligations to return individually relevant research to consider include: governance and oversight of results, future unforeseen research uses, the need the bank; protections in place for confidentiality/ to recontact participants (some of whom may not privacy; policies regarding access to specimens; wish to be recontacted or who are deceased), the the nature of the research for which specimens will financial burdens of recontacting, the limits on be used; the databank’s ability to locate/contact withdrawal of the sample or information, whether subjects; the risk of introducing bias into the the sample/information can be kept indefinitely, collection process; potential anxiety/confusion for whether commercial uses of the bank should be subjects; number of subjects; length of time since treated differently from noncommercial uses, and the specimens were first collected; and the the implications of large-scale database research likelihood a subject would object to research use.23 for socially identifiable groups. Moreover, as While these factors are designed to address the use

191 Section II. Legal and Ethical Considerations for Registries

technology continues to progress, so will the structure is the potential reframing of the ability to re-identify participants from data underlying ethical issues. The July 2011 HHS- deposited into biobanks and large data issued ANPRM states, “Although the regulations repositories. have been amended over the years, they have not De-identification and aggregate reporting alone kept pace with the evolving human research does not completely conceal identity.26, 27 For enterprise, the proliferation of multi-site clinical example, there is a considerable push to make trials and observational studies, the expansion of de-identified, aggregate-level data from Genome health services research, research in the social and Wide Association Studies (GWAS) publicly behavioral sciences, and research involving available in large repositories so that the data can databases, the Internet, and biological specimen repositories, and the use of advanced technologies, be combined with other studies for more powerful 22 analysis. However, an individual can be re- such as genomics.” The current Federal research identified by assessing the probability that an regulations are based on the Belmont Report, individual or relative participated in a GWAS which focused on the traditional clinical research through composite statistics across cohorts (such context. The HIPAA Privacy Rule was put in place as allele frequency or genotype counts). BioVU, more recently to protect the privacy of individually the Vanderbilt DNA Databank, has taken steps to identifiable health information and demonstrates diminish the risk of re-identification. BioVU is the challenges involved with balancing individual linked to a de-identified version of data extracted privacy with the information needs of a from an EHR in which all personal identifiers have comprehensive health system. The future focus on been removed. Thus, there is no identifiable electronic data repositories and the potential for information attached to the records. The large-scale observational studies to replace some disadvantages or tradeoffs in such design are that it clinical trial data require consideration of whether explicitly precludes both recontacting and linking the approaches used thus far should be adapted. with any information other than that contained For example, in discussing the possible use of the within the original EHR. It also prevents the return FDA’s Sentinel System as a of individual results—an issue that remains pharmacoepidemiological research database, controversial even when the study design allows it. Professor Barbara Evans identified three “novel challenges in applying familiar ethical The informed consent documents initially used for 29 biobanking research either stated explicitly that no frameworks.” The first is the possibility that with results would be returned to participants or the shorter time period between research results remained silent on the issue. More recently, there and clinical application, the historical is general agreement in the scientific community conceptualization of research versus treatment (or supporting the return of aggregate results to even public health practice versus research) may research participants. There is less agreement on be incorrect. Perhaps IRBs will need to consider return of individual results. Moreover, there is still both the potential direct medical benefits of an debate regarding the most ethically appropriate observational study, and potential participant mechanisms for returning results (e.g., when, how, health risks such as negative insurance coverage and by whom—physician or investigator). In 2010, determinations or changes in physician prescribing a National Heart, Lung, and Blood Institute patterns. Second, the creation of these massive (NHLBI) Working Group released revised databanks that span numerous States (and recommendations providing guidance on many of sometimes countries) raises issues about whether these issues, but the issue is far from settled.28 the “local context review” that forms the basis for the IRB system continues to be relevant. Although 3.3 Reconsidering the Ethical a detailed examination of State regulations is not Framework Governing Research part of this chapter, it is worth emphasizing the challenges faced by multistate registries, which Perhaps the most challenging part of the shift to may face different requirements for informed large database research and the current regulatory consent, different privacy protections, and even

192 Chapter 8. Informed Consent for Registries

different definitions of “human subjects research” 4.1 HHS and FDA General Consent from state to state. This can add enormous burden Requirements to the regulatory oversight system and significant For activities covered by the HHS and FDA complexity to these endeavors. Finally, this type of research regulations, eight basic elements of research raises questions about the meaning of information must be provided to research vulnerability and susceptibility to harm, and who participants:30 should be identified as a “vulnerable” population in need of additional protections. It may be that the 1. A statement that the study involves research, groups traditionally considered vulnerable in the an explanation of the purposes of the research clinical research context are not especially and the expected duration of the subject’s vulnerable in this context. Conversely, there may participation, a description of the procedures be groups particularly vulnerable to re- to be followed, and identification of any identification, or for whom re-identification poses procedures which are experimental. unique risks of psychosocial or economic harms, 2. A description of any foreseeable risks or but which would not usually be considered discomforts to the subject. vulnerable in clinical research. In fact, the need to understand potential group harms highlights the 3. A description of any benefits to the subject or limitations of the traditional ethical framework that to others which may reasonably be expected assumes the focus should be on the individual. from the research. More work is needed to consider the ethical 4. A disclosure of appropriate alternative framework that should guide large-scale procedures or courses of treatment, if any, that observational studies, but such exploration is might be advantageous to the subject. beyond the scope of this chapter. The challenges 5. A statement describing the extent, if any, to raised by these studies have implications for which confidentiality of records identifying research more generally and may lead to broader the subject will be maintained. regulatory changes such as those proposed in the ANPRM. 6. For research involving more than minimal risk, an explanation as to whether any 4. Regulatory Consent compensation and an explanation as to whether any medical treatments are available Requirements if injury occurs and, if so, what they consist of, or whether further information may be While a number of issues remain unanswered in obtained. this area, there is some clear guidance for registries that fall under the Federal research 7. An explanation of whom to contact for regulations. There are two primary sets of Federal answers to pertinent questions about the regulations governing the conduct of human research and research subjects’ rights, and subjects research. HHS regulates research whom to contact in the event of a research- supported by Federal money or covered under an related injury to the subject. institutional “assurance of compliance” (see 8. A statement that participation is voluntary, Chapter 7). The FDA regulates research that will refusal to participate will involve no penalty be used to support an FDA-regulated product. or loss of benefits to which the subject is Both sets of regulations largely have the same otherwise entitled, and the subject may consent requirements; relevant differences are discontinue participation at any time without indicated below. The HIPAA Privacy Rule also penalty or loss of benefits to which the subject contains individual authorization requirements for is otherwise entitled. uses and disclosures of individually identifiable health information for research purposes. Each of In addition, the FDA announced on January 4, these Federal regulatory areas will be discussed in 2011, that informed consent forms for applicable turn. clinical trials must include a statement that the trial

193 Section II. Legal and Ethical Considerations for Registries

information will be entered into the National regulations speak to the documentation of Institutes of Health (NIH) clinical trial registry.31 informed consent.37 Unlike treatment consents, Both the HHS and the FDA regulations also research consents are usually written and the require, where appropriate, additional elements of consent form functions both as documentation of informed consent, including the following:32 the consent process, and in some cases as an 1. A statement that the particular treatment or aspect of the consent itself (since in long form the procedure may involve risks to the subject (or document contains all of the necessary consent to the embryo or fetus, if the subject is or may disclosures and participants may be given the form become pregnant) which are currently to read as part of the consent process). HHS allows unforeseeable an IRB to waive the written documentation requirement in whole or in part when (1) the only 2. Anticipated circumstances under which the record linking the subject and the research would subject’s participation may be terminated by be the consent document and the principal risk of the investigator without regard to the subject’s the study would be potential harm resulting from a consent. breach of confidentiality, or (2) the research 3. Any additional costs to the subject that may involves no more than minimal risk of harm and result from participation in the research. involves no procedures for which written consent is not normally obtained in a clinical context.38 In 4. The consequences of a subject’s decision to either case, the participant may still be provided a withdraw from the research and procedures written summary. Waivers may be used to allow for orderly termination of participation by the oral consent procedures without written subject. documentation. Such an approach is often used in 5. A statement that significant new findings research involving interviews conducted by developed during the course of research which telephone. may related to the subject’s willingness to Under certain conditions, an IRB may approve the continue participation will be provided to the use of a short-form written consent.39 In these subject. cases, oral presentation of informed consent 6. The approximate number of subjects involved information is accompanied by a short-form in the study. written consent document (stating that the elements of consent have been presented orally) The HHS regulations allow an IRB to approve a and a written summary of what has been presented waiver or alteration of the consent requirements orally. A witness to the oral presentation is for minimal risk research where the waiver or required, and the participant (or representative) alteration will not affect the rights of the subjects, must be given copies of the short form document the research cannot be carried out without the and the summary. The participant must sign the waiver, and, when appropriate, subjects will be short form document and the witness must sign provided information after participation.33 The both the short form and the summary. FDA regulations do not allow waivers or alterations under these circumstances, but do allow E-consents may be considered written for waivers in life-threatening situations and allow documentation under either set of regulations and Presidential waivers for some military research.34 are within the scope of the IRBs’ power to Both sets of regulations allow for waiver of authorize as an alternate way of fulfilling written consent requirements for research conducted in documentation requirements under the HHS specific types of emergency situations.35, 36 regulations. HHS specifically allows electronic signatures on research consent documents, 4.2 Documentation and Format of provided they are legally valid in the specific Consent jurisdiction.40 FDA also has provisions for There are varying requirements for documentation e-signatures on electronic records, but does not of the consent process. Both FDA and HHS speak directly to e-consent for research

194 Chapter 8. Informed Consent for Registries

participation.41 While the Federal Electronic followup for health-related reasons, leaving an Signatures in Global and National Commerce Act overall healthier group of participants. (E-SIGN) attempts to provide some uniformity Additionally, even among those who can be among State laws governing electronic contacted, individuals who agree to continue transactions, there remain some variations.42, 43 participation may be different from those who The primary goals of e-signature laws are refuse to provide consent. As a result, one authenticating the signature and ensuring privacy important requirement for studies that undertake and confidentiality of electronic information. re-consent may be to evaluate characteristics of the Although there have been some suggestions for original study population, as compared with the standardized electronic consent procedures, there subset of patients that do re-consent, and consider is little discussion focused specifically on the the implications for research outcomes. The research area.44 evaluation of whether re-consents are more common for particular populations should be done 4.3 Informed Consent Form Revisions for any analyses that have comparative arms. and Re-consent Minor changes to a consent document do not Changes to the informed consent document may necessitate re-consent. Re-consent is necessary, require re-consent of patients; for example, however, where the terms of the study or the re-consent may be necessary if there are changes background preconditions have changed. In some in the scope of the registry, substantive changes to long-term studies, re-informing participants, but the protocol, addition of procedures not previously not re-consent, may be necessary. Even where addressed in the consent, changes in reasonably re-consent is needed, IRBs may waive foreseeable risks or potential benefit, changes in requirements. Alternatively, data collection, data sharing or reporting procedures, or identified sharing, and reporting for participants who cannot errors or omissions in the original consent be re-consented could be maintained in accordance document. Such revisions must be reviewed and with the terms of the original consent. In those approved by an IRB prior to the revised consent situations in which a re-consent process is being utilized, except when necessary to eliminate implemented, participants should be told the apparent immediate hazards to subjects. Re- reasons for the recontact and given a summary of consent may also be necessary if the participants consent form changes. Additionally, as with the were below the age of consent when initially original consent, documentation of the re-consent enrolled but reach the age of majority when the must be maintained as required by the registry, the registry is still active. The decision to change the IRB, and any relevant regulations. informed consent form and subsequently obtain the re-consent of participants needs to be carefully 4.4 Applying the Federal Research considered due to possible challenges in obtaining Regulations to Registries the re-consent. Challenges may be particularly Some of the regulatory requirements appear better evident for registries that have been in place for suited to traditional clinical research trials than to several years. These difficulties are what prompted registries. For example, of the eight basic elements the interest in broad general initial consents. In listed earlier, requirements 4 (alternatives) and 6 situations where the initial consent does not cover (compensation/injury) are crafted to address issues the change, registries may seek IRB waiver of raised in traditional clinical trials, rather than re-consent requirements. registries. Other elements have aspects that clearly For studies in which re-consent is sought, registry encompass registry research (such as basic developers should consider the potential effects of elements 1, 2, and 7), but other parts seem less selection bias and the implications for external applicable, since registries will not involve validity. Re-consented participants may be “experimental procedures” that must be identified, systematically different from non-re-consented entail no physical “discomforts to the subject,” and participants. For example, participants that have do not pose a risk of (physical) “research-related not re-consented may have died or been lost to injury.”

195 Section II. Legal and Ethical Considerations for Registries

Other requirements may pose challenges for framework in this context and the specifics of registries, such as basic element 8, which requires coverage. The Privacy Rule requires a covered that subjects be informed about a right to entity to obtain written authorization for the use or withdraw. While registry participants may refuse disclosure of an individual’s PHI for research to provide additional information about their purposes unless the use or disclosure is permitted medical status or care, withdrawing from a registry by another provision of the Rule (e.g., where an may undermine the data collection. In situations IRB waiver of authorization applies). A subject’s where the data have been anonymized, withdrawal informed consent to participate in research can be will likely prove impossible. In many such cases, combined with a HIPAA authorization in one registry informed consents may contain language document. There are six core elements and three notifying subjects that in the event of withdrawal, required statements for a HIPAA authorization:47 data that was collected prior to the withdrawal may Core Elements continue to be used and disclosed according to the consent in order to preserve the scientific integrity • A description of the PHI to be used or of the registry. However, even where data have not disclosed, identifying the information in a been anonymized, some argue that the registry specific and meaningful manner. must retain all records to be a valid information • The names or other specific identification of the tool. The FDA explicitly requires the retention of person or persons (or class of persons) identifiable data even after a subject withdraws authorized to make the requested use or from a study. HHS permits the retention of such disclosure. data, but also permits the investigator to omit or destroy the data if retention is not required by FDA • The names or other specific identification of the regulations for study integrity. OHRP suggests that person or persons (or class of persons) to whom IRBs provide guidance on documentation of the covered entity may make the requested use participant withdrawal. Moreover, the OHRP or disclosure. guidance dated September 21, 2010, on this issue • A description of each purpose of the requested clarifies that once a subject withdraws, the use or disclosure. investigator must stop interacting with the subject • Authorization expiration date or expiration to obtain data, and stop collecting identifiable event that relates to the individual or to the private information from other sources unless the purpose of the use or disclosure. “End of the subject specifically provides consent to the research study” or “none” are permissible for continued data collection.45 research, including for the creation and 4.5 HIPAA maintenance of a research database or repository. The HIPAA Privacy Rule may apply to uses or disclosures of health information into or out of a • Signature of the individual and date. If the registry, or the use of such information to create a individual’s legally authorized representative registry, or both. In addition, because the HIPAA signs the Authorization, a description of the Privacy Rule protects individually identifiable representative’s authority to act for the health information held by covered entities, the individual must also be provided. Privacy Rule requirements may apply even if the Required Statements human subjects research regulations do not. • A statement of the individual’s right to revoke Moreover, the Food and Drug Administration Authorization in writing, and either: (1) a Amendments Act of 2007 (FDAAA) requires all description of how to do so, and the exceptions qualified entities with which it contracts to provide to the right to revoke authorization, or (2) analyses of drug safety data, regardless of whether reference to the corresponding section of the they are a HIPAA-covered entity, to follow the covered entity’s notice of privacy practices. minimal requirements of the Privacy Rule.46 Chapter 7 describes the general Privacy Rule

196 Chapter 8. Informed Consent for Registries

• Whether treatment, payment, enrollment, or Additionally, uses or disclosures of decedents’ PHI eligibility for benefits can be conditioned on to a research registry or from a registry for the individual’s signing the Authorization, research purposes do not require an authorization including research-related treatment, and (as long as certain representations are provided to consequences of refusing to sign the the covered entity that is providing the Authorization, if applicable. information).49 Authorizations are also not • A statement of the potential for the PHI to be required for uses or disclosures of de-identified re-disclosed by the recipient and no longer data sets, provided the information has been de-identified in accordance with the Privacy protected by the Privacy Rule. This may be a 50 general statement that the Privacy Rule may no Rule. Nor are authorizations required for uses or longer protect health information disclosed to disclosures of “limited data sets,” as defined by the the recipient. Rule (so long as a data use agreement is in place with the recipient of the limited data set).51 See Authorization is not needed for activities that are Chapter 7. “preparatory to research,” which may include scanning a patient database to determine feasibility In addition, an IRB or privacy board may waive or for creating a registry. Before allowing an alter aspects of the HIPAA authorization investigator access to PHI for such purposes, requirements. Like the requirements for a waiver however, the covered entity must obtain from the or alteration under the human subjects research researcher representations that: (1) the use or regulations described above, these are limited to disclosure of PHI is sought solely for purposes situations in which the research could not preparatory to research, (2) no PHI will be practicably be carried out without the waiver or removed from the covered entity during the review, alteration and could not practicably be carried out and (3) access to the PHI is necessary for the without access to the PHI, and the use or research purposes.48 These preparatory activities disclosure information involves no more than may aid investigators in the identification of minimal risk to privacy because there is: (a) an potential research participants. Subsequent contact adequate plan to protect the identifiers from of potential research participants for purposes of improper use or disclosure; (b) an adequate plan to obtaining authorization for the use or disclosure of destroy identifiers if possible; and (c) adequate the individual’s PHI may be permitted under the written assurances that the PHI will not be reused Privacy Rule in a variety of ways depending on the or disclosed except as required by law, as needed for research oversight, or for other research in a relationship between the investigator and the 52 covered entity. An investigator who is a workforce way permitted by the Privacy Rule. member of the covered entity is permitted to Finally, if a subject was enrolled in a research contact potential participants directly or through protocol prior to the compliance date of the another person at the covered entity, such as a Privacy Rule (for most covered entities, April 14, treating provider, to obtain authorization. 2003) and pursuant to a valid informed consent, an Alternatively, a covered entity is permitted to hire authorization may not be required unless after the a business associate—who may be an compliance date another informed consent is investigator—to contact patients to obtain sought from the subject.53 This provision may be authorization on behalf of the covered entity. especially relevant to registries that were created Finally, a covered entity is permitted to provide prior to the application of the Privacy Rule. contact information of potential research subjects The HIPAA Privacy Rule also speaks to the issue to an investigator who is not part of the covered of withdrawal from a registry. The Privacy Rule entity or a business associate, if the covered entity explicitly gives individuals the right to revoke their obtains documentation that an IRB or privacy authorization for the use and disclosure of board has waived the authorization requirement for protected health information (the revocation must the disclosure. be in writing), except to the extent that a covered

197 Section II. Legal and Ethical Considerations for Registries

entity has already relied on the authorization. HHS described in the authorization such that it would be guidance on the application of the Privacy Rule to reasonable for the individual to expect that her or research makes it clear that a covered entity that his PHI could be used or disclosed for such future has disclosed PHI for research in reliance on an research. authorization is not required to retrieve information it disclosed prior to receiving the 4.6 Special Consent Issues: revocation, and may also continue to use and Incapacitated Adults and Children disclose PHI already obtained to the extent In addition to the general requirements discussed necessary to preserve the integrity of the study above, there are also additional requirements for (e.g., as necessary to account for the subject’s certain specific research populations. HHS has withdrawal). As noted above, FDA states that the regulations that apply to pregnant women and data gathered as part of research under their fetuses, children, and prisoners. FDA has regulatory authority is necessary and must be regulations that apply to children (which, for the retained; but even for those registries outside the most part, match the HHS regulations). Both also scope of FDA oversight, HIPAA permits the allow research to be conducted with adults lacking continued use of data as necessary to protect the decisional capacity, although consent must be integrity of the research. obtained by a “legally authorized representative,” In the past, concerns had been raised that the who may be a guardian, proxy, or surrogate HIPAA authorization requirements were not decisionmaker (the terms are defined by State sufficiently harmonized with the human subjects law). Likewise, HIPAA also allows for research informed consent requirements.17, 54 This authorizations from “personal representatives” was due to the fact that, while a HIPAA (again, generally defined by State law). authorization could be combined with a research Of particular interest to registries are the research informed consent document (and elements already regulations pertaining to children. Unlike research present in the research consent were not required involving adults, research involving children must to be repeated in the authorization), there were fit into one of four categories: minimal risk,55 some situations in which an additional separate greater than minimal risk/prospect of direct authorization may have been necessary for a therapeutic benefit,56 minor increase over minimal separate research activity or future research risk/likely to yield generalizable knowledge about activity. In January 2013, HHS published changes subject’s disorder or condition,57 and research not to the HIPAA Rules that addressed these issues. In otherwise approvable but authorized by the particular, the Privacy Rule now permits the use of Secretary of HHS in consultation with an expert a compound authorization to authorize the use or panel.58 Most registry research is likely to fall into disclosure of an individual’s PHI for a conditioned the minimal risk category. For these studies, research activity (e.g., a clinical trial where permission must be obtained from at least one treatment is conditioned on the individual’s parent/guardian and assent obtained from the child, authorization) as well as an unconditioned research if the child is capable of assenting. activity (e.g., the use or disclosure of PHI to create or contribute to a separate research database or Waivers of both permission and assent are repository), provided that the compound possible. Under HHS regulations, a waiver of authorization clearly differentiates between the parental permission is allowed under the same conditioned and unconditioned research conditions that allow for a waiver of informed 59 components and provides the opportunity for the consent in adult populations; or when parental individual to opt in to the unconditioned research permission is not a reasonable requirement to 60 activities. The final rule also modifies HHS’ protect the subjects. FDA regulations do not interpretation of the Privacy Rule’s research allow for waivers of parental permission. Both authorization requirements to permit authorization HHS and FDA regulations allow a waiver of assent for future research uses and disclosures, as long as when the research involves an intervention holding the future research purposes are adequately the potential for direct therapeutic benefit and is

198 Chapter 8. Informed Consent for Registries

not available except through participation; or when such time extends beyond the child’s age of parental permission is waived in accord with majority. If the authorization expires on the date section 46.116 of Subpart A.61 Moreover, when the minor reaches the age of majority, a covered some or all of the children involved are not capable entity would be required to obtain a new of providing assent, an IRB can determine that authorization signed by the individual in order to assent in not necessary (for the child or children in further use or disclose PHI covered by the expired question, or for all children if appropriate). Both authorization (or ensure that the use or disclosure sets of regulations allow an IRB to determine that falls within another regulatory permission in the permission is only required from one parent, even Rule). Registries that involve children that will when required from both under 406 or 407, in retain identifiable information past the child’s age limited circumstances.62 Where authorization must of majority will need to take steps to gain the be obtained for the use or disclosure of PHI, the appropriate consent and, if necessary, authorization HIPAA Privacy Rule requires authorization from for continued use. Less clear is whether only one personal representative of the individual, investigators should seek a child’s assent to such as one parent of a minor child, and does not continued participation when the initial consent require assent of the child. was provided by parents at a time when the child OHRP has indicated that when the research in lacked the capacity to play any role in question involves a treatment for which the child decisionmaking. would have legal authority to consent, the child’s consent may suffice and parental permission may 5. A Proposed Framework for be unnecessary. The HIPAA Privacy Rule also Registry Consents generally provides that when a minor has legal authority to consent to a particular health care 5.1 Current Practices and Problems service without the involvement of a parent, the minor and not the parent has authority to act as the There are three current approaches to consent: individual with respect to the PHI pertaining to opt-in, opt-out, and non-consent. An opt-in that health care service. State statutes granting approach assumes that an individual will not be decisionmaking authority to minors vary. Many part of the registry until they have specifically address issues such as treatment for sexually consented to participation. An opt-out approach transmitted infections (STIs), access to assumes that all individuals will be part of a contraception, and some even allow consent for registry, unless there is a specific refusal to mental health or substance abuse treatments. participate. Finally, a non-consent model does not Registries involving these areas may be able to rely seek or require individual consent or refusal, but on the minor’s consent, rather than the parental includes all relevant individuals in a registry. The permission/assent framework. However, more labeling of the approaches may vary in the specific legal guidance on the particulars of State literature, but the general concepts remain statutory interpretation may be warranted in these consistent. Additionally, some registries involve a situations. mix of one or more approaches or a combined consent mechanism, where an opt-in approach is Another important consideration is what to do used for one aspect (access to a particular when a minor who is involved in a registry reaches treatment) and non-consent for the other (listing in the age of majority. OHRP interprets the the treatment registry). This may also be referred continuing consent standard to require that legal to as “conditional access.” consent be sought from the participant upon reaching the age of majority. An authorization 5.1.1 The Opt-In Approach under the Privacy Rule, including one signed by a An opt-in procedure may involve a consent process parent as the personal representative of a minor, similar to that used for clinical research protocols. remains valid until it expires or is revoked, even if It may be used separately for a registry, or it may

199 Section II. Legal and Ethical Considerations for Registries

be appended to a consent document used for a individuals are given an easy way to opt-in or particular treatment (for example, individuals who opt-out.64 While the Federal research regulations consent to the use of a particular device may also appear to assume an opt-in approach, in some be asked to participate in a registry for that circumstances an IRB could approve a device). While an opt-in approach has the benefit modification that allowed a shift to an opt-out. In of assuring compliance with the Federal order for an IRB to approve an opt-out approach regulations, a number of the regulatory for non-exempt, HHS-supported human subjects requirements are difficult to apply to registries (as research, the researchers must document that the discussed above). This has led many to suggest a waiver of informed consent is appropriate for the modified opt-in approach—using elements of the research. An opt-out approach may be especially clinical research framework but adjusting to fit the useful for registries. Nonetheless, Privacy Rule registry model. But, even with a modified model, requirements will preclude this approach unless there are concerns that the strict informed consent the situation fits within one of the delineated requirements of the clinical research consent will permissible uses without an individual have negative effects on subject selection, resulting authorization (e.g., with a waiver of authorization in biases that will undermine the validity and thus for research or for public health activities). affect the usefulness of the registry. An analysis of IRBs could consider the opt-out approach for the Canadian Stroke Network estimated that research that meets the four criteria for a waiver or dealing with consent issues cost $500,000 over the alteration of consent under the HHS guidelines: first 2-3 years of the registry, and the requirement (1) the research involves no more than minimal to obtain written informed consent introduced risk to participants; (2) the waiver or alteration will significant selection biases undermining the 63 not adversely affect the rights and welfare of usefulness of the registry. Alternative consent participants; (3) the research could not practicably approaches may need to be considered for large- be carried out without the waiver or alteration; scale observational studies. (4) participants will be provided with additional 5.1.2 The Opt-Out Approach pertinent information after participation. For An opt-out procedure shifts the presumption from example, the Vermont Diabetes Information one in which each individual must consent to System (VDIS) is a quality-improvement, registry- participate, to one in which each individual must based decision support and reminder system refuse to participate. There is a great deal of targeted to primary care physicians and their patients with diabetes. With IRB approval, VDIS discussion about the usefulness of an opt-out 65 model, particularly for registries (e.g., organ incorporated an opt-out consent process. Patients donation registries). To be a valid opt-out model, are notified by mail of their eligibility and individuals must be fully informed about the inclusion in the registry and given a mechanism to existence of the registry and their rights to opt-out opt-out by calling a toll-free number. of participation. In many cases, the information 5.1.3 The Non-Consent Approach requirements are the same as the information Non-consent is not really a consent mechanism requirements for an opt-in procedure—the only and thus will not be addressed here in detail. difference is that instead of explicitly agreeing to Nonetheless, this approach may, and probably participate, the person must take steps explicitly to should, entail providing participants with refuse to participate. While the information information about the registry. The format and requirements may not change, the psychological process of disclosure may vary. In some cases, shift may be significant. If the expectation is that general public notifications (perhaps listing on a everyone will participate, people may be more Web site, or posting prominently in a place likely inclined to acquiesce. There is evidence in other to be seen) will be sufficient. In other cases, areas of decisionmaking that setting the default to individual notification may be appropriate. A participation results in greater inclusion than non-consent approach is used currently for setting the default to non-participation, even when registries that fall outside the Federal research

200 Chapter 8. Informed Consent for Registries

regulations such as State-mandated public health The real question related to the scope of consent is reporting or quality improvement activities. One to what extent consent can and should authorize primary methodological advantage of the non- future unspecified uses. In other words, how broad consent approach in no-risk and minimal risk a consent is permissible? The exercise of studies is that it can function to reduce concerns autonomy should include the ability to consent about biases introduced by the consent process, both to specific and to non-specific research such as those that occur when individuals who participation. An individual who would like to give consent to participate in the registry systematically broad permission for the use of their data in any differ from those who do not or cannot consent. future registry (or for use in a particular registry, Besides debates about when the use of a non- but include permission that the information may consent approach is acceptable (based on the level be shared with investigators for any future research of permissible risk), most of the focus in this area query) is exercising a form of autonomy. In should be on the type and extent of required addition, part of the issue is in determining notifications. whether a broad consent was truly informed. In the absence of specific details about the future uses, 5.2 Scope of Consent decisionmaking is necessarily less informed than if Consents may be broad or narrow. A so-called every future use is spelled out clearly. However, “blanket consent” approach asks for consent to a the ethical doctrine of informed consent does not wide category of uses and assumes that consent require this level of detail. Moreover, requiring will cover all uses, unless one is specifically multiple consent dialogues may respect autonomy excluded. Blanket consent should be distinguished less than permitting broad consent if the individual from broad or general consent that does not does, in fact, wish to give broad permission and necessarily imply “blanket” consent to all uses. In does not want continued recontact. In some agreement with legislation, broad consent refers to contexts, such as the donation of biological use in biomedical research, not to other kinds of samples, broad consents are more acceptable (e.g., uses, such as for forensic use or for use by there is a long history of allowing unrestricted immigration authorities. A blanket consent model tissue donations). It has become common to has historically been relatively common and still provide a menu of options in a consent form for exists in some contexts. For example, patients biological or genetic databanking. These allow entering a health institution or agreeing to a participants to specify any constraints they would procedure sometimes have a notation at the bottom like to place on the of their samples, such as of the general consent form allowing the use of permitting use only for the specific study listed, or leftover tissue in any way deemed appropriate by for all studies in a particular research areas (e.g., the institution. Extremely broad blanket consents heart disease), or for any future study in any area. are not generally viewed as valid exercises of Details regarding whether and under what autonomy and thus may not truly be considered to circumstances the participant would like to be be “informed consent.” At best, blanket consent recontacted may also be collected. By contrast, in may be viewed as a type of notification procedure, other areas such as consent to participate in a alerting individuals to the possible uses of their clinical trial, broad consents (e.g., “I give consent information. Neither the Federal human subjects to participate in any clinical trial”) are insufficient research regulations nor the Privacy Rule permit on both ethical and regulatory grounds. For extremely broad blanket consents. Some registries situations such as the use of medical information, will have been created, with the use of a prior the scope of a broad consent is less clear. The express legal permission from individuals, before debates about the scope of consent are ongoing. the compliance date of the Privacy Rule, and may additionally fall under an exemption to the human 5.3 Oversight and Community subjects research regulations; in these Consultation circumstances previously obtained broad blanket Consent is only one aspect of the protections in consent may be deemed sufficient. place for human research participants. The second

201 Section II. Legal and Ethical Considerations for Registries

major part involves IRB review and oversight. development. The sensitive nature of information Other chapters discuss the oversight roles for IRBs about participants and the potential for broad data and registry governance boards. distribution highlight the importance of the Community consultation, a third concept that informed consent process. Moving forward, usually appears in the context of discussions on informed consent elements and guidelines specific human subjects research consent, is not actually to registry research should be developed. part of this system of protections. In fact, there is 6.1 Special Considerations no simple community analog to individual consent. Consent requirements for research arise from the Given the nature of registry research, some principle of autonomy, and there is no elements of informed consent should be given corresponding principle at the community level. special consideration, including: the scope of the Thus, concepts such as “community consent” or use of registry data, potential for recontact, “community authorization” can be incoherent, in withdrawal, and information regarding the part because there is no unitary concept of a electronic data security and management to be community. Communities may be defined on employed. social, biological, religious, racial, cultural, or 6.1.1 Scope of Use of Registry Data geographic grounds. Most people belong to multiple, sometimes overlapping, communities. Registries constitute a valuable resource, since Some of these communities may have a designated investigators often draw upon them to address spokesperson, but this individual may not questions extending far beyond those envisioned represent the interests of all members of the when the registries were first created. Therefore, community. (Consider, for example, the complex informed consent for registry research that allows relationship between the Pope and Catholics in the broad data sharing is optimal for promoting United States.) Other communities have no clearly science. There may be instances, however, such as identified spokesperson. It is inappropriate to with respect to research on specific diseases (e.g., consider community consultation as a replacement HIV/AIDS research), where more specific consent for individual consent. Rather than view may be appropriate. Additional Federal-level community involvement as an aspect of consent, it guidance on the appropriate scope of broad should be considered as part of oversight (and an consent for future uses will be important. In the analog of IRB review).66 Nevertheless, community meantime, registry developers should not only involvement in the design and oversight of a provide clear parameters regarding the scope of registry may be particularly important when the use of registry data when first creating the registry, registry involves socially identifiable groups that but should also develop a mechanism to consider have been subject to historic discrimination or how future, possibly unanticipated, requests for when it involves sensitive genetic information. In data access will be evaluated. The registry addition, in some cases, community involvement governance board can play an important role in can enhance participant understanding of consent this situation. and thereby increase individual participation. 6.1.2 Recontact Individuals should be informed how their data/ 6. Consent Guidance samples will be used at the point of entry into the registry. Whether and how participants will be Although general agreement has been reached recontacted should be established at the outset and about the required elements of informed consent included in the consent form. Exceptions should for clinical research, this model may not be be considered specifically where data/samples entirely applicable to informed consent for the were made irretrievably anonymous, since creation of and participation in registries. Risks to recontact would then be impossible. It is important participants (and, when applicable, risks to groups to inform registry participants that the and/or communities) should be balanced carefully anonymization of their data will make withdrawal with the public health benefits of registry from the registry impossible.

202 Chapter 8. Informed Consent for Registries

6.1.3 Withdrawal sponsors, investigators, Protocol Review Many issues governing withdrawal from a registry Committees (PRCs), and IRBs. Moreover, despite have been discussed in this chapter. Consensus the multitude of elements listed below, every effort needs to be developed regarding whether should be made to keep consent forms as short as withdrawal should be presented as an option to possible and at approximately a fifth to eighth participants in the initial consent, and, if it is an grade reading level. option, how withdrawal will be managed. While The following elements should be considered: withdrawal from a traditional research study is a 1. A statement that the individual is being asked basic subject right, withdrawal of collected data, to take part in a registry (or a research study, even from clinical trials, may be restricted. It is if applicable), including— extremely important that registry creators develop initial rules and procedures for withdrawal and a. The name of the specific registry for which fully inform participants of them. consent is being obtained. 6.1.4 Electronic Data Security b. An explanation of the purposes of the registry (why it was created, who will be Given the public concerns about electronic data included). security, participants in the registry should be clearly informed as to the physical security of their c. The expected duration of participation. data and/or biospecimens, including methods of d. A description of the procedures entailed. coding and removal of identifiers, encryption techniques, potential for cloud computing, and e. The approximate number of subjects quality assurance policies. As well, participants involved (if applicable). should be informed about the process of releasing 2. A description of any foreseeable risks or and transferring data to future investigators as it inconveniences (specifically risks related to relates to maintaining confidentiality. In some any potential breach of confidentiality related cases, this information will reassure participants, to the data being collected). potentially increasing consent rates. a. When human genetic research is 6.2 Proposed Consent Form Elements anticipated, information should include possible consequences of genetic testing The following is an outline of potential elements to (e.g., insurance risks, paternity consider when developing consent forms and determinations, potential risks to family engaging in consent dialogs for registry research. and community) and other related These elements were generated from the applicable confidentiality risks. HHS, FDA, and Privacy Rule requirements and include consent aspects particularly relevant to 3. A description of the types of research that the registry research. These are all issues that should repository will support, and any benefits to the be considered; there may be additional legal subject or to others which may reasonably be requirements (e.g., for a HIPAA authorization). expected, including— The outline below should not be viewed as a. A statement about whether and how comprehensive or even applicable to all registries. findings will be communicated to Some registries will modify this outline to meet participants. specific needs, while others will follow a consent 4. A statement describing the extent to which procedure similar to one used for traditional confidentiality of data/biospecimens clinical trials. However, this outline provides a identifying the subject will be maintained starting place for understanding the scope of (including a description of the operations of informed consent for registry participation. It is the repository—how data/specimens will be important to note that the responsibility for stored and managed), including— obtaining and assuring appropriate informed consent rests on multiple parties, including

203 Section II. Legal and Ethical Considerations for Registries

a. If applicable, a statement about whether 7. A statement of whether there are any costs to registry results will be published. participation and/or any payment for b. A statement about the impact of participation. participation on the subject’s access to his/ 8. A statement that participation is voluntary, her medical records (e.g., that access may that refusal to participate will involve no be limited until all work on the registry is penalty or loss of benefits to which the subject completed). is otherwise entitled, and that the subject may 5. The conditions and requirements under which discontinue participation at any time without data and/or specimens will be shared with penalty or loss of benefits to which the subject recipient investigators, including— is otherwise entitled. a. If applicable, a description that the data/ a. The consequences of a subject’s decision to specimens will be broadly shared and may withdraw from the research, including the be used for future research that is not yet possibility that the previously collected identified. data will continue to be used, and procedures for orderly termination of b. The fact that the data/specimens may be participation by the subject. transferred to other institutions and explanation of a data transfer security plan. 9. Details on who to contact for answers to pertinent questions about the research and 6. A description of when recontact might be research subjects’ rights. necessary, and how recontact will be handled. 10. As appropriate, any State-specific addenda.

204 Chapter 8. Informed Consent for Registries

Case Examples for Chapter 8

Case Example 14. Issues with obtaining patients for consent. Informed patient consent informed consent was required for full data collection, linkages to Description The Registry of the Canadian administrative data, and 6-month followup Stroke Network (RCSN), now interviews. known as the Ontario Stroke Informed consent was required for full data Registry, is a registry of stroke collection. Unfortunately, consent was obtained patients in Canada. The registry, for only 39 percent of eligible patients. currently in Phase V, is a non– Subsequent analyses showed that patients who consent-based registry that consented to participate were not representative collects detailed clinical data on of the overall stroke population, as they were less the acute stroke event from the likely to have severe or fatal stroke, and also less onset of symptoms, including likely to have minor stroke or TIA. emergency medical service Phase II of the registry began in 2002, with 21 transport, emergency department hospitals and 4 Ontario Telestroke sites. In this care, and hospital discharge phase, all patients admitted to the hospital or seen status. The purposes of the in the emergency department with symptoms of registry are to monitor stroke acute stroke within 14 days of onset or TIA were care delivery, to evaluate the included. Patients with in-hospital stroke were no Ontario Stroke System, and to longer recruited. In order to standardize workload provide a rich clinical database across the country, a random sample of eligible for research. patients was selected to be approached for Sponsor Canadian Stroke Network, consent for full data collection. Consent was Networks of Centres of obtained from 50 percent of eligible patients. Excellence, and Ministry of After obtaining consent of only 39 percent and 50 Health and Long Term Care of percent of patients in Phases I and II, the team Ontario realized that obtaining written patient consent for Year Started 2001 participation in the registry on a representative Year Ended Ongoing sample of stroke patients was impractical and costly. Patient enrollment threatened the viability No. of Sites 154 and generalizability of the stroke registry. The No. of Patients More than 60,000 registry team published these findings in the New England Journal of Medicine in April 2004. Challenge Proposed Solution The registry began in 2001 with Phase I, in which data were gathered from 21 hospitals in Canada. The registry team approached the Ontario All patients admitted to the hospital or seen in the Information and Privacy Commissioner to discuss emergency department with symptoms of acute a non-consent-based registry for Phase III. As a stroke within 14 days of onset or transient result of these discussions, the registry was ischemic attack (TIA), as well as those with acute “prescribed” by the Privacy Commissioner under in-hospital stroke, were included in this phase. the Personal Health Information Protection Act, Research nurse coordinators identified eligible 2004. This decision allowed the registry to collect patients through daily reviews of emergency and data legally on stroke patients without written admission patient lists and approached these consent.

205 Section II. Legal and Ethical Considerations for Registries

Case Example 14. Issues with obtaining Key Point informed consent (continued) The impact of obtaining informed consent should Results be considered in developing a registry. Requiring Phase III of the registry included all patients that registries obtain the consent of patients with presenting to emergency departments of the 11 acute medical conditions such as stroke may “Stroke Centres” in Ontario and 1 center in Nova result in limited selective participation, as it is not Scotia with a diagnosis of acute stroke or TIA possible to obtain consent from all patients. For within 14 days of onset. Nurse coordinators example, patients who die in the emergency identified eligible patients through daily reviews department and patients who have brief hospital of emergency and admission patient lists. Patients visits may be missed. Mechanisms such as were identified prospectively, with retrospective obtaining a waiver of informed consent or using chart review, without consent. No followup the approach outlined in this case may be interviews were done. Because informed consent alternatives. was not required, the data collected provided a For More Information representative sample of stroke patients seen at Tu JV, Willison DJ, Silver FL. et al. The tertiary care centers in Canada, making the data impracticability of obtaining informed consent in more viable for use in research and in developing the Registry of the Canadian Stroke Network. initiatives to improve quality of care. The registry N Engl J Med. 2004;350:1414–21. has now expanded to include a population-based, province-wide audit of stroke care delivery on a 20-percent sample of patients from every acute care institution in Ontario.

Case Example 15. Operationalizing informed patient registry, patients are recruited at their consent for children annual well-child visits and followed up for ten Description TARGetKids! (Toronto Applied years during subsequent annual well-child visits. Research Group) is a prospective Participation involves completion of age-specific registry enrolling healthy questionnaires related to the child (asking for children aged 0-5 years. The aim nutritional, behavioural, and developmental of the registry is to link early information), collection of anthropometric nutritional exposures to later measurements, and collection of the child’s blood health outcomes including sample (4-7 mL) by a trained pediatric obesity, micronutrient deficiency, phebotomist. and developmental outcomes. Consent for the registry is provided by one or Sponsor University of Toronto both parents. By signing consent, parents also authorize the collection of their child’s health Year Started 2008 card number to allow researchers to access the Year Ended Ongoing child’s health records. Registry staff anticipated challenges in obtaining informed consent for No. of Sites 7 primary care practices in these activities, particularly given the infrequency Toronto, Canada of contact with patients and the fact that blood No. of Patients 4,287 sample collection is not part of normal clinical care during these annual visits. After reviewing Challenge the registry protocol, a research ethics board Research involving children faces unique recommended that steps be taken to minimize challenges, including special requirements related coercion when recruiting and obtaining consent to the informed consent process. In this pediatric from patients.

206 Chapter 8. Informed Consent for Registries

Case Example 15. Operationalizing informed Of consenting parents, about 50 percent consent consent for children to the registry blood sample. One possible reason Proposed Solution parents choose to consent to the blood sample is that they see value in the test results (i.e., for iron Two weeks before a scheduled well-child visit for or vitamin D deficiency) which are not standard an eligible patient, the physician’s office mails a of care for children in Canada. Parents may short informational letter to the child’s home. The perceive this as an added benefit, although, in an purpose of the letter is to provide information attempt to minimize coercion, the informed about the registry and prepare parents for their consent form and registry staff do not emphasize contact with registry staff during the visit. By this fact. providing this information in advance, the letter minimizes the possibility that parents will feel Key Point coerced to consent to registry participation. Providing patients (and parents of pediatric On the day of the visit, the child’s parents are patients) with information about the registry in approached by a registry research assistant to advance can give them time to prepare questions provide consent for participation of their child in and thoughtfully consider whether they wish to the registry. The research assistant explains what consent to participation. A flexible consent participation entails (i.e., completing structure that allows patients to opt out of questionnaires, collection of anthropometric activities of a sensitive nature can reduce barriers measurements, and collection of a blood sample). to consent and participation. If the parent spontaneously expresses that they For More Information wish to participate in the registry but don’t wish http://targetkidssu.wordpress.com to participate in one of these activities, they are given the option to opt out of one portion of the Morinis J, Maguire J, Khovratovich M, et al. registry (e.g., blood collection) while still Paediatric obesity research in early childhood and consenting for others (e.g., questionnaires and the primary care setting: The TARGet Kids! anthropometric measurements). Research Network. Int J Environ Res Public Health. 2012 Apr;9(4):1343-54. Epub 2012 Results Apr 16. The registry is now following 4,287 children aged Birken CS, Maguire J, Mekky M, et al. Parental 0 to 5 years from seven primary care practices in factors associated with screen time in preschool Toronto, Canada. The participation rate for the children in primary-care practice: a TARGet registry is 49 percent of all eligible children in the Kids! study. Public Health Nutr. 2011 Apr 5:1-5. targeted practices (defined as children aged 0-5 years with a well-child appointment scheduled). Maguire JL, Birken CS, Jacobson S, et al. The primary reasons parents decline to participate Office-based intervention to reduce bottle use in the registry include lack of time to answer the among toddlers: TARGet Kids! Pragmatic, questionnaire, no legal guardian accompanying randomized trial. Pediatrics. 2010 the child, the need to discuss participation with Aug;126(2):e343-50. Epub 2010 Jul 12. spouse, and their child not feeling well that day.

207 Section II. Legal and Ethical Considerations for Registries

Case Example 16. Using a patient-centered activity levels can vary widely after surgery. study design to collect informed consent, FORCE-TJR collects data to track patient, maximize recruitment and retention, and provider, and site characteristics in order to provide meaningful clinical data evaluate their contributions to patient-reported Description Function and Outcomes and clinical outcomes of TJR over time. Research for Comparative TJR patients often have limited contact with their Effectiveness in Total Joint surgeons immediately after making the decision Replacement (FORCE-TJR) is a to have surgery, instead interacting with office prospective research registry and hospital staff to complete insurance or tracking and studying long-term anesthesia pre-operative paperwork. outcomes of elective total joint Administrative site staff often do not have the replacement (TJR) surgery, time or training to effectively inform patients funded by an Agency for about the risks and benefits of participating in Healthcare Research and Quality patient-centered studies. Further, clinical (AHRQ) award to the University information that may contain important data of Massachusetts Medical about adverse events resides in various, School. The registry seeks to disconnected points of care. Patients may return understand patient-reported and to the hospital in which TJR was performed or clinical outcomes by collecting they may present at another hospital, urgent care data on baseline patient center, or doctor’s office. Often these disparate attributes, procedure approach sites of care are not linked with the same and technology, inpatient electronic medical record, making data hospital stay, surgeon and challenging to collect. Collecting informed institutional characteristics, consent, patient reported outcomes, and other longitudinal patient pain and followup data from TJR patients can therefore be function, and post-procedure challenging and requires an innovative approach. complications and revisions. A Proposed Solution diverse patient cohort allows the generation of aggregate severity- Successful approaches to maximizing patient adjusted national and regional participation in research are based in creating a data against which participating relationship with each patient and minimizing the surgeons can compare their own burden on site staff. Patients who schedule a TJR practice. are asked by administrative staff at the participating site to sign a short study contact Sponsor Agency for Healthcare Research form, giving permission for registry staff to and Quality contact them. Site staff give the patient an Year Started 2011 informational packet and fax the signed contact Year Ended Ongoing forms to the registry. To collect informed consent, registry research staff contact patients at their No. of Sites 36 convenience via telephone to review the study No. of Patients 9,000 as of January 2013; procedures, informed consent form, and medical 30,000 projected release forms in the informational packet. At this point, patients have the opportunity to ask Challenge questions of registry staff and discuss with them Total joint replacement (TJR) is a common any concerns, facilitating a deep understanding of procedure, with more than 700,000 primary hip the registry and their role in its success. Patients and knee replacements performed in the United return the signed informed consent and medical States each year. Although TJR can result in release forms to registry staff via U.S. mail. significant pain relief, physical function and

208 Chapter 8. Informed Consent for Registries

Case Example 16. Using a patient-centered reported outcomes and medical records data, thus study design to collect informed consent, enabling more precise severity adjustment than maximize recruitment and retention, and relying on administrative data. Sites report high provide meaningful clinical data (continued) satisfaction with the model, contributing to an 80 Proposed Solution (continued) percent overall patient recruitment ratio in the registry. Collecting clinical data that does not reside in a single medical record also relies upon patient Key Point engagement. Upon enrollment in the registry, Registries and other patient-centered research can patients are asked to authorize release of their benefit from a study design that engages patients medical records; at each contact following at enrollment, thereby increasing their surgery, patients are asked if they sought medical participation over the life of the study. For care since their last contact with the registry. If registries that require clinical data from patients registry staff determine the medical care could be who may not access all their care within one related to the TJR, the related medical records are system, an approach that follows the patient obtained and analyzed. across settings can be beneficial. Contacting Results patients at their convenience rather than in a health care setting can allow them more time to The model described above uses registry staff to have their questions answered, increasing patient enroll patients, obtain informed consent, and commitment. deliver longitudinal information and motivation, enhancing participant enrollment and For More Information commitment over the long term. This procedure http://www.force-tjr.org/FORCE-index.html facilitates the longitudinal collection of patient-

References for Chapter 8 7. Council of State and Territorial Epidemiologists. Public Health Practice vs. Research. May 24, 1. 21 CFR 50.3(c). 2004. http://www.vdh.virginia.gov/OFHS/policy/ documents/2012/irb/pdf/public%20Health%20 2. 46 CFR 102 (d) - (f). Practice%20versus%20Research.pdf. Accessed 3. 45 CFR 46.101(b). August 15, 2012. 4. Casarett D, Karlawish J, Andrews E, et al. 8. 46.101(b)(2) and (3). Bioethical Issues in Pharmacoepidemiologic 9. 46.101(b)(4). Research. In: Strom BL, ed. Pharmacoepidemiology. 4th ed. Sussex, England: 10. 45 CFR 164.501. John Wiley & Sons; 2005. p. 593. 11. Johnson N, Vermeulen L, Smith KM. A survey of 5. U.S. Department of Health & Human Services. academic medical centers to distinguish between Text Version of OHRP Decision Charts. http:// quality improvement and research activities. Qual www.hhs.gov/ohrp/policy/decisioncharttext.html. Manag Health Care. 2006 Oct-Dec;15(4):215-20. Accessed August 15, 2012. PMID: 17047495. 6. Centers for Disease Control and Prevention. 12. U.S. Department of Health and Human Services. Guidelines for Defining Public Health Research Office for Human Research Protections. Quality and Public Health Non-Research. October 4, Improvement Activities – FAQs. http://answers. 1999. http://www.cdc.gov/od/science/integrity/ hhs.gov/ohrp/categories/1569. Accessed August docs/defining-public-health-research-non- 15, 2012. research-1999.pdf. Accessed August 27, 2012.

209 Section II. Legal and Ethical Considerations for Registries

13. National Committee on Vital and Health Statistics. 22. Human Subjects Research Protections: Enhancing Recommendations on Privacy and Confidentiality, Protections for Research Subjects and Reducing 2006-2008. Section I: Privacy and Confidentiality Burden, Delay, and Ambiguity for Investigators, in the Nationwide Health Information Network. 76 Fed. Reg. 143 (proposed July 26, 2011). May 2009. http://www.ncvhs.hhs.gov/ http://www.gpo.gov/fdsys/pkg/FR-2011-07-26/ privacyreport0608.pdf. Accessed August 27, 2012. html/2011-18792.htm. Accessed August 15, 2012. 14. AF Westin. How the Public Sees Privacy and 23. Office for Human Research Protections. U.S. Health Research. February 2008. http://www.iom. Department of Health and Human Services. edu/~/media/Files/Activity%20Files/Quality/ Attachment A to January 24, 2011 SACHRP VSRT/S4_1Westin.pdf. Accessed August 15, Letter to the Secretary: FAQs, Terms and 2012. Recommendations on Informed Consent and Research Use of Biospecimens. http://www.hhs. 15. Rothstein MA. Currents in contemporary ethics. gov/ohrp/sachrp/20110124attachmentatosecletter. Improving privacy in research by eliminating html. Accessed August 15, 2012. informed consent? IOM Report misses the mark. J Law Med Ethics. 2009 Fall;37(3):507-12. 24. Greely HT. Breaking the stalemate: a prospective PMID: 19723261. PMCID: 3077820. regulatory framework for unforseen research uses of human tissue samples and health information. 16. Schneider S, Kerwin J, Robins C, et al. Consumer Wake Forest Law Rev. 1999 Fall;34(3):737-66. engagement in developing electronic health PMID: 12664910. information systems: Final report. (Prepared by Westat, Rockville, Maryland, under Contract No. 25. Wendler D. One-time general consent for research 233-02-0087). AHRQ Publication Number on biological samples: is it compatible with the 09-0081-EF. Rockville, MD: Agency for health insurance portability and accountability Healthcare Research and Quality. July 2009. act? Arch Intern Med. 2006 Jul 24;166(14): http://healthit.ahrq.gov/sites/default/files/docs/ 1449-52. PMID: 16864754. citation/09-0081-EF.pdf. Accessed August 15, 26. Homer N, Szelinger S, Redman M, et al. 2012. Resolving individuals contributing trace amounts 17. Institute of Medicine. Beyond the HIPAA Privacy of DNA to highly complex mixtures using Rule: Enhancing Privacy, Improving Health high-density SNP genotyping microarrays. Through Research. January 27, 2009. http://www. PLoS Genet. 2008 Aug;4(8):e1000167. iom.edu/Reports/2009/Beyond-the-HIPAA- PMID: 18769715. PMCID: 2516199. Privacy-Rule-Enhancing-Privacy-Improving- 27. Jacobs KB, Yeager M, Wacholder S, et al. A new Health-Through-Research.aspx. Accessed August statistic and its power to infer membership in a 15, 2012. genome-wide association study using genotype 18. Rodwin MA. Patient data: property, privacy & the frequencies. Nat Genet. 2009 Nov;41(11):1253-7. public interest. Am J Law Med. 2010;36(4): PMID: 19801980. PMCID: 2803072. 586-618. PMID: 21302847. 28. National Heart L, Blood Institute working g, 19. Care FH. Protecting Privacy in Health Research: Fabsitz RR, et al. Ethical and practical guidelines The Limits of Individual Choice. Calif Law Rev. for reporting genetic research results to study 2010 Dec;98:1765-804. participants: updated guidelines from a National Heart, Lung, and Blood Institute working group. 20. Willison DJ, Schwartz L, Abelson J, et al. Circ Cardiovasc Genet. 2010 Dec;3(6):574-80. Alternatives to project-specific consent for access PMID: 21156933. PMCID: 3090664. to personal information for health research: what is the opinion of the Canadian public? J Am Med 29. Barbara J. Evans. Appropriate Human-Subject Inform Assoc. 2007 Nov-Dec;14(6):706-12. Protections for Research use of Sentinel System PMID: 17712084. PMCID: 2213476. Data. FDA Sentinel System Meeting Series Issue Brief: Legal Issues in Active Medical Product 21. Miller FG. Research on medical records without Surveillance. March 2010. http://www.brookings. informed consent. J Law Med Ethics. 2008 edu/~/media/events/2010/3/08%20fda%20 Fall;36(3):560-6. PMID: 18840249. legal%20issues/panel%203%20issue%20brief.pdf. Accessed August 15, 2012.

210 Chapter 8. Informed Consent for Registries

30. 45 CFR 46.116(a); 21 CFR 50.25(a). 46. Pub. L. No. 110-85, 121 Stat. 823 (2007) (Codified in 21 U.S.C.). 31. Informed Consent Elements; Final rule. 76 Federal Register 2 (4 January 2011), pp. 256-270. http:// 47. 45 CFR 164.508. For additional information on www.gpo.gov/fdsys/pkg/FR-2011-01-04/ authorizations for research, see http:// pdf/2010-33193.pdf. Accessed August 15, 2012. privacyruleandresearch.nih.gov/authorization.asp. Accessed August 15, 2012. 32. 45 CFR 46.116(b); 21 CFR 50.25(b). 48. 45 CFR 164.512(i)(1)(ii). 33. 45 CFR 46.116(d). 49. 45 CFR 164.512(i)(1)(iii). 34. 21 CFR 50.23. 50. 45 CFR 164.514 (a)-(c). 35. U.S. Department of Health & Human Services. Informed Consent Requirements in Emergency 51. 45 CFR 164.514(e). Research. http://www.hhs.gov/ohrp/policy/ 52. 45 CFR 164.512(i)(2). hsdc97-01.html. Accessed July 18, 2013. 53. 45 CFR 164.532. 36. 21 CFR 50.24. 54. Mark Barnes & David Forster, SACHRP, 37. 45 CFR 46.117; 21 CFR 50.27. Subcommittee on Harmonization (SOH) Update: 38. 45 CFR 46.117(c). July 20, 2011 (2011). http://www.hhs.gov/ohrp/ sachrp/mtgings/mtg07-11/soh.pdf. Accessed 39. 45 CFR 46.117(b); 21 CFR 50.27(b)(2). August 15, 2012. 40. U.S. Department of Health and Human Services. 55. 404; 21 CFR 50.51. Office for Human Research Protections. Can an electronic signature be used to document consent 56. 405; 50.52. or parental permission? http://answers.hhs.gov/ 57. 406; 50.53. ohrp/questions/7260. Accessed August 15, 2012. 58. 407; 50.54. 41. Donawa M. New FDA guidance on electronic records and signatures. Med Device Technol. 2001 59. 45 CFR 46.116. Nov;12(9):32-5. PMID: 12938536. 60. 45 CFR 46.408(c). 42. Buchanan Ingersoll & Rooney PC. Electronic 61. 46.408(a); 50.55. Records and Signatures in Healthcare and the Interplay of E-Sign, HIPAA and UETA. 1-7, 1. 62. 408b. http://corporate.findlaw.com/business-operations/ 63. Tu JV, Willison DJ, Silver FL, et al. electronic-records-andsignatures-in-healthcare- Impracticability of informed consent in the and-the-interplay.html. Registry of the Canadian Stroke Network. N Engl 43. Paterick TJ, Paterick BB, Paterick TE. Expanding J Med. 2004 Apr 1;350(14):1414-21. Electronic Transmissions in the Practice of PMID: 15070791. Medicine and the Role of Electronic Informed 64. Thaler RH, Sunstein CR. Nudge: Improving Consent. J Patient Saf. 2008 Dec;4(4):217-20. Decisions About Health, Wealth, and Happiness. 44. Goldstein MM. Health information technology Yale: Yale University Press; 2008. and the idea of informed consent. J Law Med 65. Littenberg B, MacLean CD. Passive consent for Ethics. 2010 Spring;38(1):27-35. clinical research in the age of HIPAA. J Gen PMID: 20446981. Intern Med. 2006 Mar;21(3):207-11. 45. U.S. Department of Health & Human Services. PMID: 16637821. PMCID: 1828090. Guidance on Withdrawal of Subjects from: Data 66. Marshall PA, Berg JW. Protecting communities in Retention and Other Related Issues. www.hhs.gov/ biomedical research. Am J Bioeth. 2006 May- ohrp/policy/subjectwithdrawal.html. Accessed Jun;6(3):28-30; discussion W46-8. July 18, 2013. PMID: 16754445.

211

Chapter 9. Protecting Data: Confidentiality and Legal Concerns of Providers, Manufacturers, and Health Plans

1. Background upon them by the public, credentialing bodies, regulatory agencies, payers, and the institutions in As the cost of care delivered in the United States which they practice.”10 Likewise, the American continues to grow at an unsustainable rate without Orthopaedic Association spearheads the Own the parallel improvements in the quality of care,1-8 Bone registry, designed to better coordinate patient health care policy experts and lawmakers are care among a patient’s providers, close the gaps paying increasing attention to initiatives that associated with physician treatment measure and publicly report information about the recommendations, and alter patient and physician performance of physicians, hospitals, and other behaviors to reduce future incidence of bone health care providers and about the services and fractures due to osteoporosis.11 Quality procedures they are delivering. They believe this is improvement and clinical research registries are an important step toward improving health care also organized by the American College of quality and controlling costs. For example, Rheumatology,12 the American College of advancing quality improvement through greater Radiology,13 the Society for Thoracic Surgeons,14 access to and use of health information is a and the National Cardiovascular Data Registry.15 specific goal of the Patient Protection and Other registries, such as the American Joint Affordable Care Act,9 which includes a number of Replacement Registry, are endorsed by surgeons provisions to incentivize quality measurement, but use a multi-stakeholder funding and improvement, and reporting and to enable more sponsorship model. Manufacturers and health informed decisionmaking by consumers and other plans also often contribute data to or sponsor stakeholders. registries for quality improvement and clinical research, including identifying care delivery trends Critical to the success of these initiatives is the and determining whether or not a particular availability and accessibility of relevant service, procedure, medical device, or administrative and clinical data. Data registries (or pharmaceutical achieves the desired effect. repositories) are often used to collect, process, maintain, and release relevant data for these Administrative and clinical information submitted purposes. For example, many professional by or about providers, medical device and associations and societies organized around pharmaceutical manufacturers, and health plans provider specialties or specific diseases and and included in registries for research, quality conditions administer their own registries. measurement and improvement activities, and Typically, these registries are used for a variety of patient safety initiatives often includes sensitive patient safety and quality improvement activities, patient, provider, and/or manufacturer or health including: matching patients with researchers, plan–identifiable information. Release of this tracking the course of patients’ care, tracking and information in an identifiable or even identifying trends in medical errors or other nonidentifiable manner may compromise the patient safety issues, and tracking outcomes privacy of individual patients and providers and related to specific diseases or conditions or the compromise sensitive financial, commercial, or effectiveness of specific treatments used to treat proprietary manufacturer or health plan (e.g., them. For example, the American College of Chest benefit design, reimbursement) information. As Physicians (ACCP) directs the ACCP Quality more and more registries are developed and used Improvement Registry, Evaluation and Education, for a variety of research (including comparative or AQuIRE, which is intended to “assist the chest effectiveness research), quality improvement, and physician with meeting increasing demands placed patient safety programs, more and more

213 Section II. Legal and Ethical Considerations for Registries

information about patient safety, quality of patient included as a direct or indirect subject of) relevant care, performance, and other details about information to support research, quality providers, medical devices, pharmaceuticals, and improvement, and patient safety initiatives health plans becomes available. This information conducted through registries. For example, is incredibly useful because it helps providers commentary in the 2006 Journal of the American better understand and improve the care they are Medical Association reported that “wariness about delivering, helps manufacturers refine and improve liability exposure in the medical community may the devices and pharmaceuticals they are stymie public and private efforts.”17 developing, and helps patients make more The 1999 Institute of Medicine report, To Err is informed choices about their providers and Human, explicitly identified this “wariness” as a treatment options. However, this information is significant issue that hampers voluntary reporting also desirable for use in litigation or other judicial and collaborative efforts, including the free or administrative proceedings to demonstrate that a exchange of information, to identify medical certain level of care was adhered to or not, or that errors and prevent their repetition. To address this a certain device or pharmaceutical works in a issue, the report recommended that Congress pass particular way. “legislation to extend peer review protections to Considerable attention has been directed towards data related to patient safety and quality ensuring the privacy and confidentiality of improvement that are collected and analyzed by individually identifiable patient health information health care organizations for internal use or shared maintained in registries, particularly in regards to with others solely for purposes of improving safety the requirements of the Health Insurance and quality.”18 To date, however, no comprehensive Portability and Accountability Act (HIPAA) Federal legislation has been passed. Thus, Privacy Rule.16 As such, the privacy and providers, manufacturers, and health plans must confidentiality of individually identifiable patient look to a variety of Federal and State laws that may health information is well established and offer protection from disclosure of information recognized by Federal and State agencies, courts pursuant to a discovery request or other judicial or of law, and others. (See Chapter 7.) administrative proceedings. Significantly less attention has been directed towards the privacy and potentially proprietary 2. Relevant Laws and nature of information about providers, medical Regulations: Variety of device and pharmaceutical manufacturers, and health plans. Often providers, manufacturers, and Sources, But Limited Protection health plans are the best source of information While no general Federal statutory privilege exists about the care they deliver and/or their products to to protect information held in a registry submitted support the effectiveness of a registry, whether it is by or relating to providers, manufacturers, or used for research, quality improvement, patient health plans, a number of Federal laws may safety initiatives, or other related activities. Even provide protection from discovery or disclosure in when they do not directly give a registry judicial or administrative proceedings. In addition, information about the care they deliver and/or their most States have specific peer review or quality products, they may be included in information assurance laws that may provide additional provided by other sources. (For example, a protection as well, but again in limited provider may submit information about Company circumstances. This chapter will primarily focus Y’s medical device related to a procedure on available Federal evidentiary protections, but performed on a patient.) Numerous policymakers will also address State-specific protections. It and researchers have noted the “chilling effect” concludes with an overview of mechanisms that that the lack of protection for the information may be used to protect information included in a provided may have on the willingness of providers, registry during judicial or administrative manufacturers, and health plans to provide (or be proceedings. While registries may collect

214 Chapter 9. Protecting Data: Confidentiality and Legal Concerns of Providers, Manufacturers, and Health Plans

information from both within the United States care institution are gathering identifiable medical and internationally, treatment of registries by error information as part of AHRQ-supported international laws is outside the scope of this grant or contract research, and it is conveyed chapter. outside the institution (e.g., for analysis in an AHRQ-supported central databank), even if the 2.1 Federal Laws reporters lost their protection against being 2.1.1 AHRQ’s Confidentiality Statute subpoenaed to testify under State law, the Federal statute would cover and protect the identifiable All identifiable research data obtained by the information they acquired pursuant to AHRQ’s Agency for Health Research and Quality (AHRQ) statutory research authority.”20 While this is protected by the agency’s confidentiality memorandum is not binding on any court of law 19 statute. The statute requires that data collected by and has yet to be introduced in a legal challenge, it AHRQ-sponsored entities that identify individuals clearly establishes AHRQ’s protective position as or establishments be used only for the purposes for it relates to any information collected under the which the data are supplied. Any effort to auspices of an AHRQ-supported project. determine the identity of a person in an AHRQ Registries participating in AHRQ-sponsored database, or to use the information for any purpose activities would certainly be able to avail other than for research, analysis, and aggregate themselves of this protection and, given Merewitz’s statistical reporting, violates the AHRQ comments, this protection may even extend to confidentiality statute. Recipients of a data set are non-AHRQ activities in which an AHRQ- also prohibited from releasing, disclosing, sponsored entity holds the data as a repository or publishing, or presenting any individually intermediary. identifying information. Specifically, the statute provides: Importantly, this protection is limited to AHRQ- sponsored registries. Therefore, registries No information, if an establishment or person maintained by professional associations or other supplying the information or described in it is organizations that are not sponsored by or identifiable, obtained in the course of activities otherwise participating in an AHRQ-sponsored undertaken or supported under this subchapter project would not benefit from the protections the may be used for any purpose other than the AHRQ confidentiality statute affords. purpose for which it was supplied unless such establishment or person has consented (as 2.1.2 HHS Certificate of Confidentiality determined under regulations of the Director) The U.S. Department of Health and Human to its use for such other purpose. Such Services (HHS) may issue a “Certificate of information may not be published or released Confidentiality” for any research project that in other form if the person who supplied the collects personally identifiable, sensitive information or who is described in it is information and has been approved by an identifiable unless such person has consented institutional review board. A Certificate protects (as determined under regulations of the an investigator, and others who have access to Director) to its publication or release in other research records, from being required to disclose form.19 identifying information on research participants in Concerns have been raised that this protection may any Federal or State judicial, administrative, or be vulnerable if the information is disclosed to an legislative proceeding. The Certificate may be used outside entity such as a registry. However, AHRQ for biomedical, behavioral, clinical, or other types has interpreted this provision to protect all AHRQ- of research. funded research from discovery requests, including In the research arena, the National Institutes of discovery requests in the course of litigation. A Health (NIH) is the most common source for memorandum from senior AHRQ attorney Susan Certificates of Confidentiality. The NIH considers Merewitz noted that “if individuals inside a health research to be sensitive if disclosing the

215 Section II. Legal and Ethical Considerations for Registries

information could have adverse consequences for Certificate of Confidentiality are limited and do subjects or damage their financial standing, not provide meaningful opportunities for registries employability, insurability, or reputation. that are not engaged in research and that have not According to NIH, examples of studies that may applied for and been granted a Certificate. be considered sensitive include those collecting 2.1.3 The Patient Safety and Quality genetic information; information on subjects’ Improvement Act of 2005 psychological well-being; information on sexually transmitted diseases or on subjects’ sexual The Patient Safety and Quality Improvement Act attitudes, preferences or practices; and information of 2005 (PSQIA)24 creates a Federal privilege on substance abuse or other illegal conduct; as from discovery in connection with Federal or State well as studies where subjects may be involved in judicial or administrative proceedings for certain litigation related to exposures under study (i.e., information identified as patient safety work breast implants, environmental or occupational product. To claim the privilege, providers must exposures).21 create the patient safety work product by collecting event information and reporting it to a formally The specific statutory language provides that: recognized patient safety organization (PSO) for The Secretary [of the U.S. Department of aggregation and analysis. The term “patient safety Health and Human Services] may authorize work product” encompasses any data, reports, persons engaged in biomedical, behavioral, records, memoranda, analyses, or written or oral clinical, or other research (including research statements that meet one of two criteria: (1) the on mental health, including research on the use materials “could improve patient safety, health and effect of alcohol and other psychoactive care quality, or health care outcomes” and are drugs) to protect the privacy of individuals gathered by a provider to be reported and are who are the subject of such research by reported to a PSO or are developed by a PSO to withholding from all persons not connected conduct patient safety activities; or (2) the with the conduct of such research the names or materials “identify or constitute the deliberations other identifying characteristics of such or analysis of, or fact of reporting to, a patient individuals. Persons so authorized to protect safety evaluation system.” the privacy of such individuals may not be Materials not gathered to be reported to the PSO compelled in any Federal, State, or local civil, and not actually transmitted to a PSO would not criminal, administrative, legislative, or other 22 qualify for a privilege. The privilege specifically proceedings to identify such individuals. does not apply to medical records, billing and The application of Certificates of Confidentiality discharge information, or other records kept to registries has four inherent shortcomings.23 outside of a patient safety evaluation system. First, the protections apply only to the identity of Furthermore, providers must comply with any research subjects, or to data that would allow the State laws that require reporting of patient safety possible identification of such individuals. Thus, information and may not use PSO activities to de-identified patient safety data is still potentially shield them from reporting. Thus, if a patient discoverable. Second, there are questions as to safety investigation references medical records, the whether a Certificate applies to patients who records themselves do not become part of the work presumably have not consented to becoming product eligible for protection. research subjects. Third, individual patients may be Documents created, maintained, or developed able to waive the Certificate protections as to their separately from a patient safety evaluation system own information, which they presumably would do are excluded from the definition of patient safety if they were plaintiffs in a malpractice lawsuit. work product. Thus, individual patient medical Fourth, the protections apply only to research records, billing and discharge information, and any information for which a Certificate of original patient or provider records are not Confidentiality has been applied for and has been considered work product and are thus not granted. Therefore, the protections afforded by a

216 Chapter 9. Protecting Data: Confidentiality and Legal Concerns of Providers, Manufacturers, and Health Plans

privileged. In addition, information collected to accreditation bodies responsible for licensing or comply with external reporting requirements is not certifying providers or practitioners.25 work product. Furthermore, the statute explicitly states that “no The preamble to the final regulations (44 FR patient record in the possession of ” a QIO may be 70,732) implementing the confidentiality subject to subpoena or discovery proceedings in a protections of the PSQIA identifies several civil action.26 Additionally, no document or other examples of information that must be reported and information produced by a QIO in connection with that does not merit protection as work product. its deliberations may be subject to subpoena or These examples of required reportinginclude State discovery in any administrative or civil incident reporting, adverse drug event information proceeding. However, a QIO is required to provide, reporting, records for compliance with health upon the request of a practitioner or other person oversight agency requirements, reporting physician adversely affected by such deliberations, a disciplinary actions to the National Practitioner summary of the QIO’s findings and conclusions. Data Bank, and disclosures required under Additionally, QIO regulations state that quality Medicare’s conditions of participation. Thus, a review study information with a patient identifier significant amount of data remains outside the is not subject to subpoena or discovery in a civil patient safety work product definition. This action, including administrative, judicial, or includes registry data that is not maintained by a arbitration proceedings.27 This restriction, PSO and used for specific patient safety activities however, does not apply to HHS administrative or is not identifiable. Therefore, the PSO statute subpoenas issued in the course of an audit or and regulations provide no protection for registries investigation of HHS programs, in the course of acting outside the protected scope of the PSO administrative hearings held under the Social arena. Security Act, or to disclosures to the U.S. 2.1.4 Quality Improvement Organization Statute Government Accountability Office as necessary to and Regulations carry out its statutory responsibilities. Quality Improvement Organizations (QIOs) are Similar to the PSQIA, the QIO statute and responsible for improving the effectiveness, regulations provide protection only to information efficiency, economy, and quality of services that has been collected by a QIO under contract delivered to Medicare beneficiaries. The Centers with CMS to perform specific statutory functions. for Medicare & Medicaid Services (CMS) To the extent a QIO is the owner and operator of a contracts with one private, generally not-for-profit registry used to perform required functions, the organization in every State, as well as the District information included in the registry would be of Columbia, Puerto Rico, and the U.S. Virgin protected. However, this does not apply to the vast Islands, to serve as that jurisdiction’s QIO. QIO majority of registries currently in existence today. employees, consisting primarily of doctors and 2.1.5 HIPAA Privacy Rule other health care professionals, are instructed to review medical care and assist beneficiaries with The HIPAA Privacy Rule protects the privacy of quality of care issues and complaints, as well as to individually identifiable health information.28 The implement improvements to the quality of Privacy Rule applies to “covered entities,” which providers’ care. include health plans, health care clearinghouses, and health care providers who conduct certain The QIO statute states that any data or information electronic health care transactions. Some of the acquired by a QIO in its course of duties must be Privacy Rule’s provisions also apply to business kept confidential and may not be disclosed to any associates of covered entities (e.g., contractors person, except as the information assists Federal performing specific functions on their behalf).29 and State agencies responsible for investigating The purpose of the Rule is to protect “individually fraud, abuse, and risks to the public health, or identifiable health information” held or transmitted assists appropriate State agencies and national

217 Section II. Legal and Ethical Considerations for Registries

by a covered entity or its business associate, in any information for any purpose other than the form or media, whether electronic, paper, or oral. litigation or proceeding for which the records are “Individually identifiable health information” is requested; and (2) requires the return to the information, including demographic data, that covered entity or destruction of the protected relates to an individual’s (1) past, present, or future health information (including all copies made) at physical or mental health condition; (2) receipt of the end of the litigation or proceeding. Satisfactory health care; or (3) past, present, or future payment assurances of reasonable efforts to secure a for health care. The information also must either qualified protective order are a statement and directly identify the individual or be able to documentation that the parties to the dispute have reasonably lead to identification of the individual. agreed to a protective order and that it has been Common identifiers include an individual’s name, submitted to the court or administrative tribunal address, birth date, and Social Security number.29 with jurisdiction, or that the party seeking the The Privacy Rule refers to this information as protected health information has requested a “protected health information” (PHI). qualified protective order from such court or The Privacy Rule permits the disclosure of PHI in tribunal. the course of any judicial or administrative Importantly, the protections of the HIPAA Privacy proceeding in response to an order of a court or Rule will apply only if a registry is considered a administrative tribunal.30 Absent a court order, a covered entity or the business associate of a covered entity also may respond to a subpoena or covered entity. A registry may be considered a discovery request from a party to the proceeding if covered entity if the registry is maintained and the covered entity obtains either: (1) satisfactory administered by a HIPAA-covered health care assurances that reasonable efforts have been made provider. More commonly, the registry will be to give the individual whose information has been acting as the business associate of a covered entity requested notice of the request; or (2) satisfactory (e.g., collecting and processing PHI on behalf of assurances that the party seeking such information provider(s) and/or health plan(s)). Even if the has made reasonable efforts to secure a qualified registry is considered a covered entity or business protective order that will guard the confidentiality associate, the HIPAA requirements that apply to of the information. disclosures pursuant to a court order, subpoena, or In meeting the first test, a covered entity is discovery request only apply to PHI. To the extent considered to have received satisfactory assurances the requested information does not include PHI from the party seeking the information if the (e.g., the information is considered to be de- covered entity receives a written statement and identified according to the requirements of the documentation that the party has made a good- Privacy Rule and thus does not contain identifiable faith effort (such as by sending a notice to the health information about individuals), HIPAA does individual’s last known address) to provide written not protect information about providers, notice to the individual whose information is the manufacturers, or any other entities. Therefore, in subject of the request, that the written notice the case of providers, manufacturers, and health included sufficient information about the plans seeking protection for information specific to proceeding to permit the individual to raise an them or their products but not part of PHI, HIPAA objection, and that the time for the individual to does not shield them from discovery requests in raise objections to the court or administrative litigation or any other court proceedings. tribunal has elapsed and no objections were filed, 2.1.6 Privacy Act of 1974 or any objections the individual filed have been The Privacy Act of 197431 protects information resolved. about individuals, such as patients and providers, A “qualified protective order” means an order of a held or collected by the Federal Government that court or of an administrative tribunal or a can be retrieved by personal identifiers such as stipulation that: (1) prohibits the parties from name, Social Security number, or other identifying using or disclosing the protected health number or symbol. The Privacy Act authorizes a

218 Chapter 9. Protecting Data: Confidentiality and Legal Concerns of Providers, Manufacturers, and Health Plans

Federal agency to release individually identifiable information—in order of their importance—are information to identified persons or to their Exemptions 4, 6, and 3. designees with written consent or pursuant to 1 of Exemption 4 protects “trade secrets and 12 exemptions for disclosure. These exemptions commercial or financial information obtained from include disclosure to Federal agency employees, a person and privileged or confidential.”36 the Census Bureau, the National Archives and Importantly, however, it is not a mandatory bar to Records Administration, other government entities disclosure, but rather limits an agency’s obligation for civil and criminal law enforcement purposes, to disclose specified information. “Trade secrets” the Comptroller General, Congress or its 32 are defined as commercially valuable plans or committees, and a consumer reporting agency. formulas for producing trade commodities to Additional exemptions include disclosures for which has been invested substantial effort or statistical research, disclosures required by innovation.37 In the case of registries that contain Freedom of Information Act, disclosures in information supplied by or about providers, response to emergency circumstances, and medical device or pharmaceutical manufacturers, importantly for purposes of this chapter, and health plans, it is likely that only the disclosures pursuant to a court order. pharmaceutical and medical device manufacturers Unless the Federal Government maintains the and health plans would have information that registry, the Privacy Act of 1974 offers no could be considered “trade secrets.” Furthermore, protection from discovery for litigation or related information that manufacturers contribute to a court proceedings. Furthermore, even if the registry is more likely to be considered Federal Government maintains the registry, the “commercial” or “financial.” For example, in Privacy Act specifically allows for the release of Public Citizen Health Research Group v. Food & identifiable information about individuals without Drug Administration,38 the court found that their written consent pursuant to a court order. “because documentation of the health and safety This could include information not only about experience of their products will be instrumental individual patients, but also individual providers in gaining market approval for their products it (e.g., individual practitioners). seems clear that the manufacturers … have a 2.1.7 Freedom of Information Act commercial interest in the requested information.”39 Enacted by Congress in 1966, and expanded in 1996 to cover electronic records,33 the U.S. Exemption 6 states that Federal agencies can Freedom of Information Act (FOIA)34 generally withhold from disclosure information about provides that any person has the right to obtain individuals in “personnel and medical files and access to information contained in the records of similar files” when the disclosure of such information “would constitute a clearly Federal agencies, unless FOIA specifically protects 40 the information from disclosure. With a goal of unwarranted invasion of personal privacy.” In ensuring an informed citizenry, capable of holding order to warrant protection under Exemption 6, the the government accountable, FOIA effectively information at issue must first meet the threshold establishes a statutory right of public access to requirement of falling into one of three executive branch information, requiring that categories—personnel files, medical files, and virtually every record held by a Federal agency be similar files. The Supreme Court found that provided to individuals upon request.35 Congress intended these categories to be interpreted broadly and to protect information that Information that is subject to FOIA is likely to be 41 disclosable pursuant to a discovery request or other “applies to a particular individual.” Once it has court proceeding. However, FOIA does have been established that the information meets this limited exemptions and exclusions to the broad threshold, the focus shifts to whether the disclosure disclosure requirements. The most relevant of such information would be an unwarranted exemptions for purposes of protection of registry invasion of privacy. This requires balancing the public’s right to disclosure of the information

219 Section II. Legal and Ethical Considerations for Registries

against the individual’s right to privacy. After or refers to particular types of matters to be determining that a protectable privacy interest withheld.”46 An example of a statute that may exists, the public’s interest in disclosure of the prevent disclosure or discovery of information information will be weighed against the contained in a registry under FOIA Exemption 3 individual’s privacy interest in not disclosing the would be the Patient Safety Quality Improvement information. Act of 2005 (discussed above), if the information The landmark Supreme Court decision in United met the PSQIA requirements. States Department of Justice v. Reporters 2.1.8 Federal Trade Secrets Act 42 Committee for Freedom of the Press governs how The Federal Trade Secrets Act47 imposes fines or privacy interests under Exemption 6 are imprisonment on any Federal employee who determined and balanced with public interest in the discloses any information that relates to trade information. First, the Court clarified that a secrets, processes, operations, style of work, or substantial privacy interest may exist in apparatus, or to the identity, confidential statistical information that has already been released to the data, amount or source of any income, profits, public at some point. Second, the Court held that losses or expenditures of any person or the identity of the individual or party requesting corporation. The Act applies only to public the information may not be taken into disclosures, and does not reach internal agency use consideration when determining if information of the data. should be released and “has no bearing on the merits of his or her FOIA request.”43 When There is no private right of action under the considering the public interest in disclosing the statute; however, the Administrative Procedure Act information, the Court ruled that the determination may create a right of action to prevent a violation should be based on the nature of the requested of the Trade Secrets Act or review a decision to information and its relationship to the public disclose information.48 Similar to Exemption 4 interest generally, and not solely the purpose for under FOIA, the Federal Trade Secrets Act may which the request is made. Finally, the Court protect proprietary information, however, only to narrowed the scope of the public interest to the the extent that is held by the Federal Government kind of interest to information that will “shed light and disclosed publicly (e.g., the Act may not reach on an agency’s performance of its statutory information disclosed pursuant to a protection duties.”44 order as part of a discovery proceeding). It is important to note, however, that Exemption 6 2.1.9 Federal Rules of Evidence and Civil protects only information that identifies the Procedure individual in question. Thus, while patient records Federal legal rules of evidence and civil procedure included in a registry are likely to be protected, if may place limits on what information may be identifying information is removed, the discoverable or otherwise used in a court information is no longer protected under proceeding. For example, Rule 401 of the Federal Exemption 6. The most common types of Rules of Evidence defines “relevant evidence” as information protected under Exemption 6 are age, evidence having any tendency to make the home address, Social Security number, medical existence of any fact that is of consequence to the information about individuals participating in determination of the action more probable or less clinical research trials, claims files, and other probable than it would be without the evidence.”49 personal information held by CMS.45 Rule 403 narrows the scope of Rule 401 stating Exemption 3 protects information if it is “although relevant, evidence may be excluded if its “specifically exempted from disclosure by statute, probative value is substantially outweighed by the provided that such statute (A) requires that the danger of unfair prejudice, confusion of the issues, matters be withheld from the public in such a or misleading the jury…”50 Often of the most manner as to leave no discretion on the issue, or relevance to information contained in registries is (B) establishes particular criteria for withholding the prohibition in Rule 404 against “evidence of

220 Chapter 9. Protecting Data: Confidentiality and Legal Concerns of Providers, Manufacturers, and Health Plans

other … acts … to prove the character of a person outweighed by the compelling social interest in in order to show action in conformity therewith.”51 preventing harm to the Registry and the vital work This may apply in instances where information is it conducts.”56 The court, however, did allow sought from a registry to show evidence of similar discovery of information contained in the registry medical outcomes. related to the plaintiffs. Squibb appealed, and in 57 Turning to Civil Procedure, Rule 26(b)(2)(C)(iii) Deitchman v. E.R. Squibb & Sons, the appellate of the Federal Rules of Civil Procedure provides court noted that the privilege was not “absolute” that a court may limit discovery if “the burden or and remanded the case back to the lower court to expense of the proposed discovery outweighs its determine the best way to provide information from the registry while preserving likely benefit, considering … the importance of the 58 52 confidentiality. In a secondhand-smoke case, discovery in resolving the issues.” Rule 26(c) 59 also allows parties from whom discovery is sought Wolpin v. Phillip Morris, Inc., the court ordered to move for a protective order.53 Rule 45(c) disclosure of data from a statewide tumor registry protects individuals from unduly burdensome or from a study by the University of South California expensive subpoenas. Specifically, 45(c) states, “a and California Department of Health, despite party or attorney responsible for issuing and objections that the data was protected by State and serving a subpoena must take reasonable steps to Federal privacy laws. The court held that the need avoid imposing undue burden or expense on a for the information in the lawsuit outweighed the person subject to the subpoena. The issuing court confidentiality interest, but did require patient must enforce this duty and impose an appropriate names to be removed prior to release. sanction—which may include lost earnings and 2.1.10 Patient Protection and Affordable Care reasonable attorney’s fees—on a party or attorney Act: Release of Medicare Claims Data for who fails to comply.”54 It is left to the discretion of Provider Performance Measurement and the court in each case to determine whether the Reporting facts and circumstances merit quashing a specific Section 10332 of the Affordable Care Act requires subpoena. the Secretary to make Medicare claims data While these Federal rules of evidence and civil available to qualified entities for the evaluation of procedure may offer protection against discovery provider performance on measures of quality, of registry information in certain situations, their efficiency, effectiveness, and resource use.60 The application is left entirely to the discretion of the data are standardized extracts of claims data under particular court in which the case is heard. Thus, Medicare Parts A, B, and D, for items and services the case law is mixed, with some courts allowing furnished for one or more specified geographic discovery and others not, depending on their areas and time periods requested by a qualified application of a balancing test of the need for entity. The qualified entities will be required to pay confidentiality versus hardship to the party seeking a fee to obtain the data and must submit to the discovery. For example, in Andrews v. Eli Lilly & Secretary of HHS a description of the Co., Inc., E.R. Squibb & Sons and Rexall Drug methodologies that will be used to evaluate the Company,55 Squibb sought production of data performance of providers and suppliers. from a University of Chicago registry that included All subsequent reports a qualifying entity information about a disease related to a products publishes must include an understandable liability action against Squibb. The University of description of the measures, risk adjustment Chicago claimed that the contents of the registry methods, physician attribution methods, other were privileged and confidential. In balancing the applicable methods, data specifications and privacy interests of the registry against the need limitations, and the sponsors, so that consumers, for the information, the court stated, “the balance 56 providers of services and suppliers, health plans, … tips in favor” of the registry. “Squibb’s need researchers, and other stakeholders can assess such for the information is speculative and uncertain. Its reports. In addition, the reports must be made essentially private interest in defending itself is available to any provider or supplier identified in

221 Section II. Legal and Ethical Considerations for Registries

the report, with an opportunity to appeal and harbors is sparse, with most cases affording correct any errors. Finally, the reports may include immunity for health care professionals who report information on a provider or supplier only in an sexually transmitted diseases discovered in minors aggregate form as the Secretary determines in potential child abuse cases.65 appropriate. 2.2.2 State Peer Review and Quality Assurance The Secretary may not make claims data available Laws to a qualified entity unless the entity agrees to Presently, all 50 States have enacted statutes to release the information on the evaluation of protect the confidentiality of the peer review performance of providers of services and process. Most statutes offer a blanket protection suppliers. Section 10332 requires that data for all accounts, records, and conclusions of the released to a qualified entity shall not be subject to review process from being introduced into discovery or admission as evidence in judicial or evidence during any court proceeding.66 Without administrative proceedings without consent of the 61 such protection, providers and hospitals may be applicable provider or supplier. less inclined to truthfully monitor their peers, While limited in application to Medicare claims evaluate the quality of care that is provided to data provided to qualified entities, this provision is patients, or adequately prevent or correct for a significant recognition by Congress of concerns adverse events. related to the “chilling effect” the fear of A few States also have passed legislation disclosure has on provider, supplier (including specifically protecting information collected for manufacturer), and health plan participation in quality improvement and other related purposes. quality measurement, improvement, and reporting These “safe harbor” laws protect a broader set of programs. By shielding this information from information beyond the traditional peer review discovery or admission as evidence without process, including collection by organizations consent, Congress has explicitly protected and outside the scope of an internal peer review board incentivized the activities of qualified entities, or committee. For example, in Minnesota, including the development of registries to support information relating to patient care a nonprofit their performance measurement and reporting organization collects for purposes of “evaluating efforts. and improving the quality of health care” or “reviewing the safety, quality or cost of health care 2.2 State Laws services” provided to health plan enrollees “shall not be disclosed to anyone … and shall not be 2.2.1 State Surveillance Laws subject to subpoena or discovery.”67 The Virginia Patient Safety Act protects the collection of patient To encourage practitioner participation in State safety data by “statewide or local associations” surveillance registries, many States have passed representing licensed health care providers. It legislation providing immunity from civil and treats the information collected as “privileged criminal penalties that may arise in conjunction communications which may not be disclosed or with such reports. Most States protect providers obtained by legal discovery proceedings unless a from any liability that may result from a disease circuit court, after a hearing and for good cause report unless the provider acted with some level of arising from extraordinary circumstances being negligence or malicious intent.62 Fewer States shown, orders the disclosure of such proceedings, provide complete immunity for reporting disease minutes, records, reports, or communications.”68 cases to a registry, with no distinction made for Similarly, the Illinois Medical Studies Act protects negligent or intentionally malicious reports.63 all information “used in the course of internal Additionally, other States provide immunity only quality control or of medical study for the purpose for certain causes of action related to the of reducing morbidity or mortality, or for information reported, or protect from civil liability improving patient care or increasing organ and only.64 However, case law implicating such State tissue donation.” Such information is “privileged,

222 Chapter 9. Protecting Data: Confidentiality and Legal Concerns of Providers, Manufacturers, and Health Plans

strictly confidential, and shall be used only for data production, including costs for redaction medical research, increasing organ and tissue (particularly where patient identifiable information donation, [or] the evaluation and improvement of is involved), and the costs of legal representation. 69 quality care.” It is not “admissible as evidence, Registries and their participants can take several nor discoverable in any action of any kind in any steps to reduce their vulnerability to disclosure court or before any tribunal, board, agency or 69 requests. As described above, several Federal person.” programs protect information collected for specific However, it is important to note that the peer patient safety and quality improvement purposes. review privilege and other State-based protections (See Case Example 18.) If registries qualify for or are often not recognized in Federal cases outside are funded through these programs, the the jurisdiction of State law. Federal courts have information collected and maintained would been resistant to the establishment of a Federal automatically be entitled to protection from peer review or other privilege, and often discovery or other judicial or administrative subordinate the State peer review privileges in proceedings. While not always possible or practical favor of other interests. For instance, the 11th given their goals or priorities, registries should Circuit Court of Appeals recently declined to consider whether participation in any of these recognize such a privilege during a Federal civil programs would be appropriate. rights discrimination case, ordering the discovery 70 To the extent participation in one of these Federal of peer review documents. Similar outcomes programs is not possible, registries and their have been reached in many other Federal courts, participants should consider forming in a State that including cases deciding Federal antitrust71 and 72 provides broader protection, beyond the peer wrongful termination claims. review process, for information collected (e.g., Virginia, Minnesota, or Illinois). Furthermore, 2.3 Practical Considerations registries and their participants should clearly As described above, Federal and State law articulate their roles and responsibilities, including currently do not provide any consistent or how discovery or other requests will be handled. comprehensive protection from disclosure, Registries should develop specific policies and pursuant to discovery or other judicial or procedures to guide their response to such requests administrative proceedings, of information and should ensure that all participants are familiar submitted to registries by (or related to) providers, with the policies and procedures. (See Case medical device or pharmaceutical manufacturers, Examples 17 and 19.) For example, registries or health plans. This leaves registries that are might stipulate that they will direct all disclosure operating outside of the government-sponsored requests to the original source of the information programs described above, their participants and where possible. Where information held within a subjects, vulnerable to discovery requests ranging registry has been aggregated and analyzed such from preliminary fact-finding requests to court that it is significantly modified from its original orders. As the Institute of Medicine noted, this can state, the registry will notify the original data have a “chilling effect” on willingness of providers sources prior to compliance with any discovery to participate. The same is also true for request and give them the opportunity to object. manufacturers and health plans that similarly may be both a source of information as well as the In the event a registry is compelled to release subject of information included in a registry. information pursuant to a court order or other Beyond concern for the privacy and confidentiality judicial or administrative order, registries may of the information, the costs and burden associated request that certain information be redacted or a with discovery or other requests can be substantial, protective order issued. A court protective order as the litigation process often takes months or can stipulate who can see the information, who has years to unfold. These costs may include not only access to the information, and how the data should costs related to challenging the request, but also be returned or destroyed. Similarly, a registry may request that the court “seal” information so that it

223 Section II. Legal and Ethical Considerations for Registries

is not made public. These types of actions have to support efforts to generate comparative historically been used to protect patient- effectiveness research, begins its work. Given this identifiable information held in registries; however, heightened interest in registries as data sources, they may be similarly applied to confidential or the issue of protection of registry data from proprietary information related to providers, disclosure pursuant to a discovery request or other manufacturers, or health plans. judicial or administrative proceedings will be increasingly important. Registry sponsors may be 3. Summary able to address concerns from potential participants about data protection by considering As more attention is focused on the development these issues during the registry development stage. and implementation of quality improvement In particular, providers, manufacturers, and health activities, including those tied to new payment plans that are developing registries or considering models, availability and accessibility of underlying participation in registries should look to the clinical and administrative data that registries can Federal and State laws described here that may provide to support these efforts will be offer protection and should consider the practical increasingly important. This emphasis will be steps outlined above to reduce their vulnerability further strengthened as the new Patient-Centered to disclosure requests. Outcomes Research Institute, which is authorized

Case Examples for Chapter 9

Case Example 17. Handling discovery Challenge requests for registry data Before its launch, the ICD Registry received Description The ICD Registry captures the significant attention from the public and characteristics, treatments, and researchers because of the CMS coverage outcomes of patients receiving decision. The registry sponsors anticipated that implantable cardioverter the registry would generate interest from outside defibrillators (ICDs). entities seeking to discover registry data, Participation in the registry is particularly those investigating potential fraud required by a coverage decision (e.g., the Office of the Inspector General), those of the Centers for Medicare & desiring to use registry data in malpractice Medicaid Services (CMS). In lawsuits (e.g., litigation attorneys), and those addition to CMS reporting, seeking to corroborate information published in participating hospitals may use the peer-reviewed literature (e.g., media their data to monitor and representatives). Driven by these concerns, the improve the outcomes and sponsor and registry staff sought to address this management of ICD patients anticipated issue proactively. through implementation of Proposed Solution evidence-based clinical guidelines. Registry staff implemented procedures to assist all parties involved in handling a discovery Sponsor American College of Cardiology request. In coordination with general counsel and Foundation (ACCF) outside counsel, internal policies were drafted Year Started 2005 and provisions were written into contracts with Year Ended Ongoing participating sites that explained the way registry staff would handle discovery requests, the process No. of Sites Over 1,600 laboratories of responding to a discovery request from an No. of Patients Over 800,000 procedures

224 Chapter 9. Protecting Data: Confidentiality and Legal Concerns of Providers, Manufacturers, and Health Plans

Case Example 17. Handling discovery Results requests for registry data (continued) Since 2005, the registry has received five Proposed Solution (continued) different requests for registry data, from sources attorney, the process of cooperating with the as varied as attorneys, the Office of the Inspector Office of Inspector General during an audit or General, and members of the press. The requests investigation, and best practices for protecting are managed in a consistent, documented way, registry data from discovery. This language regardless of whether the registry first receives prepared sites for these procedures should they these requests via site support staff or other ever occur, made them aware of their options in venues. The registry maintains a relationship with these situations (i.e., cooperate with or dispute the ACCF general counsel so that questions about the request), and clarified the level of protection future requests can be promptly resolved. that the registry offered for their data. Key Point Standard operating procedures were implemented Registries can take proactive steps to manage to train staff to recognize a discovery request and discovery requests for their data. Appropriate subpoena and to describe the actions that should steps may include confidentiality provisions in be taken to appropriately triage and respond to contracts with sites and targeted training for all discovery requests for registry data (e.g., that site levels of registry staff. support staff direct such requests to the registry For More Information compliance office). More in-depth staff training was provided, including role-play scenarios in https://www.ncdr.com/webncdr/ICD/Default.aspx which staff tested their skills and confidence through simulated discovery requests.

225 Section II. Legal and Ethical Considerations for Registries

Case Example 18. Meeting the confidentiality anesthesia-related cardiac arrest was 1.4 per and quality improvement needs of providers 10,000 anesthetics, with a mortality rate of 26 through a patient safety organization percent. The most common causes were Description The Pediatric Peri-Operative medication related and cardiovascular, with Cardiac Arrest (POCA) Registry cardiovascular depression from halothane (a investigated the incidence, commonly used anesthetic in children) causes, and outcomes of cardiac accounting for two-thirds of medication-related arrest among children cardiac arrests. This suggested a target for undergoing anesthesia. The preventive strategies, including avoidance of Wake Up Safe (WuS) Initiative halothane when a new agent (sevoflurane) was is a quality improvement available. In 2007, the registry published an initiative and registered patient update on its findings, comparing cardiac arrests safety organization (PSO) that from 1998 through 2004 with the cardiac arrests aims to improve processes of in the initial report. This report found fewer care and outcomes for children medication-related cardiac arrests (associated undergoing anesthesia. with a decline in use of halothane) and more arrests with undetermined causes. Sponsor POCA: American Society of Anesthesiologists and the Despite these promising findings, the pediatric University of Washington; WuS: anesthesiology community sought a more Society for Pediatric Anesthesia comprehensive approach to quality care improvement. The registry collected only data on Year Started 1994 (POCA) and 2008 (WuS) pediatric cardiac arrest, and not on any other Year Ended 2005 (POCA) and ongoing outcomes, processes, or quality of care indicators. (WuS) In addition, because data was submitted anonymously to the registry, benchmarking at the No. of Sites 58–79 (POCA) and 18 (WuS) institution level was not possible. There were also No. of Patients 374 events from >3.5 million concerns about the protection of registry data. anesthetics (POCA) and 518,000 The registry was housed in the State of anesthetic records (WuS) Washington, which had legal protections in place to protect quality improvement data from legal Challenge discovery. However, not all States had these same Children undergoing anesthesia have an increased protections in place. risk of cardiac arrest compared with adults. Proposed Solution Although factors associated with this increased risk had been identified, the causes of these In 2005, the POCA Steering Committee decided arrests and their outcomes were not well to halt case collection, as compliance with understood. The American Society of reporting had declined. While the registry had Anesthesiologists and the University of contributed much to understanding of anesthesia- Washington formed the POCA Registry in 1994 related cardiac arrest in children, it was felt that to collect detailed case reports of cardiac arrests further data collection using the same during anesthesia in pediatric patients. methodology was unlikely to produce additional Institutions submitted these case reports insights. anonymously to protect the participating In 2007, the Society for Pediatric Anesthesia institutions and the registry from legal risks of (SPA) began garnering support to form the Wake disclosure. Up Safe Initiative, a new quality improvement In 2000, the registry analyzed the first 4 years of initiative for pediatric anesthesiology. As data (1994–1997) and found that the incidence of institutions joined, they signed participation agreements with the SPA that were intended to

226 Chapter 9. Protecting Data: Confidentiality and Legal Concerns of Providers, Manufacturers, and Health Plans

Case Example 18. Meeting the confidentiality Key Point and quality improvement needs of providers Registries, particularly those that collect sensitive through a patient safety organization information on provider performance, should (continued) consider taking advantage of the legal protections Proposed Solution (continued) that are available to patient registries. Official address intellectual property issues, but evolved designation as a patient safety organization (PSO) as each hospital had its own concerns about offers broader Federal protections that individual privacy, protection, and anonymity in reporting. States may not be able to offer. This may provide By 2008, 10 institutions had signed on to an incentive for participation, especially for participate in the initiative. The same year, the registries that collect sensitive information on Patient Safety and Quality Improvement Act of performance or quality of care. 2005 came into effect, which provides Federal For More Information legal protections to information reported by http://www.wakeupsafe.org/ providers to a patient safety organization (PSO). WuS applied for and was granted PSO status in Morray JP, Geiduschek JM, Ramamoorthy C, et 2008. al. Anesthesia-related cardiac arrest in children: initial findings of the Pediatric Perioperative Results Cardiac Arrest (POCA) Registry. Anesthesiology. The WuS initiative captures a broad range of 2000 Jul;93(1):6-14. outcomes, and individual institutions are not Posner KL, Geiduschek J, Haberkern CM, et al. identifiable. Anesthetic records for pediatric Unexpected cardiac arrest among children during patients are extracted from the administrative surgery, a North American registry to elucidate billing systems of member hospitals and provided the incidence and causes of anesthesia related to the registry on a quarterly basis. This cardiac arrest. Qual Saf Health Care. 2002 Sep; information provides background data on which 11(3):252-7. to base incidence calculations. Individual event case reports are provided to the registry on an ad Bhananker SM, Ramammoorthy C, Geiduschek hoc basis, as they occur. To date, 18 member JM, et al. Anesthesia-related cardiac arrest in hospitals have contributed 518,000 anesthetic children: update from the Pediatric Perioperative records and 450 event case reports to the registry. Cardiac Arrest Registry. Anesth Analg. 2007 Aug; 105(2):344-50. As a PSO, the registry provides more complete protections for the providers contributing data. Ramamoorthy C, Haberkern CM, Bhananker SM, These include limits on the use of registry data in et al. Anesthesia-related cardiac arrest in children civil, administrative, and some criminal with heart disease: data from the Pediatric proceedings, and provisions for monetary Perioperative Cardiac Arrest (POCA) registry. penalties for violations of confidentiality or Anesth Analg. 2010 May 1;110(5):1376-82. privilege protections.

227 Section II. Legal and Ethical Considerations for Registries

Case Example 19. Protections available to registry data. Following university policy, all registry data from institutional review boards requests were referred to the office of public and academic institutions information. All such public information requests Description The Postoperative Visual Loss were denied based on the institutional review (POVL) Registry consists of board–approved confidentiality procedures. anonymous case reports of One public information request for registry data blindness after non- was appealed through the court system. A registry ophthalmologic surgery. Its goal investigator was serving as a defense expert in a is to identify patient and clinical malpractice lawsuit, basing her testimony on factors associated with this published registry results. The plaintiff requested complication. raw registry data. When the university denied this Sponsor American Society of request, the plaintiff sought to strike the Anesthesiologists investigator’s testimony because she would not produce the raw registry data underlying the Year Started 1999 publication that formed the basis of her Year Ended Ongoing testimony. The trial court determined that the raw registry data was discoverable because the No. of Sites Not applicable defense expert considered such data in forming No. of Patients 191 her opinion. The university supported an appeal to the State Supreme Court in order to support Challenge institutional protections of research data. The Blindness after non-eye surgery is a rare but State Supreme Court ruled that the author could devastating complication. Blindness due to testify based on the published results and that ischemic optic neuropathy after spine surgery release of the underlying data was not required. appeared to be increasing in the 1990s, and its Key Point causation was unknown. Its rarity created difficulty in studying causative factors. It is critical to consider protection of sensitive registry data from legal discovery. When Proposed Solution developing and implementing registries, consider A registry was created to collect detailed case protections that may be available through IRB reports of blindness after non-ophthalmologic study approval and academic institutions. Seek surgery, and included data pertinent to all known guidance from IRBs or university counsel, as theories of causation of postoperative visual loss. needed. Due to the potential for malpractice litigation For More Information when postoperative blindness occurs, case reports were submitted without patient, provider, or http://www.cobar.org/opinions/opinion. institutional identifiers. It was hoped that cfm?opinionid=7861&courtid=2 anonymity of case reports would protect the Lee LA, Roth S, Posner, KL, et al. The American registry from legal discovery and encourage case Society of Anesthesiologists Postoperative Visual report submission by health care providers. The Loss Registry: Analysis of 93 spine surgery cases registry was housed at the University of with postoperative visual loss. Anesthesiology. Washington, and its institutional review board 105:652-9, 2006 (IRB) approved these confidentiality procedures. The Postoperative Visual Loss Study Group. Risk Results factors associated with ischemic optic neuropathy In spite of these procedures to protect after spinal fusion surgery. Anesthesiology. confidentiality of case reports, plaintiff attorneys 116:15-24, 2012. submitted numerous requests for release of

228 Chapter 9. Protecting Data: Confidentiality and Legal Concerns of Providers, Manufacturers, and Health Plans

References for Chapter 9 10. American College of Chest Physicians. AQuIRE Registry. http://www.chestnet.org/Guidelines-and- 1. Fisher E, Goodman D, Skinner J, et al. Health Resources/Payment-Practice-and-Quality/ Care Spending, Quality and Outcomes: More Isn’t Practice-Resources/AQuIRE. Accessed August 7, Always Better. Dartmouth Atlas Project Topics 2013. Brief. February 27, 2009. http://www. 11. American Orthopaedic Association. http://www. dartmouthatlas.org/downloads/reports/Spending_ ownthebone.org/. Accessed August 15, 2012. Brief_022709.pdf. Accessed August 15, 2012. 12. American College of Rheumatology. ACR 2. Fisher ES, Wennberg DE, Stukel TA, et al. The Rheumatology Clinical Registry. http://www. implications of regional variations in Medicare rheumatology.org/Practice/Clinical/Rcr/ spending. Part 1: the content, quality, and Rheumatology_Clinical_Registry/. Accessed accessibility of care. Ann Intern Med. 2003 Feb August 7, 2013. 18;138(4):273-87. PMID: 12585825. 13. American College of Radiology. National 3. Fisher ES, Wennberg DE, Stukel TA, et al. The Radiology Data Registry. http://www.acr.org/ implications of regional variations in Medicare Quality-Safety/National-Radiology-Data-Registry. spending. Part 2: health outcomes and satisfaction Accessed August 7, 2013. with care. Ann Intern Med. 2003 Feb 18;138(4):288-98. PMID: 12585826. 14. Society for Thoracic Surgeons National Database. http://www.sts.org/national-database. Accessed 4. McGlynn EA, Asch SM, Adams J, et al. The August 15, 2012. quality of health care delivered to adults in the United States. N Engl J Med. 2003 Jun 15. National Cardiovascular Data Registry. https:// 26;348(26):2635-45. PMID: 12826639. www.ncdr.com/webncdr/. Accessed August 7, 2013. 5. Wennberg JE, Fisher ES, Skinner JS. Geography and the debate over Medicare reform. Health Aff 16. 45 CFR pts. 160, 164 (2010). (Millwood). 2002 Jul-Dec;Suppl Web 17. Kesselheim AS, Ferris TG, Studdert DM. Will Exclusives:W96-114. PMID: 12703563. physician-level measures of clinical performance 6. Wennberg JE, Cooper MM, Bubolz TA, et al. The be used in medical malpractice litigation? JAMA. Dartmouth Atlas of Health Care in the United 2006 Apr 19;295(15):1831-4. PMID: 16622145. States: American Hospital Publishing, Inc.; 1996. 18. Kohn LT, Corrigan JM, Donaldson MS, eds. To http://www.dartmouthatlas.org/downloads/ Err is Human: Building a Safer Health System. atlases/96Atlas.pdf. Accessed August 15, 2012. Washington, D.C.: National Academies Press; 7. Wennberg JE, Brownlee S, Fischer ES, et al. An 1999. Agenda for Change: Improving Quality and 19. 42 U.S.C. § 299c-3(c). Curbing Health Care Spending: Opportunities for the Congress and the Obama Administration. The 20. Memorandum from Susan Merewitz, Senior Dartmouth Institute for Health Policy & Clinical Attorney, AHRQ, on Statutory Confidentiality Practice. December 2008. http://www. Protection of Research Data, to Nancy Foster, dartmouthatlas.org/downloads/reports/agenda_ Coordinator for Quality Activities, AHRQ. April for_change.pdf. Accessed April 30, 2012. 16, 2001. http://www.ahrq.gov/fund/datamemo. htm. Accessed August 15, 2012. 8. de Brantes F, Rosenthal MB, Painter M. Building a bridge from fragmentation to accountability--the 21. See, National Institutes of Health, Certificates of Prometheus Payment model. N Engl J Med. 2009 Confidentiality: Background Information. http:// Sep 10;361(11):1033-6. PMID: 19692682. grants.nih.gov/grants/policy/coc/background.htm. Accessed August 15, 2012. 9. Patient Protection and Affordable Care Act of 2010, Public Law No. 111-148. 22. 42 U.S.C. § 241(d).

229 Section II. Legal and Ethical Considerations for Registries

23. Rothman K, Greenland S, Poole C, et al. 45. 45 CFR § 5.67(c). Causation and Causal Inference. In: Rothman K, 46. 5 U.S.C. § 552(b)(3). Greenland S, Lash TL, eds. Modern Epidemiology. 3rd ed. Philadelphia: Lippincott 47. 18 U.S.C. § 1905. Williams & Wilkins; 2008, pp. 5-31. 48. 76 C.J.S. Records §111 (2011). 24. 42 U.S.C. §§ 299b-21-26. 49. Fed. R. Evid. 401. 25. 42 U.S.C. § 1320c-9(b). 50. Fed. R. Evid. 403. 26. 42 U.S.C. § 1320c-9(d). 51. Fed. R. Evid. 404(b). 27. 42 CFR § 480.140. 52. Fed. R. Civ. P. 26(b)(2)(C)(iii). 28. Health Insurance Portability and Accountability 53. Fed. R. Civ. P. 26(c). Act of 1996, Public Law No. 104-191, 110 Stat. 139 (1996) (codified as amended in scattered 54. Fed. R. Civ. P. 45(c). sections of 42 U.S.C.). HIPAA Privacy Rule 55. Andrews v. Eli Lily & Co., Inc., 97 F.R.D. 494 Regulations codified at 45 CFR pts. 160 & 164 (1983). (2010). 56. 97 F.R.D. at 502. 29. 45 CFR § 160.103 (2010). 57. Deitchman v. E.R. Squibb & Sons, Inc., 740 F.2d 30. 45 CFR § 164.512(e). 556 (7th Cir. 1984). 31. Privacy Act of 1974, Public Law No. 93-579, § 3, 58. 740 F.2d at 561. 88 Stat. 1896, 1896 (codified as amended at 5 U.S.C. § 552a (2006)). 59. Wolpin v. Phillip Morris, Inc., 189 F.RD. 418 (C.D. Cal. 1999). 32. Id. at 5 U.S.C. § 552a(b). 60. Patient Protection and Affordable Care Act, Public 33. Electronic Freedom of Information Amendment Law 111-148, §10332 (2010). Acts of 1996, Public Law No. 104-231, 110 Stat. 3048 (1996) (codified as amended at 5 U.S.C. § 61. Patient Protection and Affordable Care Act, Public 552). Law 111-148, §10332(e)(4)(D) (2010). 34. Freedom of Information Act, 5 U.S.C. § 552 62. See, e.g. Va. Code Ann. §32.1-38 (2011) (2006), amended by OPEN Government Act of (immunity from civil liability or criminal penalty 2007, Public Law No. 117-175. unless such person acted with “gross negligence or malicious intent”); Ariz. Rev. Stat. Ann. 35. U.S. Department of Justice. Freedom of §36-666 (2010) (immunity from civil or criminal Information Act Guide (2004). http://www.justice. liability if the provider acted in “good faith and gov/oip/introduc.htm#N_2_. Accessed August 15, without malice”); Iowa Code §139A.3 (2010) 2012. (immunity from any liability, civil or criminal, for 36. 5 U.S.C. § 552(b)(4). any person “acting reasonably and in good faith” while reporting); Kan. Stat. Ann. §65-118 (2010) 37. 45 CFR § 5.65(a). (immunity from any liability, civil or criminal, if 38. Pub. Citizen Health Research Group v. Food & provider reported “in good faith and without Drug Admin., 704 F.2d 1280 (D.C. Cir. 1983). malice”). 39. 704 F.2d at 1290. 63. See, e.g. N.C. Gen Stat. §130A-142 (2010) (a provider who reports “shall be immune from any 40. 5 U.S.C. § 552(b)(6). civil or criminal liability that might otherwise be 41. See U.S. Dept of State v. Washington Post Co., 456 incurred or imposed as a result of making that U.S. 595, 602 (1982). report”). 42. U.S. Dept of Justice v. Reporters Comm. for 64. See, e.g. Neb. Rev. Stat. §71-503.01 (2010) Freedom of the Press, 489 U.S. 749 (1989). (immunity for providers from suits for slander, libel, or breach of privileged communication); 43. 489 U.S. at 771. Nev. Rev. Stat. §629.069 (2010) (immunity for 44. 489 U.S. at 773. providers from civil liability only).

230 Chapter 9. Protecting Data: Confidentiality and Legal Concerns of Providers, Manufacturers, and Health Plans

65. See, e.g., KB v. Mills, 639 N.W.2d 261 (Mich. Ct. 67. Minn. Code § 154.61 (a, g), § 145.64 (2010). App. 2002); State v. Superior Court, 930 P.2d 488 68. Va. Code Ann. § 8.01-581.17 (2011). (Ariz. Ct. App. 1997); Alicia T. v. Cnty of L.A., 222 Cal App. 3d 869 (Cal. Ct. App. 1990); 69. 735 Ill. Comp. Stat. 5/8-2101 (2010). Criswell v. Brentwood Hospital, 551 N.E.2d 1315 70. Adkins v. Christie, 488 F.3d 1324 (11th Cir. 2007). (Ohio Ct. App. 1989). 71. Marshall v. Spectrum Medical Group, 198 F.R.D. 66. See e.g., Idaho Code § 39-1392 (2011) (“all peer 1 (D. Me. 2000). review records shall be confidential and privileged, and shall not be directly or indirectly subject to 72. Price v. Howard County Gen. Hosps., 950 F. Supp. subpoena or discovery proceedings or be admitted 141 (D. Md. 1996). as evidence, nor shall testimony relating thereto be admitted in evidence, or in any action of any kind in any court or before any administrative body, agency or person for any purpose whatsoever”); Ohio Rev. Code Ann. § 2305.252 (2011) (“proceedings and records within the scope of a peer review committee of a health care entity shall be held in confidence and shall not be subject to discovery or introduction in evidence in any civil action against a health care entity or health care provider…).

231

Section III Operating Registries

Chapter 10. Recruiting and Retaining Participants in the Registry

1. Introduction Registries that are not voluntary have different drivers for participation. In general, the burden of Recruitment and retention of participants are participation should be kept as low as possible, essential elements in the design and operation of a while the relative rewards, particularly registry. Registries are often intended to be nonmonetary rewards, should be maximized. As representative of a certain population of patients described in Chapters 2 and 4, minimizing burden and reflective of the practices of certain providers typically starts with focusing on the key goals of and geographic areas. The problems commonly the registry. associated with clinical studies—such as Building participation incentives into a registry difficulties with patient enrollment, losses to should also be included in the planning phase. A followup, and certain sites contributing the broad range of incentives—spanning a spectrum majority of patients—can also have profound from participation in a community of researchers, consequences on validity of registry data. When to access to useful data or quality improvement registry patients are not representative of the target benefits, to continuing medical education, to public population, the value of the results is diminished. recognition or certification, to payments or access For example, in regard to policy determinations, to patients—have been used in registries. The the enrolled sites or providers must be ability to offer certain incentives (e.g., linking representative of the types of sites and providers to payment for a service to participation in a registry which the policy determination would apply in or access to patients) may be available only to order for the results of the registry to be certain registry developers (e.g., payers, licensing generalizable. Differences in how effectively sites entities). Many registries incorporate multiple enroll or follow patients can skew results and types of incentives, even when they pay for overly reflect the sites with the most data. This participation. Monetary incentives (e.g., from oversampling within a particular site or location payers or sponsors) are very helpful in recruiting must also be considered in sample size sites. However, because the payments should not calculations. If the sample size of a key unit of exceed fair market value for work performed, analysis (patient, provider, or institution) is not registries cannot solely rely on these incentives. A sufficient to detect a clinically important number of nonmandated registries have achieved difference, the validity of the entire registry is success in recruitment and retention by providing a weakened. (See Chapters 3 and 13.) combination of ethical incentives that are tailored Well-planned strategies for enrollment and to and aligned with the specific groups of sites, retention are critical to avoiding these biases that providers, and patients that are asked to may threaten registry validity. Because registries participate. (See Case Examples 20, 21, 22, and typically operate with limited resources and with 23.) voluntary rather than mandatory participation, it is particularly important to balance the burdens and 2. Recruitment rewards of participation in the registry. The term “voluntary” in this context is intended to mean that Depending on the purpose of a registry, participation in the registry by either providers or recruitment may occur at any of three levels: patients is not mandated (e.g., by the U.S. Food facility (e.g., hospital, practice, and pharmacy), and Drug Administration), nor is participation provider, or patient. While recruitment at these required as a necessary condition for a patient to levels is frequently part of a design to accrue a gain access to a health care product or for a sufficient number of patients for sample size provider to be eligible for payment for a health purposes, such as for a safety registry, the care service. individual levels may also constitute potential units

235 Section III. Operating Registries

of analysis (and as such, may further affect sample registry sponsor, provided that registry size, as discussed in Chapter 3, Section 8). As an participation allows the hospital to meet external example, a registry focused on systems of care that quality-of-care mandates. In other cases, is examining both hospital system processes and participation in a quality monitoring or health patient outcomes might need to consider system surveillance registry may be required by characteristics of the individual patients, the payers or governments for reimbursement, providers, and/or the places where they practice differential payments, or patient referrals under (i.e., clusters). If the question is about the practices various programs, ranging from the Centers for of orthopedic surgeons in the United States, the Medicare & Medicaid Services (CMS) public registry will be strengthened by describing the reporting initiative, to centers of excellence number and characteristics (e.g., age, gender, and programs, to pay-for-performance programs. One geographic distribution) of U.S. orthopedic particular example, CMS’s Coverage With surgeons, perhaps by citing membership data from Evidence Development programs,2 which may the American Academy of Orthopedic Surgeons. require participation in a registry for the center or This will allow documentation of the similarities provider to qualify for payment for a procedure, and differences in the characteristics of the can have a dramatic impact on registry surgeons participating in the registry compared participation. Registry participation requirements with the target population. (See Chapter 3, have existed for implantable cardioverter Section 7.) defibrillators for preventing sudden cardiac death in heart failure, for bariatric surgery, for positron 2.1 Hospital Recruitment emission tomography scan use in cancer, and for A hospital or health system may choose to other procedures and devices. These requirements participate in a patient registry for many reasons, have rapidly resulted in high participation rates for including the research interest of a particular registries meeting them. investigator or champion, the ability of the hospital The presence of quality assurance departments in to achieve other goals through the registry (such as U.S. hospitals provides an infrastructure for requirements for reimbursement, certification, or participation in many hospital-based registries, and recognition), or the general interest of the these departments are therefore a natural target for particular institution in the disease area (e.g., recruiting. However, hospital size, service line specialty hospitals). Increasingly, external (e.g., disease-specific centers), and competing mandates to document compliance with practice activities may limit institutional interest. The standards provide an incentive for hospitals to American Hospital Association database provides participate in registries that collect and report a valuable resource for identifying hospitals by key mandatory hospital performance or quality-of-care characteristics, including hospital ownership, data. For example, a number of registries allow number of beds, and the presence of an intensive hospitals to document their performance to meet care unit. the Joint Commission requirements for hospital accreditation.1 Hospitals in the United States must Table 10–1 summarizes the key factors for submit these data to maintain accreditation. successful hospital recruitment, and lists specific Therefore, hospital administrators may be willing methods that might be used to recruit hospitals. to supply the staff time to collect these data While programs need not incorporate all of these without additional financial incentives from the characteristics or use all of these methods, successful programs typically incorporate several.

236 Chapter 10. Recruiting and Retaining Participants in the Registry

Table 10–1. Hospital recruitment

Keys to hospital • The condition being studied satisfies one of the hospital’s quality assurance mandates. recruitment Sufficient funds, data, or other benefits will be realized to justify the effort required to participate. • The confidentiality of the hospital’s performance data is ensured, except to the extent that the hospital elects to report it. • Clinically relevant, credible, timely, and actionable self-assessment data—ideally, data that are risk adjusted and benchmarked—are provided back to the hospital to help it identify opportunities for enhancing patient care outcomes. • High-profile hospitals (regional or national) are participating in the registry. • Burden is minimized. • Participation assists the hospital in meeting coverage and reimbursement mandates, gaining recognition as a center of excellence, or meeting requirements for pay-for- performance initiatives. Methods • Identify eligible hospitals from the American Hospital Association database. of hospital • Use stakeholder representatives to identify potentially interested hospitals. recruitment • Enroll hospitals through physicians who work there and are interested in the registry. • Use invitation letters or calls to directors of quality assurance or the chief of the clinical department responsible for the condition targeted by the registry. • Ask physician members of an advisory board (if applicable) to network with their colleagues in other hospitals. • Reach out to physicians or hospital administrators through relevant professional societies or hospital associations. • Leverage mandates by external stakeholders, including third-party payers, health plans, or government agencies.

2.2 Physician Recruitment • Disruption: Will participation disrupt workflow of the staff? A physician practice may or may not choose to participate in a voluntary registry for many • Value: What benefits will be derived from reasons. As with hospitals, these reasons can participation? Will it improve the care include the research interests of the physician and provided? Will it enhance the evidence base for the ability of the practice to achieve other goals future practice? through the registry (such as reimbursement or Physicians who manage only a few patients per recognition). When deciding to participate, year with the disease that is the subject of the physicians often focus on several concerns: registry are less likely to be interested in enrolling • Relevance: Does the registry have meaning for their patients than physicians who see many such the practice and patients? patients—unless the disease is rare or extremely rare, in which case the registry may be of great • Trust: Are the registry leaders credible? Are the interest. goals clearly stated? Because most registries are voluntary and • Risks: Will confidentiality be maintained? Are physicians in nonacademic practice settings may patient records secure? have less infrastructure and staff available to enroll • Effort: Will the amount of effort expended be their patients, recruitment of representative fairly compensated? physicians is a major challenge for registries that aim to compare physician practices across a full

237 Section III. Operating Registries

spectrum of practice settings. In general, not be shared (except at the direction of the community-based physicians are less well physician) or published, and that registry outcomes equipped than hospital-based or academic data will be released only in large aggregates that physicians to collect data for research studies protect the identities of individual hospitals, because they work in busy practices geared to physicians, and patients. In addition, any routine clinical care rather than research. To incentives should be clearly articulated. increase recruitment of nonacademic physicians, it Table 10–2 describes the key factors for successful can be helpful to clearly explain the purpose and physician recruitment and lists several methods objectives of the registry; how registry data will be that might be used for recruiting physicians. used; and, specifically, that individual results will

Table 10–2. Physician recruitment

Keys to physician • The condition being studied is part of the physician’s specialty. recruitment • The registry is a valuable scientific endeavor. • The registry is led by respected physician opinion leaders. • The registry is endorsed by leading medical, government, or patient advocacy organization(s). • The effort needed to recruit patients and collect and submit data is perceived as reasonable. • Useful practice pattern and/or outcome data are provided. • The registry meets other physician data needs, such as maintenance of certification requirements, credentialing requirements, or quality-based, differential, reimbursement payment programs (pay-for-performance). Methods of physician • Purchase mailing lists from physician specialty organizations. recruitment • Ask opinion leaders in the field to suggest interested colleagues. • Partner with local and national medical societies or large physician hospital organizations. • Use stakeholder representatives to identify interested physicians. • Recruit and raise awareness at conferences. • Advertise using email and the Web. • Register in the Registry of Patient Registries (RoPR) to increase awareness. • Leverage practice-based research networks.

238 Chapter 10. Recruiting and Retaining Participants in the Registry

2.3 Vetting Potential Hospital and Since registries should not modify the usual care Physician Participants that physicians provide to their patients, there should be little or no conflict between their role of Once potential hospital or physician participants physician and that of participant in the registry. have been identified, it is important to vet them to (See Chapter 7.) In addition, patients may see ensure that the registry is gathering the appropriate participation in the registry as an opportunity to mix of data. Issues to consider when vetting increase their communication with their clinician. potential participants include— Another incentive for many patients is the feeling • Representativeness that they are contributing to the knowledge base of • Hospital characteristics (e.g., bed size, sometimes poorly understood and undertreated geographic location) conditions. • Physician characteristics (e.g., specialty Recruitment of patients presents different training) challenges, depending on the nature of the condition being studied. In general, patient • Practice setting (health maintenance recruitment plans should address the following organization [HMO], private practice) questions: • Ability to recruit patients • Does the plan understand the needs and • Volume of target cases interests of potential participants? • Internal resources • Does the plan address patient recruitment issues and procedural challenges, including • Availability of a study coordinator informed consent and explanation of risks? • Availability of Internet connectivity for studies • What are the patient retention goals? What is a with electronic data capture reasonable followup period? What is a • Prior performance, including reliability and reasonable followup rate? When does reduced accuracy of data entry retention compromise validity?

2.4 Patient Recruitment • What, if any, patient incentives are offered, including different types of incentives and the Patients may be recruited based on the judgment ethical, legal, or study validity issues to be of the physician who provides their care; the considered with patient incentives? diagnosis of a disease; receipt of a procedure, operation, device, or pharmaceutical; membership • What are the costs of patient recruitment and in a health insurance plan; or membership in a retention? group of individuals who have a particular Table 10–3 summarizes the key factors for patient exposure. Recruitment of patients by the physician recruitment and lists several specific methods that who is providing their care is one of the most might be used for recruiting patients, grouped by successful strategies. The direct involvement in the basic categories of patients at the time of and support of the registry by their personal recruitment. physicians is an important factor for patients.

239 Section III. Operating Registries

Table 10–3. Patient recruitment

Keys to patient • Recruit through a physician who is caring for the patient. recruitment • Communicate to the patient that registry participation may help to improve care for all future patients with the target condition. • Write all patient materials (brochures, consent forms) in a manner that is easily understandable by the lay public. • Keep the survey forms short and simple. • Provide incentives. These can be nonmonetary, such as functions relevant to the patient’s care (reports) or community (newsletters, portals). In some cases, monetary incentives can be offered if approved by the institutional review board. • Actively plan how to include minorities or other populations of interest. Methods • Noninstitutionalized residents of the general U.S. population: of patient –– Recruit via letter survey, telephone, or email. recruitment –– Recruit during well-patient visits to outpatient clinics. –– Recruit via patient advocacy and support groups, health information Web sites, etc. –– Register in the Registry of Patient Registries (RoPR) to increase awareness. • Outpatients attending the clinic of a physician who is participating in the registry: –– Recruit through the patient’s physician. –– Recruit via brochures placed in physician’s office. • Hospital inpatients who are hospitalized for treatment of a condition that is the subject of the registry: –– Recruit through the patient’s physician. –– Recruit through hospitalists or consultant specialists. –– Recruit through a hospital research coordinator. • Residents of nursing homes and similar long-term care facilities: –– Establish a relationship with the nursing home and staff.

2.5 Partnerships To Facilitate • Industry (e.g., pharmaceutical companies) Recruitment • HMOs and other third-party insurance Many agencies/organizations can assist in the providers recruitment of physicians and patients. These partners may have access to patients or their 2.6 Procedural Considerations Related families and physicians who treat the condition, To Recruitment and they may lend credibility to the effort. These When developing a recruitment plan for a registry, agencies/organizations include— consideration should be given to the procedural • Government agencies concerns that may be factored into potential participants’ decisions. These concerns include the • Physician professional associations or State roles and responsibilities of each party, the need medical associations and process for obtaining institutional review • Certifying boards (e.g., American Board of board (IRB) approval, and the management of Neurological Surgeons) patient and provider confidentiality. • Patient advocacy groups (e.g., Muscular The contract between registry sites and the sponsor Dystrophy Association) or coordinating center should clearly state the roles and responsibilities of the participants, the • Nonprofit foundations (e.g., Robert Wood registry-coordinating center, and the sponsor. If Johnson Foundation)

240 Chapter 10. Recruiting and Retaining Participants in the Registry

remuneration is being offered, the data-entry documentation. Case report forms and patient logs requirements that need to be fulfilled before must be designed to minimize patient identification payments are made should be stated. It is often (such as by transmitting limited data sets rather helpful to explain to sites the concept of fair than more identifiable information, if such market value. Where the Health Insurance information is not required to meet a registry Portability and Accountability Act (HIPAA) objective). applies, consideration should also be given to new The intended management of the confidentiality of requirements in the HIPAA Privacy Rule that participating providers should be explained in the prohibit a covered entity from disclosing protected contract. Mechanisms for protecting provider health information in exchange for remuneration confidentiality, including Certificates of from the recipient of that information without the Confidentiality and Patient Safety Organizations, individual’s authorization, subject to a number of are discussed in Chapters 7 and 9. If third-party or exceptions (see 45 CFR 164.502(a)(5)(ii)). There public reporting is an intended component of the is no specific formula (such as whether to separate registry, the specific data to be shared, the level of startup payments from per-patient payments), but the disclosure (e.g., hospital and/or physician total remuneration must reflect work effort for the level), and the permitted receiving entities need to specific registry. Some individual factors, ranging be articulated and the control mechanisms from location to specialty, may have a bearing on explained. fair market value. It is also important to spell out which entity will have ownership of the data and how the data will be used. 3. Retention The contract should clearly explain the registry 3.1 Providers policy regarding any necessary approvals. If review by an IRB is required, generic templates Once hospitals and physicians are recruited to can be offered to participants to assist them in participate in a registry, retaining them becomes a obtaining ethical and IRB approval. Because the key to success. All of the factors identified as costs of obtaining IRB approval are often important for recruitment are important for substantial, it is essential that the contract with the retention as well. A critical factor in retention is participants clearly indicate which party is delivery on promises made during recruitment responsible for bearing this cost. If the registry (e.g., that the burden of participation is low). By developer believes that IRB or privacy board carefully pilot testing all aspects of the registry review or approval is not required or may be prior to full recruitment, there is less likelihood waived, then a clear rationale should be provided that problems will arise that threaten the registry’s to the prospective participants (see Case Example reputation. Registries with an advisory board or 57). As discussed in Chapter 7, the research steering committee can use this resource to help purpose of the registry, the type of entity that with retention. A visible and independent advisory creates and maintains it, whether the Common board adds transparency and credibility, sets Rule applies to the particular site, and the extent to appropriate expectations among its peers on what which the data are individually identifiable largely to expect from a registry (e.g., compared with a determine which regulatory requirements apply. clinical trial), ensures that the burden of the For example, for registries limited to certain registry is minimized (or at least never outweighs purposes, such as quality improvement, institutions its value to participants), and maintains the may not need IRB approval.3 relevance and currency of the registry for the investigators. Ideally, advisory board members Patient privacy and participant confidentiality serve as ambassadors for the program. The level of should be addressed in the registry materials. credibility, engagement, practicality, and Methods of ensuring patient privacy need to be enthusiasm of the advisory board can significantly clearly elucidated in all registry-related affect provider recruitment and retention. For

241 Section III. Operating Registries

example, an advisory board whose clinical • Regular data reports to stakeholders members are not themselves participating in the • Presentations at conferences registry will have greater difficulty than a board with participating members in addressing the • Regular reports to registry participants on concerns of participating practices that invariably registry growth and publications arise over the course of the registry. Including • Ability of participating physicians to publish patients or patient advocacy organization based on registry data (depending on the data representatives on the advisory board also can access policy of the registry) support patient retention efforts. These representatives can provide feedback to the board 3.2 Patients on patient issues or concerns about the registry, Retaining patients as active participants in and they can facilitate communication about the registries with longitudinal followup is an ongoing registry’s purpose and value to their peers or challenge. Many factors need to be considered in members. developing a retention plan, including how long Throughout the registry’s duration, communication the patient is likely to return to the enrolling site. from the data coordinating center and the advisors, Patients enrolled in a primary care practice for a as well as community building, are important for chronic illness can likely be followed in that strong retention. Early and continued engagement practice for some time, although there should be a of the site champions or principal investigators is plan for how the registry will (or will not) address very important. Some registries use periodic the issue of patients who transfer to unenrolled face-to-face meetings of principal investigators practices. Different solutions are needed for from participating sites. When this approach is not patients enrolled in a hospital at discharge or economically feasible, well-planned online through a specialist who does not follow the meetings can serve the same purpose. patient long term. Several options exist. They Visibility of the registry at relevant national include enlisting site staff to reach out to patients 4 meetings can help maintain clinician awareness beyond their standard interactions, following and sense of community, and regular patients directly through a central patient 5 demonstration of its value through presentations management center, and linking to other data and publications reinforces the credibility of the sources (e.g., National Death Index, claims data) registry to its participants. As the data set grows, to obtain key long-term outcomes data on patients so too does the value of the registry for all who are lost to followup. Retention plans, participants, and regular updates on the registry including contingencies, should be considered growth can be important. Finally, enhancing site during registry planning, as they may require value through nonfinancial rewards can be additional permissions (e.g., for direct contact) or particularly useful in retention, and the registry data elements (e.g., for linkage). Maintaining should continually seek to bring value to the ethical incentives for patient participation (ranging participants in creative and useful ways. from newsletters to payments) is important for some registries (e.g., those that collect patient- Participation retention tools include— reported outcomes data). Beyond planning for how • Web sites to retain patients in a registry, it is important to track actual versus expected followup rates over • Newsletters time and to respond if rates are not meeting • Telephone helplines expectations. The resources available for patient • Instruction manuals retention efforts should also be clear. Followup rates can often be improved with more efforts, • Training meetings such as more attempts to contact the patient, but • Site audit/retraining visits these efforts add costs and, at some level, will yield diminishing returns. • Customer satisfaction/opinion surveys

242 Chapter 10. Recruiting and Retaining Participants in the Registry

4. Pitfalls in Recruitment and consecutive or randomized enrollment). In addition, overly demanding data collection Retention requirements can affect retention. The schedule Pitfalls abound in recruitment and retention. The should be designed to obtain relevant data in a most important of these pitfalls is the risk of timely fashion without overtaxing the resources of selection bias. Targeting hospital-based or patients and providers. It is also important to academic physicians to the exclusion of consider approaches that will distinguish patients community-based physicians is tempting because who are lost to followup from those who have the former are often more accessible and are missing data for other reasons (such as a patient frequently more open to involvement in, and more who missed a visit but is still in the registry). experienced in, research projects. Similarly, Another major pitfall is confusing terminology. targeting high-volume practices or centers will This can be a major problem when the registry is improve efficiency of patient enrollment, but may international. When designing training materials, not yield an adequately representative sample of instruction manuals, and questionnaires, it is care practices. If an advisory board or committee critical that the language and terminology be clear is used to help design the registry and aid in and concise. Materials that are translated into other recruitment, there may be a tendency for advisors languages must undergo strict quality assurance to recruit known colleagues or to target disease measures to ensure that terms are translated experts, when a wider range of participants may be properly (e.g., back translation). necessary to provide the appropriate data to meet the research goals. Including representatives from 5. International the range of anticipated site types on the advisory board can be helpful. Considerations Even with an appropriate mix of physician While many general principles are similar for participants in a registry, biases in patient participant enrollment and retention in other parts recruitment may still occur. For example, older and of the world, there are many different customs or more seriously ill patients may be excluded regulations regarding contract language, because of challenges in enrollment and followup requirements for ethics committee or other or poorer outcomes. From the outset, physicians submissions, informed consent, and allowable involved in recruitment efforts need to be aware of approaches to patient retention. Registries that the potential for bias, and they must understand the extend to other countries should consult national importance of adhering to well-delineated and local regulations in those countries. inclusion and exclusion criteria. They must also adhere to the registry’s enrollment strategy, which is typically designed to reduce this bias (e.g.,

243 Section III. Operating Registries

Case Examples for Chapter 10

Case Example 20. Building value as a means to competition. The registry team wanted to recruit hospitals motivate resource-strapped hospitals to Description Get With The Guidelines® is the consistently and proactively enter data and flagship program for in-hospital analyze improvement. quality improvement of the Proposed Solution American Heart Association The registry team began by listening to the (AHA) and American Stroke hospitals through in-depth interviews designed to Association (ASA). The understand the motivations and deterrents program uses the experience of underlying behavior. Interviews were conducted the AHA and ASA to ensure that with hospital decisionmakers at all levels (nurses, the care that hospitals provide QI professionals, administrators/chief executive for heart failure, stroke, and officers, and physicians). resuscitation is aligned with the latest evidence-based guidelines. Based on the research findings, the team developed strategies that differentiated and built Sponsor American Heart Association and value for the program. Some of the more American Stroke Association noteworthy strategies included the following: Year Started 2000 • Systems were designed to allow data Year Ended Ongoing transmission from and to Joint Commission No. of Sites 3,150 and CMS vendors, enabling hospitals to reduce the burden of duplicate data entry No. of Patients 3,174,462 patients and 4,253, while still participating in other programs. 461 patient records in Get With The Guidelines-Inpatient • A new tagline, Turning Guidelines into (Stroke, HF, and Resuscitation) LifelinesSM, linked the brand’s value proposition to the brand name and logo. Key Challenge messages for each target audience were Recruiting hospitals for registries or quality included in marketing communications. improvement (QI) programs can be arduous. • A newly designed national recognition Human and financial capital is constrained. program motivated participation and Accreditation and reimbursement programs, such advancement, and received the attention of as those of The Joint Commission (formerly the hospital decisionmakers. Joint Commission on Accreditation of Healthcare Organizations, or JCAHO) and Centers for • Return-on-investment studies for the program Medicare & Medicaid Services (CMS), contend demonstrated the value of participation. for the same valuable human and financial Product innovations/enhancements created capital. As a result, in the absence of specific additional incentives to participate. Immediate benefits, many hospitals defer the data collection point-of-care flags highlighted from and report utilization required for successful QI guidelines. Benchmarking filters/reports execution. empowered decisionmakers to benchmark Like most registries and QI programs, the performance with national averages and data sponsor’s program faced barriers to data entry. from similar institutions. Customizable notes Unlike other registries, Get With The Guidelines explaining diseases, tests, and medications can be offered no reimbursements for data entry and sent to both the referring physician and the entered a market characterized by significant patient.

244 Chapter 10. Recruiting and Retaining Participants in the Registry

Case Example 20. Building value as a means to Key Point recruit hospitals (continued) Nonfinancial incentives that meet the needs of Results decisionmakers can assist in recruitment of sites. By providing a mix of innovative nonfinancial When creating such incentives, consider both incentives, the program increased both enrollment tangible and nontangible benefits. and advancement by about one-third in 12 For More Information months. Currently, 3,150 hospitals participate in http://www.heart.org/quality the program. The database includes 4,253,461 patient records and is considered by many to be the most robust database for heart failure, stroke, and resuscitation. In 2004, the program received the Innovation in Prevention Award from the Department of Health and Human Services.

Case Example 21. Using registry tools to Proposed Solution recruit sites To increase compliance with guidelines, the Description The objective of the OPTIMIZE- registry team promoted the implementation of a HF (Organized Program to process-of-care improvement component and the Initiate Lifesaving Treatment in use of comprehensive patient education materials. Hospitalized Patients with Heart They combined these materials into a hospital Failure) registry was to improve toolkit, which included evidence-based practice quality of care and promote algorithms, critical pathways, standardized evidence-based therapies in orders, discharge checklists, pocket cards, and heart failure. The registry chart stickers. The toolkit also included provided a comprehensive algorithms and dosing guides for the guideline- process-of-care improvement recommended therapies and a comprehensive set program and gathered data that of patient education materials. The team engaged allowed hospitals to track their the steering committee in designing the toolkit to improvement over time. ensure that the materials reflected both the Sponsor GlaxoSmithKline guideline-recommended interventions and the practical aspects of hospital processes. Year Started 2003 In addition to the toolkit, the registry offered Year Ended 2005 point-of-care tools, such as referral notes and No. of Sites 270 patient letters, that could be customized for each patient based on data entered into the registry. No. of Patients More than 50,000 The registry also included real-time performance Challenge reports that hospitals could use to assess their improvement on a set of standardized measures The registry was designed to help hospitals based on the guidelines. improve care for patients hospitalized with heart failure. The objective was to accelerate the Results adoption of evidence-based guidelines and The hospital toolkit was a key component of the increase the use of the guideline-recommended registry’s marketing campaign. Hospitals could therapies, thereby improving both short-term and view the toolkit at recruitment meetings, but they long-term clinical outcomes for heart failure did not receive their own copy until they joined patients. the program. The toolkit gained credibility among

245 Section III. Operating Registries

Case Example 21. Using registry tools to For More Information recruit sites (continued) Fonarow GG, Abraham WT, Albert NM, et al. Results (continued) Organized Program to Initiate Lifesaving hospitals because its creators included some of Treatment in Hospitalized Patients with Heart the most prominent members of the heart failure Failure (OPTIMIZE-HF): rationale and design. research and treatment community. Hospitals also Am Heart J. 2004;148(1):43–51. actively used the reports to track their Gheorghiade M, She L, Abraham WT. et al. improvement over time and identify areas for Systolic blood pressure at admission, clinical additional work. Overall, the registry recruited characteristics, and outcomes in patients 270 hospitals and met its patient accrual goal six hospitalized with acute heart failure. JAMA. months ahead of schedule. 2006;296:2217–26. Key Point Fonarow GC, Abraham WT, Albert NM. et al. Nonfinancial incentives, such as patient education Association between performance measures and materials, toolkits, and reports, can encourage clinical outcomes for patients hospitalized with sites to join a registry. Incentives that also add heart failure. JAMA. 2007;297:61–70. value for the site by improving their processes or providing materials that they use frequently can aid retention.

246 Chapter 10. Recruiting and Retaining Participants in the Registry

Case Example 22. Using a scientific advisory to facilitate research among the community board to support investigator research investigators, both to increase interest in the projects registry and to increase the scope of research Description The National LymphoCare questions addressed using registry data. Study is a large, prospective, Proposed Solution disease-based registry in the The registry sponsors and the SAB developed a area of follicular lymphoma in plan to allow investigators at enrolling sites to the United States. There are a propose a question of interest; work with an SAB number of open clinical member, clinical scientists, epidemiologists, and questions about follicular biostatisticians to develop an analysis plan to lymphoma treatment, including answer the question; and present findings at whether anthracyclines should scientific meetings. The plan was implemented in be used early in the course of 2007, when the registry issued a call for research disease, and whether there is a proposals to all participating investigators. The group of patients for whom proposal outlined the types of data that were observation (as opposed to available at that point (e.g., descriptive data on active treatment) is the best demographics, initial treatments, etc.). Several choice, given the indolent nature community-based investigators sent in proposals, of the disease. The registry which the SAB then reviewed. The SAB selected follows patients for up to 10 the proposals that it felt were most appropriate years, and specific outcomes of for the available data and that answered the most interest include overall response valuable questions from a clinical standpoint. rate, progression-free survival, time to subsequent therapy, and The community investigator for each selected overall survival for common proposal was then paired with a member of the frontline and subsequent SAB to further develop the research question. therapeutic strategies. This process included conference calls and emails to refine the question and the high-level analytic Sponsor Genentech, Inc., and Biogen plan. Once the plan was ready, the investigator Idec, Inc. and the SAB member submitted the proposal and Year Started 2004 analytic plan to the registry sponsor. The sponsor Year Ended Ongoing provided support for analytic design and biostatistics. The investigator, in consultation No. of Sites 250 community and academic with the SAB member, developed an abstract sites based on the results. Abstracts were reviewed by No. of Patients Over 2,700 the full SAB before being submitted for presentation. Challenge Results The National LymphoCare Study includes a large number of community-based sites in addition to In 2007, a community-based investigator project many academic sites. Many of the principal developed through this process was accepted for investigators at the community-based sites are abstract presentation at the annual American interested in using the registry data to answer Society of Hematology (ASH) meeting. In 2009, clinical questions, but they do not have sufficient a community-based investigator and a fellow at research experience to design a research question, an academic institution developed abstracts that conduct data analysis, and share the results with have been submitted for presentation at the the scientific community. One aim of the registry annual ASH meeting. sponsors and scientific advisory board (SAB) is

247 Section III. Operating Registries

Case Example 22. Using a scientific advisory For More Information board to support investigator research Friedberg JW, Taylor MD, Cerhan JR. et al. projects (continued) Follicular lymphoma in the United States: first Results (continued) report of the National LymphoCare Study. J Clin With outcomes data now available in the registry, Oncol. 2009;27:1202–8. registry sponsors plan to issue calls for proposals Friedberg JW, Wong EK, Taylor MD, et al. twice per year, with the goal of generating Characteristics of patients with stage I follicular abstracts for the annual ASH meeting and the lymphoma (FL) selected for watchful waiting annual American Society of Clinical Oncology (WW) in the US: report from the National meeting. To date, the research program has been LymphoCare Study (NLCS). American Society of well received by community-based investigators, Hematology; 2007. Abstract 3315. who have the opportunity to author their own Link BK, Taylor MD, Brooks JM, et al. research projects with mentoring from an Correlates of treatment intensity for initial experienced advisor. The SAB has also been management of follicular lymphoma (FL) in the enthusiastic about working with community- United States: report from the National based physicians on research methodology and LymphoCare Study (NLCS). American Society of adding to the scientific knowledge about this Hematology; 2007. Abstract 2612. disease. Matasar MJ, Saxena R, Wong EK, et al. Practice Key Point patterns in the diagnosis of follicular lymphoma Community-based investigators who participate (FL): report from the National LymphoCare in a registry may be interested in pursuing Study (NLCS). American Society of Hematology; research opportunities but may not have all of the 2007. Abstract 2613. necessary resources or expertise. By utilizing an Nabhan C, Morawa E, Bitran JD, et al. Patterns of engaged advisory board, a registry can provide care in follicular lymphoma (FL): are minorities investigators with research opportunities, being treated differently? Report from the resulting in more publications and presentations National LymphoCare Study (NLCS). American based on registry data, and potentially more Society of Hematology; 2007. Abstract 367. engaged investigators.

248 Chapter 10. Recruiting and Retaining Participants in the Registry

Case Example 23. Identifying and addressing ASPS members choosing not to participate at all, recruitment and retention barriers in an and some ASPS members who choose to stop ongoing registry participating or participate sporadically. In order Description Tracking Operations and to encourage broad and continued participation Outcomes for Plastic Surgeons from society members, the society sought to fully (TOPS) is a national registry of understand the needs of its members and develop plastic surgery procedures and strategies to retain and recruit participants for the outcomes used to track and ongoing registry. assess 30-day post-operative Proposed Solution outcomes. To improve the program’s value to member Sponsor American Society of Plastic surgeons, the ASPS regularly evaluates member Surgeons (ASPS) and organization needs to make upgrades and Year Started 2002 improve the registry. The ASPS surveyed its society membership in 2008 and 2012 to better Year Ended Ongoing understand reasons for both continued No. of Sites 425–450 annually participation and lack of use. Survey results indicated that earning credits towards continuing No. of Patients 611,682 complete cases and medical education, society-awarded patient safety 1,094,268 plastic surgery credits, and contributing to important scientific procedures endeavors were strong reasons for continued Challenge participation. In addition, individual physician practices perceived value in the registry’s ability TOPS was initially launched in 2002 as a to track patients’ outcomes over time and program designed to provide ASPS member benchmark practice data against other practices in plastic surgeons with a mechanism to track the registry. Many nonusers reported their demographic, procedural, and outcomes willingness to participate if the registry supported information to help physicians benchmark and integration with their practice’s electronic evaluate their practice. The registry uses an medical records (EMR) system. Other reasons for electronic data capture interface to collect nonparticipation included perceived limited common demographic, risk factor, procedural, usefulness of data collection to physician practice and 30-day outcome data elements, which allow and site-based barriers such as insufficient registry users to evaluate outcomes based on resources. patient comorbidities and risk factors and track the rate and type of surgical incidences that could Results occur postoperatively. The registry has become an Based on the survey feedback, the ASPS elected integral part of ASPS efforts and is used in many to increase functionality to make the registry of the society’s key initiatives, including more relevant to members. Updates to the registry developing evidence-based practice parameters, were made in 2007, 2009, and 2011, and included monitoring clinical outcomes and emerging the introduction of credentialing reports, trends, supporting research and educational enhanced benchmarking reports, collection of programs, and compilation of the National patient-reported outcomes (PRO), and enhanced Clearinghouse Plastic Surgery Statistics. data entry and querying reporting functions. The A majority of the registry participants contribute credentialing reports allow registry users to data annually. However, due to the voluntary review their individual patient data by facility, nature of the registry, the registry also includes with options to filter by medical record number, “one-time” users. Since 2002, more than 1,600 name, date range, procedure type, and outcome. ASPS members have participated in the registry The existing benchmarking reports were at any given time, leaving a large number of enhanced, allowing users to benchmark their

249 Section III. Operating Registries

Case Example 23. Identifying and addressing member requests, the upgrades also included new recruitment and retention barriers in an functionality to support data transfer from EMR ongoing registry (continued) and practice management programs to ease data Results (continued) entry burden on sites. individual data against aggregate registry data; Key Point customizations were also added, allowing users to Within an ongoing, voluntary registry, retaining filter by time period and procedure type. The and recruiting participants requires maintaining registry also began electronically collecting PRO relevance to users. Surveys or other methods of data from breast augmentation and reconstruction collecting feedback from registry participants and patients using the BREAST-Q© questionnaire; the potential participants can be useful tools for PRO data supports metrics for documenting discovering recruitment or retention barriers and clinical performance appraisal and improvement identifying potential improvements to maintain based on the patients’ responses. Users can relevance. develop PRO reports and create dynamic For More Information customized graphs and charts in order to assess patient satisfaction across PRO domains, such as http://www.plasticsurgery.org/for-medical- satisfaction with outcome, psychosocial well- professionals/surgeon-community/tops.html being, and satisfaction with care. In response to

References for Chapter 10 4. Gheorghiade M, Abraham WT, Albert NM, et al. Systolic blood pressure at admission, clinical 1. Scios. “Scios Announces ADHERE Heart Failure characteristics, and outcomes in patients Registry Meets JCAHO Standards for Hospital hospitalized with acute heart failure. JAMA. 2006 Accreditation.” http://www.prnewswire.com/ Nov 8;296(18):2217-26. PMID: 17090768. news-releases/scios-announces-adhere-heart- 5. Spertus JA, Peterson E, Rumsfeld JS, et al. The failure-registry-meets-jcaho-standards-for- Prospective Registry Evaluating Myocardial hospital-accreditation-77976192.html. Accessed Infarction: Events and Recovery (PREMIER)-- August 15, 2012. evaluating the impact of myocardial infarction on 2. Centers for Medicare & Medicaid Services. patient outcomes. Am Heart J. 2006 Medicare Coverage Database. http://www.cms. Mar;151(3):589-97. PMID: 16504619. gov/medicare-coverage-database/overview-and- quick-search.aspx?id=8. Accessed August 15, 2012. 3. Dokholyan RS, Muhlbaier LH, Falletta JM, et al. Regulatory and ethical considerations for linking clinical and administrative databases. Am Heart J. 2009 Jun;157(6):971-82. PMID: 19464406.

250 Chapter 11. Data Collection and Quality Assurance

1. Introduction purposes of the registry. This chapter describes a broad range of centralized and distributed data This chapter focuses on data collection procedures collection and quality assurance activities and quality assurance principles for patient currently in use or expected to become more registries. Data management—the integrated commonly used in patient registries. system for collecting, cleaning, storing, monitoring, reviewing, and reporting on registry 2. Data Collection data—determines the utility of the data for meeting the goals of the registry. Quality 2.1 Database Requirements and Case assurance, on the other hand, aims to assure that Report Forms the data were, in fact, collected in accordance with these procedures and that the data stored in the Chapter 1 defined key characteristics of patient registry database meet the requisite standards of registries for evaluating patient outcomes. They quality, which are generally defined based on the include specific and consistent data definitions for intended purposes. In this chapter, the term collecting data elements in a uniform manner for registry coordinating activities refers to the every patient. As in randomized controlled trials, centralized procedures performed for a registry, the case report form (CRF) is the paradigm for the and the term registry coordinating center refers to data structure of the registry. A CRF is a formatted the entity or entities performing these procedures listing of data elements that can be presented in and overseeing the registry activities at the site and paper or electronic formats. Those data elements patient levels. and data entry options in a CRF are represented in the database schema of the registry by patient-level Because the range of registry purposes can be variables. Defining the registry CRFs and broad, a similar range of data collection procedures corresponding database schema are the first steps may be acceptable, but only certain methodologies in data collection for a registry. Chapter 4 may be suitable for particular purposes. describes the selection of data elements for a Furthermore, certain end users of the data may registry. require that data collection or validation be performed in accordance with their own guidelines Two related documents should also be considered or standards. For example, a registry that collects part of the database specification: the data data electronically and intends for those data to be dictionary (including data definitions and used by the U.S. Food and Drug Administration parameters) and the data validation rules, also (FDA) should meet the systems validation known as queries or edit checks. The data requirements of that end user of the data, such as dictionary and definitions describe both the data Title 21 of the Code of Federal Regulations Part 11 elements and how those data elements are (21 CFR Part 11). Such requirements may have a interpreted. The data dictionary contains a detailed substantial effect on the registry procedures. description of each variable used by the registry, Similarly, registries may be subject to specific including the source of the variable, coding processes depending on the type of data collected, information if used, and normal ranges if relevant. the types of authorization obtained, and the For example, the term “current smoker” should be applicable governmental regulations. defined as to whether “smoker” refers to tobacco or other substances and whether “current” refers to Requirements for data collection and quality active or within a recent time period. Several assurance should be defined during the registry cardiovascular registries, such as the Get With The inception and creation phases. Certain Guidelines® Coronary Artery Disease1 program requirements may have significant cost define “current smoker” as someone who smoked implications, and these should be assessed on a tobacco within the last year. cost-to-benefit basis in the context of the intended

251 Section III. Operating Registries

Data validation rules refer to the logical checks on For example, some data sources may or may not be data entered into the database against predefined available for all patients. Chapter 4, Section 5 rules for either value ranges (e.g., systolic blood discusses pilot testing in more detail. pressure less than 300 mmHg) or logical 2.2.2 Documentation of Procedures consistency with respect to other data fields for the same patient; these are described more fully in The data collection procedures for each registry Section 2.5, “Cleaning Data,” below. While neither should be clearly defined and described in a registry database structures nor database detailed manual. The term manual here refers to requirements are standardized, the Clinical Data the reference information in any appropriate form, Interchange Standards Consortium2 is actively including hard copy, electronic, or via interactive working on representative models of data Web or software-based systems. Although the interchange and portability using standardized detail of this manual may vary from registry to concepts and formats. Chapter 4 further discusses registry depending on the intended purpose, the these models, which are applicable to registries as required information generally includes protocols, well as clinical trials. policies, and procedures; the data collection instrument; and a listing of all the data elements 2.2 Procedures, Personnel, and Data and their full definitions. If the registry has Sources optional fields (i.e., fields that do not have to be Data collection procedures need to be carefully completed on every patient), these should be considered in planning the operations of a registry. clearly specified. Successful registries depend on a sustainable In addition to patient inclusion and exclusion workflow model that can be integrated into the criteria, the screening process should be specified, day-to-day clinical practice of active physicians, as should any documentation to be retained at the nurses, pharmacists, and patients, with minimal site level and any plans for monitoring or auditing disruption. (See Chapter 10.) Programs can benefit of screening practices. If sampling is to be tremendously from preliminary input from the performed, the method or systems used should be health care workers or study coordinators who are explained, and tools should be provided to simplify likely to be participants. this process for the sites. The manual should 2.2.1 Pilot Testing clearly explain how patient identification numbers are created or assigned and how duplicate records One method of gathering input from likely should be prevented. Any required training for data participants before the full launch of a registry is collectors should also be described. pilot testing. Whereas feasibility testing, which is discussed in Chapter 2, Section 2.4, focuses on If paper CRFs are used, the manual should whether a registry should be implemented, pilot describe specifically how they are used and which testing focuses on how it should be implemented. parts of the forms (e.g., two-part or three-part Piloting can range from testing a subset of the no-carbon-required forms) should be retained, procedures, CRFs, or data capture systems, to a copied, submitted, or archived. If electronic CRFs full launch of the registry at a limited subset of are used, clear user manuals and instructions sites with a limited number of patients. should be available. These procedures are an important resource for all personnel involved in The key to effective pilot testing is to conduct it at the registry (and for external auditors who might a point where the results of the pilot can still be be asked to assure the quality of the registry). used to modify the registry implementation. Through pilot testing, one can assess The importance of standardizing procedures to comprehension, acceptance, feasibility, and other ensure that the registry uses uniform and factors that influence how readily the patient systematic methods for collecting data cannot be registry processes will fit into patient lifestyles and overstated. At the same time, some level of the normal practices of the health care provider. customization of data entry methods may be required or permitted to enable the participation of

252 Chapter 11. Data Collection and Quality Assurance

particular sites or subgroups of patients within originally collected for purposes other than the some practices. As discussed in Chapter 10, if the registry (e.g., standard medical care, insurance registry provides payments to sites for claims processing). The sections below incorporate participation, then the specific requirements for and expand on these definitions. site payments should be clearly documented, and 2.2.5 Patient-Reported Data this information should be provided with the registry documents. Patient-reported data are data specifically collected from the patient for the purposes of the registry 2.2.3 Personnel rather than interpreted through a clinician or an All personnel involved in data collection should be indirect data source (e.g., laboratory value, identified, and their job descriptions and respective pharmacy records). Such data may range from roles in data collection and processing should be basic demographic information to validated scales described. Examples of such “roles” include of patient-reported outcomes (PROs). From an patient, physician, data entry personnel, site operational perspective, a wide range of issues coordinator, help desk, data manager, and monitor. should be considered in obtaining data directly The necessary documentation or qualification from patients. These range from presentation required for any role should be specified in the (e.g., font size, language, reading level) to registry documentation. As an example, some technologies (e.g., paper-and-pencil registries require personnel documentation such as questionnaires, computer inputs, telephone or a curriculum vitae, protocol signoff, attestation of voice inputs, or hand-held patient diaries). intent to follow registry procedures, or Mistakes at this level can inadvertently bias patient confirmation of completion of specified training. selection, invalidate certain outcomes, or 2.2.4 Data Sources significantly affect cost. Limiting the access for patient reporting to particular languages or The sources of data for a registry may include new technologies may limit participation. Patients with information collected from the patient, new or specific diagnoses may have difficulties with existing information reported by or derived from specific technologies (e.g., small font size for the clinician and the medical record, and ancillary visually impaired, paper and pencil for those with stores of patient information, such as laboratory rheumatoid arthritis). Other choices, such as results. Since registries for evaluating patient providing a PRO instrument in a format or method outcomes should employ uniform and systematic of delivery that differs from how it was validated methods of data collection, all data-related (e.g., questionnaire rather than interview), may procedures—including the permitted sources of invalidate the results. For more information on data; the data elements and their definitions; and patient-reported outcome development and use, the validity, reliability, or other quality see Chapter 5. requirements for the data collected from each source—should be predetermined and defined for 2.2.6 Clinician-Reported Data all collectors of data. As described in Section 3, Clinician-reported or -derived data can also be “Quality Assurance,” below, data quality is divided into primary and secondary subcategories. dependent on the entire chain of data collection As an example, specific clinician rating scales and processing. Therefore, the validity and quality (e.g., the National Institutes of Health Stroke of the registry data as a whole ultimately derive Scale)3 may be required for the registry but not from the least, not the most, rigorous link. routinely captured in clinical encounters. Some In Chapter 6, data sources are classified as primary variables might be collected directly by the or secondary, based on the relationship of the data clinician for the registry or obtained from the to the registry purpose and protocol. Primary data medical record. Data elements that the clinician sources incorporate data collected for direct must collect directly (e.g., because of a particular purposes of the registry (i.e., primarily for the definition or need to assess a specific comorbidity registry). Secondary data sources consist of data that may or may not be routinely present in the

253 Section III. Operating Registries

medical record) should be specified. These may need to decipher illegible handwriting, designations are important because they determine translate obscure abbreviations and acronyms, and who can collect the data for a particular registry or understand the clinical content to sufficiently what changes must be made in the procedures the extract the desired information. Registry personnel clinician follows in recording a medical record for should develop formal chart abstraction guidelines, a patient in a registry. Furthermore, the types of documentation of processes and practical error that arise in registries (discussed in Section definitions of terms, and coding forms for the 3, “Quality Assurance”) will differ by the degree analysts and reviewers to use. of use of primary and secondary sources, as well Generally, the guidelines include instructions to as other factors. As an example, registries that use search for particular types of data that will go into medical chart abstracters, as discussed in Section the registry (e.g., specific diagnoses or laboratory 2.2.7 below, may be subject to more interpretive 4 results). Often the analyst will be asked to code the errors. data, using either standardized codes from a 2.2.7 Data Abstraction codebook (e.g., the ICD-9 [International Data abstraction is the process by which a data Classification of Diseases, 9th Revision] code) collector other than the clinician interacting with corresponding to a text diagnosis in a chart, or the patient extracts clinician-reported data. While codes that may be unique to the registry (e.g., a physical examination findings, such as height and severity scale of 1 to 5). weight, or laboratory findings, such as white blood All abstraction and coding instructions must be cell counts, are straightforward, abstraction usually carefully documented and incorporated into a data involves varying degrees of judgment and dictionary for the registry. Because of the “noise” interpretation. in unstructured, hard-copy documents (e.g., Clarity of description and standardization of spurious marks or illegible writing) and the lack of definitions are essential to the assurance of data precision in natural language, the clinical data quality and to the prevention of interpretive errors abstracted by different abstracters from the same when using data abstraction. Knowledgeable documents may differ. This is a potential source of registry personnel should be designated as error in a registry. resources for the data collectors in the field, and To reduce the potential for this source of error, processes should be put in place to allow the data registries should ensure proper training on the collectors in the field continuous access to these registry protocol and procedures, condition(s), data designated registry personnel for questions on sources, data collection systems, and most specific definitions and clinical situations. importantly, data definitions and their Registries that span long periods, such as those interpretation. While training should be provided intended for surveillance, might be well served by for all registry personnel, it is particularly a structure that permits the review of definitions on important for nonclinician data abstracters. a periodic basis to ensure the timeliness and Training time depends on the nature of the source completeness of data elements and definitions, and (charts or CRFs), complexity of the data, and to add new data elements and definitions. A new number of data items. A variety of training product or procedure introduced after the start of a methods, from live meetings to online meetings to registry is a common reason for such an update. interactive multimedia recordings, have all been 5 Abstracting data from unformatted hard copy used with success. Training often includes test (e.g., a hospital chart) is often an arduous and abstractions using sample charts. For some tedious process, especially if free text is involved, purposes, it is best practice to train abstracters and it usually requires a human reader. The reader, using standardized test charts. Such standardized whose qualifications may range from a trained tests can be further used both to obtain data on the “medical record analyst” or other health inter-rater reliability of the CRFs, definitions, and professional to an untrained research assistant, coding instructions and to determine whether individual abstracters can perform up to a defined

254 Chapter 11. Data Collection and Quality Assurance

minimum standard for the registry. Registries that the EMR system is usually interfaced with rely on medical chart abstraction should consider ancillary systems (discussed below), such as reporting on the performance characteristics laboratory, pharmacy, radiology, and pathology associated with abstraction, such as inter-rater systems. Ancillary systems, which usually have reliability.6 Examining and reporting on intra-rater their own databases, export relevant patient data to reliability may also be useful. Some key the EMR system, which imports the data into its considerations in standardizing medical chart database. abstractions are— Since EMRs include the majority of clinical data • Standardized materials (e.g., definitions, available about a patient, they can be a major instructions) source of patient information for a registry. What • Standardized training an EMR usually does not include is registry- specific (primary source) data that are collected • Testing with standardized charts separately from hard-copy or electronic forms. In • Reporting of inter-rater reliability the next several years, suitable EMR system 2.2.8 Electronic Medical Record interfaces may be able to present data needed by registries in accordance with registry-specified An electronic medical record (EMR) is an requirements, either within the EMR (which then electronic record of health-related information on populates the registry) or in an electronic data an individual that can be created, gathered, capture system (which then populates the EMR). managed, and consulted by authorized clinicians EMRs already serve as secondary data sources in and staff within one health care organization. More some registries, and this practice will continue to complete than an EMR, an electronic health record grow as EMRs become more widely used. In these (EHR) is an electronic record of health-related situations, data may be extracted from the EMR, information on an individual that conforms to transformed into registry format, and loaded into nationally recognized interoperability standards the registry, where they will reside in the registry and that can be created, managed, and consulted by database together with registry-specific data authorized clinicians and staff across more than imported from other sources. In a sense, this is 7 one health care organization. For the purposes of similar to medical chart abstraction except that it is this discussion, we will refer to the more limited performed electronically. capabilities of the EMR. Electronic capture differs from manual medical The EMR (and EHR) will play an increasingly chart abstraction in two key respects. First, the important role as a source of clinical data for data are “abstracted” once for all records. In this registries. The medical community is currently in a context, abstraction refers to the mapping and transition period in which the primary repository other decisionmaking needed to bring the EMR of a patient’s medical record is changing from the data into the registry database. It does not traditional hard-copy chart to the EMR. The main eliminate the potential for interpretive errors, as function of the EMR is to aggregate all clinical described later in this chapter, but it centralizes electronic data about a patient into one database, that process, making the rules clear and easily in the same way that a hard-copy medical chart reviewed. Second, the data are uploaded aggregates paper records from various personnel electronically, eliminating duplicative data entry, and departments responsible for the care of the potential errors associated with data reentry, and patient. Depending on the extent of the related cost of this redundant effort. implementation, the EMR may include patient demographics, diagnoses, procedures, progress When the EMR is used as a data source for a notes, orders, flow sheets, medications, and registry, a significant problem occurs when the allergies. The primary sources of data for the EMR information needed by the registry is stored in the are the health care providers. Data may be entered EMR as free text, rather than codified or structured into the EMR through keyboards or touch screens data. Examples of structured data include ICD-9 in medical offices or at the bedside. In addition, diagnoses and laboratory results. In contrast,

255 Section III. Operating Registries

physician progress notes, consultations, radiology focused on problems of data syntax and semantics. reports, et cetera, are usually dictated and The adoption of common database structures and transcribed as narrative free text. While data open interoperability standards will be critical for abstraction of free text derived from an EMR can future interchange between EHRs and registries. be done by a medical record analyst, with the This topic is discussed in depth in Chapter 15. increasing use of EMRs, automated methods of 2.2.9 Other Data Sources data abstraction from free text have been developed. Natural language processing (NLP) is Some of the clinical data used to populate the term for this technology. It allows computers to registries may be derived from repositories other process and extract information from human than EMRs. Examples of other data sources language. The goal of NLP is to parse free text into include billing systems, laboratory databases, and meaningful components based on a set of rules and other registries. Chapter 6 discusses the potential a vocabulary that enable the software to recognize uses of other data sources in more detail. key words, understand grammatical constructions, and resolve word ambiguities. Those components 2.3 Data Entry Systems can be extracted and delivered to the registry along Once the primary and any secondary data sources with structured data extracted from the EMR, and for a registry have been identified, the registry both can be stored as structured data in the registry team can determine how data will be entered into database. the registry database. Many techniques and An increasing number of NLP software packages technologies exist for entering or moving data into are available (e.g., caTIES from the National the registry database, including paper CRFs, direct Cancer Institute,8 i2b2 (Informatics for Integrating data entry, facsimile or scanning systems, Biology and the Bedside),9 and a number of interactive voice response systems, and electronic commercial products). However, NLP is still in an CRFs. There are also different models for how early phase of development and cannot yet be used quickly those data reach a central repository for for all-purpose chart abstraction. In general, NLP cleaning, reviewing, monitoring, or reporting. software operates in specific clinical domains (e.g., Each approach has advantages and limitations, and radiology, pathology), whose vocabularies have each registry must balance flexibility (the number been included in the NLP software’s database. of options available) with data availability (when Nevertheless, NLP has been used successfully to the central repository is populated), data validity extract diagnoses and drug names from free text in (whether all methods are equally able to produce various clinical settings. clean data), and cost. Appropriate decisions depend on many factors, including the number of It is anticipated that EMR/EHR use will grow data elements, number of sites, location (local significantly with the incentives provided under the preferences that vary by country, language American Recovery and Reinvestment Act of 2009 differences, and availability of different health information technology provisions. technologies), registry duration, followup Currently, only a minority of U.S. patients have frequency, and available resources. their data stored in systems that are capable of retrieval at the level of a data element. 2.3.1 Paper CRFs Furthermore, only a small number of these With paper CRFs, the clinician enters clinical data systems currently store data in structured formats on the paper form at the time of the clinical with standardized data definitions for those data encounter, or other data collectors abstract the data elements that are common across different from medical records after the clinical encounter. vendors. A significant amount of attention is CRFs may include a wide variety of clinical data currently focused on interchange formats between on each patient gathered from different sources clinical and research systems (e.g., from Health (e.g., medical chart, laboratory, pharmacy) and Level Seven [HL-7]10 to Clinical Data Interchange from multiple patient encounters. Before the data Standards Consortium2 models). Attention is also on formatted paper forms are entered into a

256 Chapter 11. Data Collection and Quality Assurance

computer, the forms should be reviewed for translation errors, and spurious marks on the page completeness, accuracy, and validity. Paper CRFs can cause errors. Error checking is based on can be entered into the database by either direct automated parameters specified by the operator of data entry or computerized data entry via scanning the system for exception handling. The comments systems. on assessing error rates in the section above are With direct data entry, a computer keyboard is applicable for scanning systems as well. used to enter data into a database. Key entry has a 2.3.2 Electronic CRFs variable error rate depending on personnel, so an An electronic CRF (eCRF) is defined as an assessment of error rate is usually desirable, auditable electronic form designed to record particularly when a high volume of data entry is information required by the clinical trial protocol performed. Double data entry is a method of to be reported to the sponsor on each trial increasing the accuracy of manually entered data subject.11 An eCRF allows clinician-reported data by quantifying error rates as discrepancies between to be entered directly into the electronic system by two different data entry personnel; data accuracy is the data collector (the clinician or other data improved by having up to two individuals enter the collector). Site personnel in many registries still data and a third person review and manage commonly complete an intermediate hard-copy discrepancies. With upfront data validation checks worksheet representing the CRF and subsequently on direct data entry, the likelihood of data entry enter the data into the eCRF. While this approach errors significantly decreases. Therefore, the increases work effort and error rates, it is still in choice of single versus double data entry should be use because it is not yet practical for all electronic driven by the requirements of the registry for a data entry to be performed at the bedside, during particular maximal error rate and the ability of the clinical encounter, or in the midst of a busy each method to achieve that rate in key measures clinical day. in the particular circumstance. Double data entry, while a standard of practice for registrational trials, An eCRF may originate on local systems may add significant cost. Its use should be guided (including those on an individual computer, a local by the need to reduce an error rate in key measures area network server, or a hand-held device) or and the likelihood of accomplishing that by double directly from a central database server via an data entry as opposed to other approaches. In some Internet-based connection or a private network. For situations, assessing the data entry error rates by registries that exist beyond a single site, the data re-entering a sample of the data is sufficient for from the local system must subsequently reporting purposes. communicate with a central data system. An eCRF may be presented visually (e.g., computer screen) With hard-copy structured forms, entering data or aurally (e.g., telephonic data entry, such as using a scanner and special software to extract the interactive voice response systems). Specific data from the scanned image is possible. If data circumstances will favor different presentations. are recorded on a form as marks in checkboxes, For example, in one clozapine patient registry that the scanning software enables the user to map the is otherwise similar to Case Example 24, both location of each checkbox to the value of a pharmacists and physicians can obtain and enter variable represented by the text item associated data via a telephone-based interactive voice with the checkbox, and to determine whether the response system as well as a Web-based system. box is marked. The presence of a mark in a box is The option is successful in this scenario because converted by the software to its corresponding telephone access is ubiquitous in pharmacies and value, which can then be transmitted to a database the eCRF is very brief. for storage. If the form contains hand-printed or typed text or numbers, optical character A common method of electronic data entry is to recognition software is often effective in extracting use Web-based data entry forms. Such forms may the printed data from the scanned image. However, be used by patients, providers, and interviewers to the print font must be of high quality to avoid enter data into a local repository. The forms reside

257 Section III. Operating Registries

on servers, which may be located at the site of the Electronic interfaces are necessary to move data registry or co-located anywhere on the Internet. To from one computer to another. If clinical data are access a data entry form, a user on a remote entered into a local repository from an eCRF form computer with an Internet connection opens a or entered into an EMR, the data must be extracted browser window and enters the address of the Web from the source dataset in the local repository, server. Typically, a login screen is displayed and transformed into the format required by the the user enters a user identification and password, registry, and loaded into the registry database for provided by personnel responsible for the Web site permanent storage. This is called an “extract, or repository. Once the server authenticates the transform, and load” process. Unless the local user, the data entry form is displayed, and the user repository is designed to be consistent with the can begin entering data. As described in “Cleaning registry database in terms of the names of Data” (Section 2.5), many electronic systems can variables and their values, data mapping and perform data validation checks or edits at the time transformation can be a complex task. In some of data entry. When data entry is complete, the cases, manual transfer of the data may be more user submits the form, which is sent over the efficient and less time-consuming than the effort to Internet to the Web server. develop an electronic interface. Emerging open Smart phones or other mobile devices may also be standards can enable data to be transferred from an used to submit data to a server to the extent such EHR directly into the registry. This topic is transmissions can be done with appropriate discussed in more detail in Chapter 15. information security controls. Mobility has If an interface between a local electronic system recently become an important attribute for clinical and registry system is developed, it is still data collection. Software has been developed that necessary to communicate to the ancillary system enables wireless devices to collect data and the criteria for retrieval and transmission of a transmit them over the Internet to database servers patient record. Typically, the ancillary data are in fixed locations. As wireless technology maintained in a relational database, and the system continues to evolve and data transmission rates needs to run an SQL (Structured Query Language) increase, these will become more essential data query against the database to retrieve the specified entry devices for patients and clinicians. information. An SQL query may specify individual patients by an identifier (e.g., a medical record 2.4 Advantages and Disadvantages of number) or by values or ranges of specific Data Collection Technologies variables (e.g., all patients with hemoglobin A1c When the medical record or ancillary data are in over 8 mg/dl). The results of the query are usually electronic format, they may be abstracted to the stored as a file (e.g., XML, CSV, CDISC ODM) CRF by a data collector or, in some cases, that can be transformed and transferred to the uploaded electronically to the registry database. registry system across the interface. A variety of The ease of extracting data from electronic interface protocols may be used to transfer the systems for use in a registry depends on the design data. of the interfaces of ancillary and registry systems, Because data definitions and formats are not yet and the ability of the EMR or ancillary system nationally standardized, transfer of data from an software to make the requested data accessible. EMR or ancillary system to a registry database is However, as system vendors increasingly adopt prone to error. Careful evaluation of the transfer open standards for interoperability, transferring specifications for interpretive or mapping errors is data from one system to another will likely a critical step that the registry coordinating center become easier. Many organizations are actively should verify. Furthermore, a series of test working toward improved standards, including transfers and validation procedures should be HL7,10 the National eHealth Collaborative,12 the performed and documented. Finally, error National Institute of Standards and Technology,13 checking must be part of the transfer process and others. Chapter 15 describes standards and because new formats or other errors not in the test certifications specific to EHR systems. databases may be introduced during actual

258 Chapter 11. Data Collection and Quality Assurance

practice, and these need to be identified and 2.5.1 Data Management Manual isolated from the registry itself. Even though each Data managers should develop formal data review piece of data may be accurately transferred, the guidelines for the reviewers and data entry data may have different representations on the personnel to use. The guidelines should include different systems (e.g., value discrepancies such as information on how to handle missing data; invalid the meaning of “0” vs. “1,” fixed vs. floating point entries (e.g., multiple selections in a single-choice numbers, date format, integer length, and missing field, alphabetic data in a numeric field); erroneous values). In summary, any system used to extract entries (e.g., patients of the wrong gender EMR records into registry databases should be answering gender-based questions); and validated and should include an interval sampling inconsistent data (e.g., an answer to one question of transfers to ensure that uploading of this contradicting the answer to another one). The information is consistent over time. guidelines should also include procedures to The ancillary system must also notify the registry attempt to remediate these data problems. For when an error correction occurs in a record already example, with a data error on an interview form, it transferred to the registry. Registry software must may be necessary to query the interviewer or the be able to receive that notification, flag the patient, or to refer to other data sources that may erroneous value as invalid, and insert the new, be able to resolve the problem. Documentation of corrected value into its database. Finally, it is any data review activity and remediation efforts, important to recognize that the use of an including dates, times, and results of the query, electronic-to-electronic interchange requires not should be maintained. only testing but also validation of the integrity and 2.5.2 Automated Data Cleaning quality of the data transferred. Few ancillary systems or EMR systems are currently validated to Ideally, automated data checks are preprogrammed a defined standard. For registries that intend to into the database for presentation at the time of report data to FDA or to other sponsors or data data entry. These data checks are particularly recipients with similar requirements, including useful for cleaning data at the site level while the electronic signatures, audit trails, and rigorous patient or medical record is readily accessible. system validation, the ways in which the registry Even relatively simple edit checks, such as range interacts with these other systems must be values for laboratories, can have a significant carefully considered. effect on improving the quality of data. Many systems allow for the implementation of more 2.5 Cleaning Data complex data edit checks, and these checks can Data cleaning refers to the correction or substantially reduce the amount of subsequent amelioration of data problems, including missing manual data cleaning. A variation of this method is values, incorrect or out-of-range values, responses to use data cleaning rules to deactivate certain data that are logically inconsistent with other responses fields so that erroneous entries cannot even be in the database, and duplicate patient records. made. A combination of these approaches can also While all registries strive for “clean data,” in be used. For paper-based entry methods, reality, this is a relative term. How and to what automated data checks are not available at the time level the data will be cleaned should be addressed the paper CRF is being completed but can be upfront in a data management manual that incorporated when the data are later entered into identifies the data elements that are intended to be the database. cleaned, describes the data validation rules or 2.5.3 Manual Data Cleaning logical checks for out-of-range values, explains Data managers perform manual data checks or how missing values and values that are logically queries to review data for unexpected inconsistent will be handled, and discusses how discrepancies. This is the standard approach to duplicate patient records will be identified and cleaning data that are not entered into the database managed. at the site (e.g., for paper CRFs entered via data

259 Section III. Operating Registries

entry or scanning). By carefully reviewing the data rates, the registry coordinating center can identify using both data extracts analyzed by algorithms problems and potentially take corrective action— and hand review, data managers identify either at individual sites or across the registry as a discrepancies and generate “queries” to send to the whole. sites to resolve. Even eCRF-based data entry with 2.5.6 Coding Data data validation rules may not be fully adequate to ensure data cleaning for certain purposes. As further described in Chapter 4, the use of Anticipating all potential data discrepancies at the standardized coding dictionaries is an increasingly time that the data management manual and edit important tool in the ability to aggregate registry checks are developed is very difficult. Therefore, data with other databases. As the health even with the use of automated data validation information community adopts standards, parameters, some manual cleaning is often still registries should routinely apply them unless there performed. are specific reasons not to use such standard codes. While such codes should be implemented in the 2.5.4 Query Reports data dictionaries during registry planning, The registry coordinating center should generate, including all codes in the interface is not always on a periodic basis, query reports that relate to the possible. Some free text may be entered as a result. quality of the data received, based on the data When free text data are entered into a registry, management manual and, for some purposes, recoding these data using standardized dictionaries additional concurrent review by a data manager. (e.g., MedDRA, WHODRUG, SNOMED®) may The content of these reports will differ depending be worthwhile. There is cost associated with on what type of data cleaning is required for the recoding, and in general, it should be limited to registry purpose and how much automated data data elements that will be used in analysis or that cleaning has already been performed. Query need to be combined or reconciled with other reports may include missing data, “out-of-range” datasets, such as when a common safety database data, or data that appear to be inconsistent (e.g., is maintained across multiple registries and positive pregnancy test for a male patient). They studies. may also identify abnormal trends in data, such as 2.5.7 Storing and Securing Data sudden increases or decreases in laboratory tests compared with patient historical averages or When data on a form are entered into a computer clinically established normal ranges. Qualified for inclusion in a registry, the form itself, as well registry personnel should be responsible for as a log of the data entered, should be maintained reviewing the abnormal trends with designated site for the regulatory archival period. Data errors may personnel. The most effective approach is for sites be discovered long after the data have been stored to provide one contact representative for purposes in the registry. The error may have been made by of queries or concerns by registry personnel. the patient or interviewer on the original form or Depending on the availability of the records and during the data entry process. Examination of the resources at the site to review and respond to original form and the data entry log should reveal queries, resolving all queries can sometimes be a the source of the error. If the error is on the form, challenge. Creating systematic approaches to correcting it may require reinterviewing the maximizing site responsiveness is recommended. patient. If the error occurred during data entry, the corrected data should be entered and the registry 2.5.5 Data Tracking updated. By then, the erroneous registry data may For most registry purposes, tracking of data have been used to generate reports or create received (paper CRFs), data entered, data cleaned, cohorts for population studies. Therefore, instead and other parameters is an important component of of simply replacing erroneous data with corrected active registry management. By comparing data, the registry system should have the ability to indicators, such as expected to observed rates of flag data as erroneous without deleting them and patient enrollment, CRF completion, and query to insert the corrected data for subsequent use.

260 Chapter 11. Data Collection and Quality Assurance

Once data are entered into the registry, the registry the conduct of a registry: internally driven change must be backed up on a regular basis. There are to refine or improve the registry or the quality of two basic types of backup, and both types should data collected, and externally driven change that be considered for use as best practice by the comes as a result of changes in the environment in registry coordinating center. The first type is which the registry is being conducted. real-time disk backup, which is done by the disk Internally driven change is generally focused on storage hardware used by the registry server. The changes to data elements or data validation second is a regular (e.g., daily) backup of the parameters that arise from site feedback, queries, registry to removable media (e.g., tape, CD-ROM, and query trends that may point to a question, DVD). In the first case, as data are stored on disk definition, or CRF field that was poorly designed in the registry server, they are automatically or missing. If this is the case, the registry can use replicated to two or more physical hard drives. In the information coming back from sites or data the simplest example, called “mirroring,” registry managers to add, delete, or modify the database data are stored on a primary disk and an exact requirements, CRFs, definitions, or data replica is stored on the mirrored disk. If either disk management manual as required. At times, more fails, data continue to be stored on the mirrored substantive changes, such as the addition of new disk until the failed disk is replaced. This failure forms or changes to the registry workflow, may be can be completely transparent to the user, who may desirable to examine new conditions or outcomes. continue entering and retrieving data from the Externally driven change generally arises in registry database during the failure. More complex multiyear registries as new information about the disk backup configurations exist, in which arrays disease and/or product under study becomes of disks are used to provide protection from single available, or as new therapies or products are disk failures. introduced into clinical practice. Change and The second type of periodic backup is needed for turnover in registry personnel is another type of disaster recovery. Ideally, a daily backup copy of change, and one that can be highly disruptive if the registry database stored on removable media procedures are not standardized and documented. should be maintained off site. In case of failure of A more extensive form of change may occur when the registry server or disaster that closes the data a registry either significantly changes its CRFs or center, the backup copy can be brought to a changes the underlying database. Longstanding functioning server and the registry database registries address this issue from time to time as restored, with the only potential loss of data being information regarding the condition or procedure for the interval between the regularly scheduled evolves and data collection forms and definitions backups. The lost data can usually be reloaded require updating. Chapter 14 discusses in more from local data repositories or re-entered from detail the process for making significant hard copy. Other advanced and widely available modifications to a registry. database solutions and disaster recovery techniques may support a “standby” database that Proper management of change is crucial to the can be located at a remote data center. In case of a maintenance of the registry. A consistent approach failure at the primary data center, the standby to change management, including decisionmaking, database can be used, minimizing downtime and documentation, data mapping, and validation, is an preventing data loss. important aspect of maintaining the quality of the registry and the validity of the data. While the 2.6 Managing Change specific change management processes might As with all other registry processes, the extent of depend on the type and nature of the registry, change management will depend on the types of change management in registries that are designed data being collected, the source(s) of the data, and to evaluate patient outcomes requires, at the very the overall timeframe of the registry. There are two least, the following structures and processes: major drivers behind the need for change during

261 Section III. Operating Registries

• Detailed manual of procedures: As described 2.7 Using Data for Care Delivery, earlier, a detailed manual that is updated on a Coordination, and Quality regular basis—containing all the registry Improvement policies, procedures, and protocols, as well as a 2.7.1 Improving Care complete data dictionary listing all the data elements and their definitions—is vital for the As registries increasingly collect data in electronic functioning of a registry. The manual is also a format, the time between care delivery and data crucial component for managing and collection is reduced. This shorter timeframe offers documenting change management in a registry. significant opportunities to use registry functionalities to improve care delivery at the • Governing body: As described in Chapter 2, patient and population levels. These functionalities Section 6, registries require oversight and (Table 11–1) include the generation of outputs that advisory bodies for a number of purposes. One promote care delivery and coordination at the of the most important is to manage change on a individual patient level (e.g., decision support, regular basis. Keeping the registry manual and patient reports, reminders, notifications, lists for data definitions up to date is one of the primary proactive care, educational content) and the responsibilities of this governing body. Large provision of tools that assist with population prospective registries, such as the National management, quality improvement, and quality Surgical Quality Improvement Program, have reporting (e.g., risk adjustment, population views, found it necessary to delegate the updating of benchmarks, quality report transmissions). A data elements and definitions to a special number of registries are designed primarily for definitions committee. these purposes. Several large national registries1, • Infrastructure for ongoing training: As 14-16 have shown large changes in performance mentioned above, change in personnel is a during the course of hospital or practice common issue for registries. Specific processes participation in the registry. For example, in one and an infrastructure for training should be head-to-head study that used hospital data from available at all times to account for any Hospital Compare, an online database created by unanticipated changes and turnover of registry the Centers for Medicare & Medicaid Services, personnel or providers who regularly enter data patients in hospitals enrolled in the American into the registry. Heart Association’s Get With The Guidelines® • Method to communicate change: Since Coronary Artery Disease registry, which includes registries frequently undergo change, there evidence-based reminders and real-time should be a standard approach and timeline for performance measurement reports, fared communicating to sites when changes will take significantly better in measures of guidelines place. compliance than those in hospitals not enrolled in 17 In addition to instituting these structures, registries the registry. should also plan for change from a budget perspective (Chapter 2) and from an analysis perspective (Chapter 13).

262 Chapter 11. Data Collection and Quality Assurance

Table 11–1. Registry functionalities

Inputs: Obtaining data • Identify/enroll representative patients (e.g., sampling). • Collect data from multiple sources and settings (providers, patients, labs, pharmacies) at key points. • Use uniform data elements and definitions (risk factors, treatments, and outcomes). • Check and correct data (validity, coding, etc.). • Link data from different sources at patient level (manage patient identifiers). • Maintain security and privacy (e.g., access control, audit trail). Outputs: Care delivery • Provide real-time feedback with decision support (evidence/guidelines). and coordination • Generate patient-level reports and reminders (longitudinal reports, care gaps, summary lists/plans, health status). • Send relevant notifications to providers and patients (care gaps, prevention support, self-management). • Share information with patients and other providers. • List patients/subgroups for proactive care. • Link to relevant patient education. Outputs: Population • Provide population-level reports. measurement and quality –– Real-time/rapid cycle improvement –– Risk adjusted –– Including standardized measures –– Including benchmarks –– Enabling different reports for different levels of users • Enable ad hoc reports for exploration. • Provide tools to manage populations or subgroups. • Generate dashboards that facilitate action. • Facilitate third-party quality reporting (transmission).

2.7.2 Special Case: Performance-Linked Access prevent severe leukopenia, or routine pregnancy System testing during thalidomide administration to A performance-linked access system (PLAS), also prevent in utero exposure to this known teratogenic known as a restricted access or limited distribution compound. Additional information on PLAS can system, is another application of a registry to serve be found in FDA’s Guidance for Industry: Development and Use of Risk Minimization Action more than an observational goal. Unlike a disease 18 and exposure registry, a PLAS is part of a detailed Plans. risk-minimization action plan that sponsors develop as a commitment to enhance the risk- 3. Quality Assurance benefit balance of a product when approved for the market. The purpose of a PLAS is to mitigate a In determining the utility of a registry for certain known drug-associated risk by ensuring decisionmaking, it is critical to understand the that product access is linked to a specific quality of the procedures used to obtain the data performance measure. Examples include systems and the quality of the data stored in the database. that monitor laboratory values, such as white blood As patient registries that meet sufficient quality cell counts during clozapine administration to criteria (discussed in Chapters 1 and 25) are

263 Section III. Operating Registries

increasingly being seen as important means to • Errors in interpretation or coding: An example generate evidence regarding effectiveness, safety, of this type of error would be two abstracters and quality of care, the quality of data within the looking for the same data element in a patient’s registry must be understood in order to evaluate its medical record but extracting different data suitability for use in decisionmaking. Registry from the same chart. Variations in coding of planners should consider how to ensure quality to specific conditions or procedures also fall a level sufficient for the intended purposes (as under the category of interpretive errors. described below) and should also consider how to Avoidance or detection of interpretive error develop appropriate quality assurance plans for includes adequate training on definitions, their registries. Those conducting the registry testing against standard charts, testing and should assess and report on those quality assurance reporting on inter-rater reliability, and re- activities. abstraction. Methods of quality assurance will vary depending • Errors in data entry, transfer, or transformation on the intended purpose of the registry. A registry accuracy: These occur when data are entered intended to serve as key evidence for into the registry inaccurately—for example, a decisionmaking19 (e.g., coverage determinations, laboratory value of 2.0 is entered as 20. product safety evaluations, or performance-based Avoidance or detection of accuracy errors can payment) will require higher levels of quality be achieved through upfront data quality assurance than a registry describing the natural checks (such as ranges and data validation history of a disease. Quality assurance activities checks), reentering samples of data to assess generally fall under three main categories: (1) for accuracy (with the percent of data to be quality assurance of data, (2) quality assurance of sampled depending on the study purpose), and registry procedures, and (3) quality assurance of rigorous attention to data cleaning. computerized systems. Since many registries are • Errors of intention: Examples of intentional large, the level of quality assurance that can be distortion of data (often referred to as obtained may be limited by budgetary constraints. “gaming”) are inflated reporting of preoperative To balance the need for sufficient quality patient risk in registries that compare risk- assurance with reasonable resource expenditure for adjusted outcomes of surgery, or selecting only a particular purpose, a risk-based approach to cases with good outcomes to report (“cherry- quality assurance is highly recommended. A picking”). Avoidance or detection of intentional risk-based approach focuses on the most important error can be challenging. Some approaches sources of error or procedural lapses from the include checking for consistency of data perspective of the registry’s purpose. Such sources between sites, assessing screening log of error should be defined during inception and information against other sources (e.g., billing design phases. As described below, registries with data), and performing onsite audits (including different purposes may be at risk for different monitoring of source records) either at random sources of error and focus on different practices or “for cause.” and levels of assessment. Standardization of Steps for assuring data quality include: methods for particular purposes (e.g., national • Training: Educate data collectors/abstracters in performance measurement) will likely become a structured manner. more common in the future if results are to be combined or compared between registries. • Data completeness: When possible, provide sites with immediate feedback on issues such 3.1 Assurance of Data Quality as missing or out-of-range values and logical inconsistencies. Structures, processes, policies, and procedures need to be put in place to ascertain the quality of • Data consistency: Compare across sites and the data in the registry and to insure against several over time. types of errors, including:

264 Chapter 11. Data Collection and Quality Assurance

• Onsite audits for a sample of sites: Review 3.1.3 Data Quality Audits screening logs and procedures and/or samples As described above, the level to which registry of data. data will be cleaned is influenced by the objectives • For-cause audits: Use both predetermined and of the registry, the type of data being collected data-informed methods to identify potential (e.g., clinical data vs. economic data), the sources sites at higher suspicion for inaccuracy or of the data (e.g., primary vs. secondary), and the intentional errors, such as discrepancies timeframe of the registry (e.g., 3-month followup between enrollment and screening logs, narrow vs. 10-year followup). These registry data ranges, and overly high or low enrollment. characteristics often affect the types and number of To further minimize or identify these errors and to data queries that are generated, both electronically ensure the overall quality of the data, the following and manually. In addition to identifying missing should be considered. values, incorrect or out-of-range values, or responses that are logically inconsistent with other 3.1.1 A Designated Individual Accountable for responses in the database, specifically trained Data Quality at Each Site registry personnel can review the data queries to Sites submitting data to a registry should have at identify possible error trends and to determine least one person who is accountable for the quality whether additional site training is required. For of these data, irrespective of whether the person is example, such personnel may identify a specific collecting the data as well. The site coordinator patient outcome question or eCRF field that is should be fully knowledgeable of all protocols, generating a larger than average proportion of policies, procedures, and definitions in a registry. queries, either from one site or across all registry The site coordinator should ensure that all site sites. Using this information, the registry personnel involved in the registry are personnel can conduct targeted followup with the knowledgeable and that all data transmitted to sites to retrain them on the correct interpretation of registry coordinating centers are valid and the outcome question or eCRF field, with the goal accurate. of reducing the future query rate on that particular 3.1.2 Assessment of Training and Maintenance of question or field. These types of “training tips” can Competency of Personnel also be addressed in a registry newsletter as a way to maintain frequent but unobtrusive Thorough training and documentation of communication with the registry sites. maintenance of competency, for both site and registry personnel, are imperative to the quality of If the registry purpose requires more stringent the registry. A detailed and comprehensive verification of the data being entered into the operations manual, as described earlier, is crucial database by registry participants, registry planners for the proper training of all personnel involved in may decide to conduct audits of the registry sites. the registry. Routine cognitive testing (surveys) of Like queries discussed above, the audit plan for a health care provider knowledge of patient registry specific registry will be influenced by the purpose requirements and appropriate product use should of the registry, the type of data being collected, the be performed to monitor maintenance of the source of the data, and the overall timeframe of the knowledge base and compliance with patient registry. In addition, registry developers must find registry requirements. Retraining programs should the appropriate balance between the extensiveness be initiated when survey results provide evidence of an audit and the impact on overall registry costs. of lack of knowledge maintenance. All registry Based on the objectives of the registry, a registry training programs should provide means by which developer can define specific data fields (e.g., key the knowledge of the data collectors about their effectiveness variables or adverse event data) on registries and their competence in data collection which the audit can be focused. can be assessed on a regular basis, particularly The term audit may describe examination or when changes in procedures or definitions are verification, may take place onsite (sometimes implemented. called monitoring) or offsite, and may be extensive

265 Section III. Operating Registries

or very limited. The audit can be conducted on a system of assessing inter-rater reliability has been random sample of participating sites (e.g., 5 to 20 devised and published; in addition to requiring that percent of registry sites); “for cause” (meaning abstracters achieve a certain level of proficiency, a only when there is an indication of a problem, such proportion of charts are scheduled for re- as one site being an outlier compared with most abstraction on the basis of predefined criteria. others); on a random sample of patients; or using Statistical measures of reliability from such sampling techniques based on geography, practice re-abstractions are maintained and reported (e.g., setting (academic center vs. community hospital), kappa statistic).21 patient enrollment rate, or query rate (“risk-based” Subsequent to audits (onsite or remote), audit strategy). communication of findings with site personnel The approach to auditing the quality of the data should be conducted face to face, along with should reflect the most significant sources of error followup written communication of findings and with respect to the purpose of the registry. For opportunities for improvement. As appropriate to example, registries used for performance meet registry objectives, the sponsor may request measurement may have a higher risk of exclusion corrective actions from the site. Site compliance of higher risk patients (“cherry-picking”), and the may also be enhanced with routine communication focus of an audit might be on external sources of of data generated from the patient registry system data to verify screening log information (e.g., to the site for reconciliation. billing data) in addition to data accuracy. (See Case Example 25.) Finally, the timeframe of the 3.2 Registry Procedures and Systems registry may help determine the audit plan. A 3.2.1 External Audits of Registry Procedures registry with a short followup period (e.g., 3 months) may require only one round of audits at If registry developers determine that external the end of the study, prior to database lock and audits are necessary to ensure the level of quality data analysis. For example, in the OPTIMIZE-HF for the specific purpose(s) of the registry, these registry, a data quality audit was performed, based audits should be conducted in accordance with on predetermined criteria, on a 5-percent random pre-established criteria. Pre-established criteria sample of the first 10,000 patient records verified could include monitoring of sites with high patient against source documents.20 For registries with enrollment or with prior audit history of findings multiyear followup, registry personnel may that require attention, or monitoring could be conduct site audits every 1 or 2 years for the based on level of site experience, rate of serious duration of the registry. adverse event reporting, or identified problems. The registry coordinating center may perform In addition to the site characteristics mentioned monitoring of a sample of sites, which could be above, sites that have undergone significant focused on one or several areas. This approach staffing changes during a multiyear registry should could range from reviewing procedures and be considered prime audit targets to help confirm interviewing site personnel, to checking screening adequate training of new personnel and to quickly logs, to monitoring individual case records. address possible inter-rater variability. To minimize any impact on the observational nature The importance of having a complete and detailed of the registry, the audit plan should be registry manual that describes policies, structures, documented in the registry manual. and procedures cannot be overemphasized in the context of quality assurance of registry procedures. Registries that are designed for the evaluation of Such a manual serves both as a basis for patient outcomes and the generation of scientific conducting the audits and as a means of information, and that use medical chart abstracters, documenting changes emanating from these should assess inter-rater reliability in data audits. As with data quality audits, feedback of the collection with sufficient scientific rigor for their findings of registry procedure audits should be intended purpose(s). For example, in one registry communicated to all stakeholders and documented that uses abstractions extensively, a detailed in the registry manual.

266 Chapter 11. Data Collection and Quality Assurance

3.2.2 Assurance of System Integrity and Security establishes the standards for security for electronic All aspects of data management processes should protected health information that must be fall under a rigorous life-cycle approach to system implemented by health plans, health care development and quality management. Each clearinghouses, and most health care providers (collectively, “covered entities”), as well as their process is clearly defined and documented. The 22 concepts described below are consistent across business associates. Therefore, covered entities many software industry standards and health care and business associates that maintain registries industry standards (e.g., 21 CFR Part 11, legal with individually identifiable health information in security standards), although some specifics may electronic form must implement the technical, vary. An internal quality assurance function at the administrative, and physical safeguards specified registry coordinating center should regularly audit and required by the HIPAA Security Rule with the processes and procedures described. When respect to the registry data. In addition, other third parties other than the registry coordinating Federal and State security laws may apply to center perform activities that interact with the registry data, depending on who maintains the registry systems and data, they are typically registry, the type of data maintained, and other assessed for risk and are subject to regular audits circumstances. by the registry coordinating center. Aside from what may be required by applicable 3.2.3 System Development and Validation laws, this section generally discusses some of the components of a security program. Security is All software systems used for patient registries achieved not simply through technology but by should follow the standard principles of software clear processes and procedures. Overall development, including following one of the responsibility for security is typically assigned. standard life-cycle (SDLC) Security procedures are well documented and models that are well described in the software posted. The documentation is also used to train industry. staff. Some registries may also maintain personal In parallel, quality assurance of system information, such as information needed to contact development uses approved specifications to create patients to remind them to gather or submit a validation plan for each project. Test cases are patient-reported outcome information. created by trained personnel and systematically 3.3.1 System Security Plan executed, with results recorded and reviewed. Depending on regulatory requirements, a final A system security plan consists of documented validation report is often written and approved. policies and standard operating procedures Unresolved product and process issues are defining the rules of systems, including maintained and tracked in an issue tracking or administrative procedures, physical safeguards, CAPA (Corrective Action/Preventive Action) technical security services, technical security system. mechanisms, electronic signatures, and audit trails, as applicable. The rules delineate roles and Processes for development and validation should responsibilities. Included in the rules are the be similarly documented and periodically audited. policies specifying individual accountability for The information from these audits is captured, actions, access rights based on the principle of summarized, and reviewed with the applicable least privilege, and the need for separation of group, with the aim of ongoing process duties. These principles and the accompanying improvement and quality improvement. security practices provide the foundation for the confidentiality and integrity of registry data. The 3.3 Security rules also detail the consequences associated with All registries maintain health information, and noncompliance. therefore security is an important issue. The HIPAA (Health Insurance Portability and Accountability Act of 1996) Security Rule

267 Section III. Operating Registries

3.3.2 Security Assessment periodically and should detect possible Clinical data maintained in a registry can be unauthorized access attempts, such as multiple assessed for the appropriate level of security. failed logins, and automatically deauthorize the Standard criteria exist for such assessments and user account if they occur. The identification code are based on the type of data being collected. Part can also be an encrypted digital certificate stored of the validation process is a security assessment on a password-protected device or a biometric of the systems and operating procedures. One of identifier that is designed so that it can be used the goals of such an assessment is effective risk only by the designated individual. management, based on determining possible Rules should be established for situations in which threats to the system or data and identifying access credentials are compromised. New potential vulnerabilities. password information should be sent to the 3.3.3 Education and Training individual by a secure method. All staff members of the registry coordinating Intrusion detection and firewalls should be center should trained periodically on aspects of the employed on sites accessible to the Internet, with overall systems, security requirements, and any appropriate controls and rules in place to limit special requirements of specific patient registries. access to authorized users. Desktop systems Individuals should receive training relating to their should be equipped with antivirus software, and specific job responsibilities and document that servers should run the most recent security appropriate training has been received. patches. System security should be reviewed throughout the course of the registry to ensure that 3.3.4 Access Rights management, operational, personnel, and technical Access to systems and data should be based on the controls are functioning properly. principles of least privilege and separation of 3.3.6 Data Enclaves duties. No individual should be assigned access privileges that exceed job requirements, and no With the growth of clinical data and demands for individual should be in a role that includes access increasing amounts of clinical data by multiple rights that would allow circumvention of controls parties and researchers, new approaches to access or the repudiation of actions within the system. In are evolving. Data enclaves are secure, remote- all cases, access should be limited to authorized access systems that allow researchers to share individuals. respondents’ information in a controlled and confidential manner.23 The data enclave uses 3.3.5 Access Controls statistical, technical, and operational controls at Access controls provide the basis for different levels chosen for the specific viewer. This authentication and logical access to critical can be useful both for enhancing protection of the systems and data. Since the authenticity, integrity, data and for enabling certain organizations to and auditability of data stored in electronic access data in compliance with their own systems depend on accurate individual organization or agency requirements. Data authentication, management of electronic enclaves also can be used to allow other signatures (discussed below) is an important topic. researchers to access a registry’s data in a controlled manner. With the growth of registries Logical access to systems and computerized data and their utility for a number of stakeholders, data should be controlled in a way that permits only enclaves will become increasingly important. authorized individuals to gain access to the system. This is normally done through a unique access 3.3.7 Electronic Signatures code, such as a unique user ID and password Electronic signatures provide one of the combination that is assigned to the individual foundations of individual accountability, helping to whose identity has been verified and whose job ensure an accurate change history when used in responsibilities require such access. The system conjunction with secure, computer-generated, should require the user to change the password

268 Chapter 11. Data Collection and Quality Assurance

time-stamped audit trails. Most systems use an forth in agency regulations (often called “predicate electronic signature. For registries that report data rules”) and for any electronic records submitted to to FDA, such signatures must meet criteria the agency. FDA publishes nonbinding guidance specified in 21 CFR Part 11 for general signature documents from time to time that outline its composition, use, and control (sections 11.100, current thinking regarding the scope and 11.200, and 11.300). However, even registries that application of the regulation. The current guidance do not have such requirements should view these document is Guidance for Industry: Part 11, as reasonable standards. Before an individual is Electronic Records; Electronic Signatures – Scope assigned an electronic signature, it is important to and Application,25 dated August 2003. Other verify the person’s identity and train the individual documents that are useful for determining in the significance of the electronic signature. In validation requirements of electronic systems are cases where a signature consists of a user ID and a Guidance for Industry: Computerized Systems password, both management and technical means Used in Clinical Investigations,26 dated May 2007, should be used to ensure uniqueness and and General Principles of Software Validation; compliance with password construction rules. Final Guidance for Industry and FDA Staff,27 Password length, character composition, dated January 11, 2002. uniqueness, and validity life cycle should be based on industry best practices and guidelines published 4. Resource Considerations by the National Institute of Standards and Technology. Passwords used in electronic Costs for registries can be highly variable, signatures should abide by the same security and depending on the overall goals. Costs are also aging constraints as those listed for system access associated with the total number of sites, the total controls. number of patients, and the geographical reach of 3.3.8 Validation the registry program. Each of the elements described in this chapter has an associated cost. Systems that store electronic records (or depend Table 11–2 provides a list of some of the activities on electronic or handwritten signatures of those of the registry coordinating center as an example. records) that are required to be acceptable to FDA Not all registries will require or can afford all of must be validated according to the requirements the functions, options, or quality assurance set forth in the 21 CFR Part 11 Final Rule,24 dated techniques described in this chapter. Registry March 20, 1997. The rule describes the planners must evaluate benefit versus available requirements and controls for electronic systems resources to determine the most appropriate that are used to fulfill records requirements set approach to achieve their goals.

269 Section III. Operating Registries

Table 11–2. Data activities performed during registry coordination

Data Management • Defines all in-process data quality control steps, procedures, and metrics. • Defines the types of edit checks that are run against the data. • Defines required file-format specifications for electronic files, as well as schedules and processes for transfers of data. • Defines quality acceptance criteria for electronic data, as well as procedures for handling exceptions. • Develops guidelines for data entry. • Identifies areas of manual review where electronic checks are not effective. • Develops and maintains process for reviewing, coding, and reporting adverse event data. • Develops and maintains archiving process. • Develops and documents the process for change management. • Develops and maintains process for query tracking and creates standard reports to efficiently identify outstanding queries, query types per site, etc. • Relates queries to processes and activities (e.g., CRF design) requiring process improvements. • Follows up on query responses and errors identified in data cleaning by performing accurate database updates. • Defines registry-specific dictionaries and code lists. • Performs database audits as applicable. • Conducts user testing of systems and applications per written specifications. • Establishes quality criteria and quality error rate acceptance limits. • Evaluates data points that should be audited and identifies potential sources of data errors for audits. • Identifies root cause of errors in order to recommend change in process/technology to ensure the error does not occur again (continuous improvement). • Ensures that sampling audit techniques are valid and support decisions made about data. • Outlines all other data flow, including external data sources. Documentation • Documents the process, procedures, standards, and checklist(s) and provides training. • Documents and maintains process and standards for identifying signals and trends in data. • Documents database quality control actions performed. Reporting • Generates standard reports of missing data from the patient database. • Creates tools to track and inventory CRFs, and reports anticipated vs. actual CRF receipts. CRF = case report form.

270 Chapter 11. Data Collection and Quality Assurance

Case Examples for Chapter 11

Case Example 24. Developing a performance- Because of the potential serious side effects, FDA linked access system requires manufacturers of clozapine to maintain a Description The Teva Clozapine Patient patient monitoring system. Designed as a PLAS, Registry is one of several the registry needs to ensure the eligibility of national patient registries for patients, pharmacies, and physicians; monitor patients taking clozapine. The white blood cell (WBC) and absolute neutrophil registry is designed as a (ANC) reports for low counts; ensure compliance performance-linked access with laboratory report submission timelines; and system (PLAS) mandated by the respond to inquiries and reports of adverse U.S. Food and Drug events. Administration (FDA) to comply Proposed Solution with a Risk Evaluation The risk of developing agranulocytosis is Mitigation Strategy. The goal is mitigated by regular hematological monitoring to prevent clozapine rechallenge and is a condition of access to the drug, also in patients at risk for developing known as the “no-blood/no drug” requirement. clozapine- induced Since there are multiple manufacturers of agranulocytosis by monitoring clozapine, FDA requires each company to share lab data for signs of leukopenia information with the single national non- or granulocytopenia. rechallenge master file (NNRMF). The Teva Sponsor Teva Pharmaceuticals USA Clozapine Patient Registry was developed to Year Started 1997 meet these goals. The core components of the system are a call center, a Web site, and a Year Ended Ongoing reminder system. Patients must be enrolled prior No. of Sites Over 50,000 active physicians to receiving clozapine, and they must be assigned and pharmacies to a dispensing pharmacy and treating physician. After the patient has initiated therapy, a current No. of Patients 57,000 active patients and acceptable WBC count and ANC value are Challenge required prior to dispensing clozapine. Once a patient is enrolled and eligibility is confirmed, a Clozapine is classified as an atypical 1-, 2-, or 4-week supply of clozapine can be antipsychotic and is indicated for patients with dispensed, depending on patient experience and severe schizophrenia who fail standard therapy, the physician’s prescription. and for reducing the risk of recurrent suicidal behavior in schizophrenia or schizoaffective Health care professionals are required to submit disorder. However, clozapine is known to be laboratory reports to the registry based on the associated with a risk of developing patients’ monitoring frequency. Patients are agranulocytosis, a potentially life-threatening monitored weekly for the first 6 months. If there condition. The primary goal of the registry is to are no low counts, the patient can be monitored prevent clozapine-induced agranulocytosis. every 2 weeks for an additional 6 months. Patients at risk of developing clozapine-induced Afterward, if no low counts are detected after agranulocytosis are those who have a history of continuous therapy, the patient may qualify for severe leukopenia or granulocytopenia (white monitoring every 4 weeks (depending on the blood cell [WBC] <2,000/mm3 or absolute physician’s prescription). The registry provides neutrophil [ANC] <1,000/mm3). reminders if laboratory data are not submitted according to the schedule. If a low count is

271 Section III. Operating Registries

Case Example 24. Developing a performance- site allows health care professionals to manage linked access system (continued) their patients electronically. A reminder system Proposed Solution (continued) permits rapid notification to providers to ensure that appropriate actions are taken when low identified, registry staff inform the health care counts are detected or if laboratory reports are providers to make sure that they are aware of the not submitted in a timely manner. A call center event and appropriate action is taken. If severe with after-hours service ensures 24/7 availability, leukopenia or granulocytopenia is detected, the and data sharing with the NNRMF prevents patient is posted to the NNRMF to prevent future rechallenge regardless of manufacturer. These exposure to the drug. systems can also help sponsors monitor the Results patient population to learn more about adherence, compliance, and the frequency of adverse events. Results indicate that the registry is achieving its goal of reducing the risk of agranulocytosis For More Information associated with the use of clozapine by serving as Clozapine Package Insert (2012). https://www. an early warning system. By linking access to clozapineregistry.com/insert.pdf.ashx clozapine to a strict schedule of laboratory data submissions, the sponsor can ensure that only Honigfeld G. The Clozapine National Registry eligible patients are taking the drug. The sponsor System: forty years of risk management. J Clin is also able to detect low counts, prevent Psychiatry Monograph. 1996;14(2):29–32. inappropriate rechallenge (or re-exposure) in Karukin M, Conner J, Lage M. Incidence of at-risk patients, and monitor the patient leukopenia and agranulocytosis with the use of population for any adverse events. This system clozapine: Evidence from the Teva Clozapine provides the sponsor with data on the frequency Patient Registry. Poster Presented at the 23rd and severity of adverse events while ensuring that Annual U.S. Psychiatric and Mental Health only the proper patient population receives the Congress, Poster #219, November 20, 2010. drug. Peck CC. “FDA’s position on the clozaril patient Key Point management system.” Hosp Community A PLAS can ensure that only appropriate patients Psychiatry. 1990 Aug;41(8): 876-7. receive treatment. A secure, fully functional Web

272 Chapter 11. Data Collection and Quality Assurance

Case Example 25. Using audits to monitor enrollment of eligible patients was one major data quality potential area for bias. It was determined that an Description The Vascular Study Group of audit of participating sites, focusing on included New England (VSGNE) is a versus eligible patients, could reasonably address voluntary, cooperative group of whether this was a significant issue. However, the clinicians, hospital group needed to overcome two logistical administrators, and research challenges: (1) the audit had to review thousands personnel, organized to improve of eligible patients at participating hospitals in a the care of patients with timely, cost-effective manner; and (2) the audit vascular disease. The purpose of could not overburden the hospitals, as they the registry is to collect and participate in the study voluntarily. exchange information to support Proposed Solution continuous improvements in the The registry team developed a plan to conduct the quality, safety, effectiveness, and audit using electronic claims data files from the cost of caring for patients with hospitals. Each hospital was asked to send claims vascular disease. data files for the appropriate time periods and Sponsor Funded by participating procedures of interest to the registry. The registry institutions. Initial funding was team at Dartmouth-Hitchcock Medical Center provided by the Centers for then matched the claims data to the registry Medicare & Medicaid Services. enrollment using ICD-9 (International Year Started 2002 Classification of Diseases, 9th Revision) codes with manual review of some patient files that did Year Ended Ongoing not match using a computer-matching process. No. of Sites 30 hospitals in Connecticut, Results Rhode Island, Massachusetts, Maine, New Hampshire, and The first audit performed in 2003 found that Vermont approximately 7 percent of eligible patients had not been enrolled in the registry. Because of No. of Patients Over 25,000 concerns that the missing patients may have had Challenge different outcomes than the patients who had been enrolled in the registry, the registry team VSGNE established a registry in 2002 as part of asked participating hospitals to complete registry an effort to improve quality of care for patients forms for all missing patients. This effort undergoing carotid endarterectomy, carotid increased the percentage of eligible patients stenting, lower extremity arterial bypass, and enrolled in the registry to over 99 percent. The open and endovascular repair of abdominal aortic team also compared the discharge status of the aneurysms. The registry collects more than 120 missing patients and the enrolled patients, and patient, process, and outcome variables for each found no significant differences in outcomes. The procedure at the time of hospitalization, and team concluded that the patients had been missed 1-year results are collected during a followup at random and that there were no systematic visit at the treating physician’s office. All patients enrollment issues. Discussions with the hospitals receiving one of the procedures of interest at a identified the reasons for not enrolling patients as participating hospital are eligible for enrollment confusion about eligibility requirements, training in the registry. issues, and questions about informed consent In considering the areas of greatest risk in requirements. evaluating the quality of this registry, the registry Subsequent audits in 2006 and 2008 had similar developers determined that incomplete outcomes, but considerable time was required to

273 Section III. Operating Registries

Case Example 25. Using audits to monitor Key Point data quality (continued) For many registries, audits of participating sites Results (continued) are an important tool for ensuring that the data clarify ICD-9 coding differences with procedures are reliable and valid. However, registries that in the registry, since ICD-9 codes are not granular rely on voluntary site participation must be for vascular procedures. In 2011, the VSGNE cautious to avoid overburdening sites during the model for regional vascular quality improvement audit process. A remote audit using readily was adopted by the Society for Vascular Surgery available electronic files, such as claims files, as the Vascular Quality Initiative, now a national provided a reasonable assessment of the network of regional quality groups like VSGNE, percentage of eligible patients enrolled in the organized under the umbrella of the Society for registry without requiring large amounts of time Vascular Surgery’s patient safety organization. In or resources from participating sites. 2012, the now nationwide audit mechanism for For More Information data completeness switched from using ICD-9 Cronenwett JL, Likosky DS, Russell MT, et al. A codes to physician current procedural regional registry for quality assurance and terminology (CPT®) claims data, since CPT improvement: the Vascular Study Group of codes are more precise for specific vascular Northern New England (VSGNNE). J Vasc Surg. procedures. Preliminary results in 2012 show 2007;46:1093–1102. more precise matching with registry data using CPT-based claims. Cronenwett JL, Kraiss LW, Cambria RP. The Society for Vascular Surgery Vascular Quality Initiative. J Vasc Surg. 2012;55:1529-37.

References for Chapter 11 6. Neale R, Rokkas P, McClure RJ. Interrater reliability of injury coding in the Queensland 1. American Heart Association. Get With The Trauma Registry. Emerg Med (Fremantle). 2003 Guidelines. http://www.heart.org/HEARTORG/ Feb;15(1):38-41. PMID: 12656785. HealthcareResearch/ 7. A Healthcare IT Primer. http://geekdoctor. GetWithTheGuidelinesHFStroke/Get-With-The- blogspot.com/2009/03/healthcare-it-primer.html. Guidelines---HFStroke_UCM_001099_ Accessed May 1, 2013. SubHomePage.jsp. Accessed May 1, 2013. 8. cancer Text Information Extraction System 2. Clinical Data Interchange Standards Consortium. (caTIES). http://caties.cabig.upmc.edu/. Accessed http://www.cdisc.org. Accessed May 1, 2013. August 7, 2013. 3. National Institutes of Health. National Institutes of 9. Informatics for Integrating Biology and the Health Stroke Scale. http://www.ninds.nih.gov/ Bedside. https://www.i2b2.org/. Accessed May 1, doctors/NIH_Stroke_Scale.pdf. Accessed May 1, 2013. 2013. 10. Health Level Seven. http://www.hl7.org. Accessed 4. Luck J, Peabody JW, Dresselhaus TR, et al. How May 1, 2013. well does chart abstraction measure quality? A prospective comparison of standardized patients 11. U.S. Food and Drug Administration. Guidance for with the medical record. Am J Med. 2000 Jun Industry. E6 Good Clinical Practice: Consolidated 1;108(8):642-9. PMID: 10856412. Guidance. April 1996. http://www.fda.gov/ downloads/Drugs/Guidances/ucm073122.pdf. 5. Reisch LM, Fosse JS, Beverly K, et al. Training, Accessed August 7, 2013. quality assurance, and assessment of medical record abstraction in a multisite study. Am J 12. National eHealth Collaborative. http://www. Epidemiol. 2003 Mar 15;157(6):546-51. nationalehealth.org. Accessed May 1, 2013. PMID: 12631545.

274 Chapter 11. Data Collection and Quality Assurance

13. National Institute of Standards and Technology. 21. Fink AS, Campbell DA, Jr., Mentzer RM, Jr., et al. http://www.nist.gov/index.html. Accessed May 1, The National Surgical Quality Improvement 2013. Program in non-veterans administration hospitals: initial demonstration of feasibility. Ann Surg. 14. OPTIMIZE-HF. Organized Program to Initiate 2002 Sep;236(3):344-53; discussion 53-4. LifeSaving Treatment in Hospitalized Patients PMID: 12192321. PMCID: 1422588. with Heart Failure. http://www.optimize-hf.org/. Accessed May 1, 2013. 22. The HIPAA Security Rule: Health Insurance Reform: Security Standards, February 20, 2003. 15. Fonarow GC, Heywood JT, Heidenreich PA, et al. 68 FR 8334. Temporal trends in clinical characteristics, treatments, and outcomes for heart failure 23. National Institutes of Health. NIH Data Sharing hospitalizations, 2002 to 2004: findings from Policy and Implementation Guidance. http:// Acute Decompensated Heart Failure National grants.nih.gov/grants/policy/data_sharing/data_ Registry (ADHERE). Am Heart J. 2007 sharing_guidance.htm#enclave. Accessed May 1, Jun;153(6):1021-8. PMID: 17540205. 2013. 16. IMPROVE-HF. http://newsroom.medtronic.com/ 24. U.S. Food and Drug Administration. CFR - Code phoenix.zhtml?c=251324&p=irol- of Federal Regulations Title 21, Part 11: newsArticle&ID=1771401&highlight=. Electronic Records; Electronic Signatures. http:// Accessed May 1, 2013. www.accessdata.fda.gov/SCRIPTs/cdrh/cfdocs/ cfcfr/CFRSearch.cfm?CFRPart=11&showFR=1. 17. Lewis WR, Peterson ED, Cannon CP, et al. An Accessed May 1, 2013. organized approach to improvement in guideline adherence for acute myocardial infarction: results 25. U.S. Food and Drug Administration. Guidance for with the Get With The Guidelines quality Industry Part 11, Electronic Records; Electronic improvement program. Arch Intern Med. 2008 Signatures — Scope and Application. http://www. Sep 8;168(16):1813-9. PMID: 18779470. fda.gov/downloads/RegulatoryInformation/ PMCID: 3086550. Guidances/ucm125125.pdf. Accessed May 1, 2013. 18. U.S. Food and Drug Administration. Guidance for Industry: Development and Use of Risk 26. U.S. Food and Drug Administration. Guidance for Minimization Action Plans. http://www.fda.gov/ Industry Computerized Systems Used in Clinical downloads/RegulatoryInformation/Guidances/ Investigations. http://www.fda.gov/downloads/ UCM126830.pdf. Accessed May 1, 2013. Drugs/GuidanceComplianceRegulatory Information/Guidances/UCM070266.pdf. 19. Mangano DT, Tudor IC, Dietzel C, et al. The risk Accessed May 1, 2013. associated with aprotinin in cardiac surgery. N Engl J Med. 2006 Jan 26;354(4):353-65. 27. U.S. Food and Drug Administration. General PMID: 16436767. Principles of Software Validation; Final Guidance for Industry and FDA Staff. http://www.fda.gov/ 20. Gheorghiade M, Abraham WT, Albert NM, et al. downloads/MedicalDevices/DeviceRegulationand Systolic blood pressure at admission, clinical Guidance/GuidanceDocuments/UCM085371.pdf. characteristics, and outcomes in patients Accessed May 1, 2013. hospitalized with acute heart failure. JAMA. 2006 Nov 8;296(18):2217-26. PMID: 17090768.

275

Chapter 12. Adverse Event Detection, Processing, and Reporting 1. Introduction For marketed products regulated by FDA, AEs are categorized for reporting purposes according to Registries that collect information on specific the seriousness and expectedness of the event (i.e., drugs and medical devices need to anticipate the whether the event was previously observed and need for adverse event (AE) detection, processing, included in local product labeling), as it is and reporting. This chapter addresses the presumed that all spontaneously reported events identification, processing, and reporting of AEs are potentially related to the product for the detected in situations in which a registry has purposes of FDA reporting. Prior to marketing contact with individual patients. This document is approval, relatedness is an additional determinant not a formal regulatory or legal document; for reporting events occurring during clinical trials therefore, any information or suggestions or preclinical studies associated with presented herein do not supersede, replace, or investigational new drugs and biologics. For AEs otherwise interpret Federal guidance documents occurring in postapproval studies and reported that touch on these subjects. Registry sponsors are during planned contacts and active solicitation of encouraged to discuss plans for AE collection and information from patients, as when registries processing with local health authorities when collect data regarding one or more FDA-approved planning a registry. products,6, 7 the requirements for mandatory reporting also include whether there is a This chapter primarily focuses on AEs related to reasonable possibility that the drug caused the pharmaceutical products. Medical devices are adverse experience.4 For registries that do not significantly different from pharmaceutical actively solicit AEs, incidentally reported events products in the manner in which AEs and product (e.g., those reported during clinician or consumer problems (complaints) present themselves, in the contact for another purpose) should typically be etiology of their occurrence, and in the regulation handled and evaluated as spontaneously reported governing the defining and reporting of these events. occurrences, as well as postapproval study requirements. Other sources provide more The medical device reporting regulations differ information about defining and reporting device- from those for drugs and biologics in that related AEs and product problems, and about reportable events include both AEs and problems postmarketing studies (including those involving with the device itself.8 Medical device reporting is registries).1-3 required for incidents in which the device may have caused or contributed to a death or serious 2. Identifying and Reporting injury, or may have malfunctioned and would likely cause or contribute to death or serious injury Adverse Drug Events if the malfunction were to recur.9 The U.S. Food and Drug Administration (FDA) Most registries have the opportunity to identify defines an adverse drug experience as any AE and capture information on AEs for associated with the use of a drug in humans, biopharmaceutical products and/or medical whether or not considered drug related,4 while the devices. With the passing of the FDA Amendments International Conference on Harmonisation (ICH) Act in September 2007 and the increased emphasis guideline ICH E2A similarly defines an AE as an on ongoing monitoring of safety profiles, untoward medical occurrence in a patient evaluation of risks unknown at the time of product administered a pharmaceutical product, whether or approval, and proactive detection of potential not the occurrence is related to or considered to safety issues, registries increasingly continue to be have a causal relationship with the treatment.5 used to fulfill safety-related objectives.10 Although no regulations in the United States specifically

277 Section III. Operating Registries

compel registries to capture and process AE or other means; perusal of electronic medical reports (aside from reporting requirements for records or insurance claims data would not be registries that are sponsored by regulated considered direct patient interaction. Reporting is industries), there is an implicit requirement from rarely required for individual AEs observed in the perspective of systematic data collection and aggregate population data, since there is no direct promoting public health: any individual who patient interaction where an association might be believes a serious risk may be associated with suggested or inferred. Nevertheless, if aggregate or exposure to a medical product should be epidemiologic analyses suggest that an AE is encouraged to report this AE either to the product associated with exposure to a drug or medical sponsor or directly to FDA. The FDA maintains product, it is desirable that the minimum dataset MedWatch, a Web-based reporting system that information be forwarded to the manufacturer of allows consumers and health professionals to the product, who will determine any need for, and voluntarily report serious adverse events and other timing of, reporting of study results to the relevant serious problems that they suspect are associated regulatory authorities. 11 with the use of an FDA-regulated product. Figure 12–1 provides a broad overview of the The minimum dataset required to consider reporting requirements for AEs and shows how the information as a reportable AE is indeed minimal, reporting differs according to whether the registry namely (1) an identifiable patient, (2) an has direct patient interaction, and whether it identifiable reporter, (3) product exposure, and receives sponsorship and/or financial support from (4) an event. However, in addition to direct data a regulated industry.12 These industries may collection, AEs can be detected through include entities with products subject to FDA retrospective analysis of a population database, regulation, including products with FDA approval, where direct patient or health care provider contact an FDA-granted license, and investigational does not occur. Patient interactions include clinical products; and other entities such as manufacturers, interactions and data collection by phone, Internet, user facilities, and distributors.

278 Chapter 12. Adverse Event Detection, Processing, and Reporting

Figure 12–1. Best practices for adverse event reporting to FDA by registries of postmarket products

Follow good public Does the registry receive Does the registry have data health practices for No sponsorship or financial Ye s collection with individual reporting new or serious support from any regulated patient interaction? AEs (recommended industry? No practice, not mandated). Yes Train site(s) in identification and reporting of AEs, including events of special interest and SAEs. Report AEs in FDA periodic Notify company and/ reports or PSUR if applicable. or FDA about new or Establish rules, roles, serious AEs.a responsibilities for involved parties for oversight and reporting in conformance with Aggregate study findings of registry design and applicable adverse events. regulations. Company FDA contact

Are SAEs in temporal association with a druga under study recognized by a No knowledgeable person?

Yes Is there a reasonable possibility No that the drug caused the SAE?

Yes

a Notify responsible entity (e.g., For devices, no attribution of expectedness is company) as soon as possible, required; "device-relatedness" is based on ideally within 24 hours. whether the device caused or contributed to death or serious injury, or, in the case of malfunction, if the chance of death or serious injury is not remote if the malfunction were to recur. Company determines if the SAE is “unexpected” (based on labeling) in terms of type, No specificity, or severity.

Yes

Company reports SAEs considered unexpected and possibly related to own drugs to FDA within 15 calendar days of original report; reports for device-related deaths, serious injury, or malfunctions are due within 10-30 calendar days.

AE = adverse event; SAE = serious adverse event; FDA = U.S. Food and Drug Administration; PSUR = periodic safety update report.

279 Section III. Operating Registries

All AE reporting begins with a suspicion by the are not consistent with information provided in physician (or responsible person who obtains or the relevant product information (e.g., approved receives information) that a patient exposed to a professional package insert or product label). medicinal product has experienced some AE and Determination of expectedness is made by the that the event has a reasonable possibility of being sponsor on a case-by-case basis. Expected causally related to the product being used; this is events typically do not require expedited referred to as the “becoming aware” principle. reporting to the regulatory authorities. Some registries also collect and record AEs • Relatedness: Relatedness is a term intended to reported directly by the patients or their caregivers. indicate that a determination has been made It is important to develop a plan for detecting, that the event had a reasonable possibility of processing, and reporting AEs for any registry that being related to exposure to the product. This has direct patient contact. If the registry receives assessment of causality may be based on sponsorship in whole or part from a regulated factors such as biological plausibility, prior industry (for drugs or devices), the sponsor has experience with the product, and temporal mandated reporting requirements, including relationship between product exposure and stringent timelines. AE reporting requirements for onset of the event, as well as dechallenge registry sponsors are discussed later in this (discontinuation of the product to determine if chapter. the AE resolves) and rechallenge Prior to registry launch, the process for detecting (reintroduction of the product to determine if and reporting AEs should be established in the AE recurs). Many terms and scales are used collaboration with the sponsor and any oversight to describe the degree of causality, including committees. (See Chapter 2.) Once the plans have terms such as certainly, definitely, probably, been developed, the registry operator or sponsor possibly, or likely related or not related, but should provide training to the physicians or other there is no standard nomenclature.13 All responsible parties (referred to as “sites” hereafter) spontaneous reports have an implied causal on how to identify AEs and to whom they should relationship as per regulatory guidance, be reported. AE reporting is based on regardless of the reporter’s assessment. categorization of the AE according to the The registry may use forms such as a structured seriousness of the event, its expectedness based on questionnaire or an AE case report form to collect product labeling, and presumed causality or the information from providers or patients. When possible association with use of the product, as solicitation of AEs is not prespecified in the follows: registry’s operating plans, the registry may permit • Seriousness: Serious AEs (SAEs) include AE detection by asking general questions to solicit events that result in death, are life threatening events, such as “Have you had any problems since (an event in which the patient was at risk of your last visit or since we last spoke?” and then death at the time of the event), require or following up any such reports with probes as to prolong inpatient hospitalization, result in what happened, diagnoses, and other persistent or significant disability or incapacity, documentation. This practice is not required. or result in a congenital anomaly. Important medical events may also be considered serious 3. Collecting AE Data in a when, based on medical judgment, they may Registry jeopardize the person exposed and may require medical or surgical intervention to prevent one There are two key considerations regarding AE of the outcomes listed above (e.g., death or collection as part of a registry: (1) what data need prolonged hospitalization). to be collected to meet the registry’s safety-related • Expectedness: All AEs that are previously objectives, and (2) what processes need to be in unobserved or undocumented are referred to as place to ensure that the registry is in compliance “unexpected,” in that their nature and severity with regulations regarding expedited and periodic

280 Chapter 12. Adverse Event Detection, Processing, and Reporting

AE event reporting, if applicable. The data fields may mean that all, some, or no AEs are collected needed for the purpose of analysis by the registry on the case report forms. However, if some AEs may be minimal (e.g., event and onset date), are collected in an intentional, solicited manner whereas a complete SAE form for a subset of (e.g., routine collection of a primary or secondary events reported to the registry may be sought to outcome via an AE case report form) and others fulfill the sponsor’s reporting requirements. Due to come to the registry’s attention in an unsolicited, the nature of registries, the goal of collecting “spontaneous” way (e.g., when an AE is reported enough data to meet the registry’s objectives must in the course of a registry contact, such as a call to constantly be balanced with the burden on sites. To the sponsor or to registry support staff), then from this end, the processes for AE reporting should be a practical perspective it is even more important to streamlined as much as possible. have a clear process, so that AEs that require The collection of AE data by a registry is generally reporting are identified. In this scenario, one best either intentionally solicited (meaning that the data practice that is often used in electronic registry are part of the uniform collection of information in studies is to have a notification sent promptly to the registry) or unsolicited (meaning that the AE the sponsor’s safety group when a case report form information is volunteered or noted in an is submitted that contains specific or potential unsolicited manner and not as a required data information indicating that a serious AE has element through a case report form). As described occurred. This process allows for rapid followup further below, it is good practice for a registry to by the sponsor, as needed. specify when and how AE information (and any other events of special interest) should and should 4. AE Reporting by the not be solicited from patients by a site and, if that Registry information has been obtained, how and when the site should inform the appropriate persons. Once suspicion has been aroused that an While an AE may be reported to the manufacturer, unexpected serious event has a reasonable to FDA (e.g., via MedWatch), or to the registry possibility of being causally related to a drug, the itself (and then from the registry to the AE should be reported to FDA through manufacturer), it is strongly encouraged that the MedWatch, to the company that manufactures the protocol describe the procedures that should be product, or to the registry coordinating center. (See followed, and that the sites be trained in these Chapter 11.) A system should be developed such procedures as well as in their general obligations that all appropriate events are captured and and the relevant public health considerations. A duplicate reporting is avoided to the extent separate safety reporting plan that fully identifies possible. Generally, AE reports are submitted the responsible parties and describes the directly by the site or by the registry to the operational considerations may also be considered manufacturer, since they are often most efficient at to ensure that potentially reportable information is evaluating, processing, and reporting for evaluated in an appropriate timeframe, and, for regulatory purposes within the required time manufacturer-sponsored registries, in accordance periods. Alternatively, sites could be instructed to with any applicable standard operating procedures. report AEs directly to FDA according to their This type of plan also should describe how normal practices for marketed products; however, deviations or systemic failures in detection and this often means that the companies are not reporting processes will be identified, addressed, notified of the AE and are not able to follow up or and considered for corrective action. evaluate the event in the context of their safety database. In fact, companies are not necessarily Determining whether a registry should use a case notified by FDA if an AE report comes directly to report form to collect AEs should be based on the FDA, since only certain reports are shared with principles described in Chapter 4, which refer to industry, and reporters have an option to request the scientific importance of the information for that the information not be shared directly with the evaluating the specified outcomes of interest. This company.14 When sites report AEs directly to

281 Section III. Operating Registries

FDA, this process can also lead to inadvertent the sponsor or centralized designated responsible duplication of information for events recorded personnel, who then reports to the regulatory both by the registry and the company. authorities. However, when products other than Systematic collection of all AEs provides a unique those exclusively manufactured by the sponsor are resource of consistent and contemporaneously involved, including other treatments, sponsors will collected comparison information that can be used need to determine how to process AE reports at a later date to conduct epidemiologic received for these other products. Since sponsors assessments. Ideally, the practice for handling AEs are not obligated to report AEs for their and SAEs should be applied to all treatments competitors, it is good practice from a public (including comparators) recorded in the registry, health perspective to specify how the site should so that all subjects are treated similarly. In fact, a address those AEs (e.g., whether to report directly strong advantage of registries with systematic data to the other product’s manufacturer or to FDA). collection and internal comparators is that they Options for the sponsor include (1) recommending provide both numerators and denominators for that sites report the AEs of comparators directly to safety events; thus, reporting of comparative the manufacturer or to FDA; (2) collecting all AEs known AE rates in the context of a safety and forwarding the AE report directly to the evaluation provides valuable information on comparator’s manufacturer (who would then, in real-world performance. The contrast with turn, report to FDA); and (3) actually reporting the comparators helps promote clarity about whether AE for the comparator product directly to FDA. As the observed effects are unique to the product, standard practice in pharmacovigilance, many unique to a class, or common to the condition sponsors report events potentially associated with being treated. Reporting AEs without denominator another manufacturer’s drug to that manufacturer’s information is less useful from a surveillance safety department as a courtesy, rather than report perspective since events rates cannot be calculated events directly to FDA, and choose to continue without both numerators and denominators. The that practice when conducting a registry or other reliability of the denominator should always be observational study. judged, however, by considering the likelihood that Some disease registries are not focused on a all events were reported appropriately. specific product, but rather on conducting natural For postapproval registries not financially history studies or evaluating treatment patterns and supported by pharmaceutical companies, health outcomes in a particular patient population prior to care providers at registry sites should be instructed marketing approval of the sponsor’s product. In that if they suspect or otherwise become aware of a these situations, it is recommended that sites serious AE that has a reasonable possibility of follow their own standard practices for being causally related to a drug or product, they spontaneous AE reporting, including reporting any should report the event directly to the product events associated with a product known to be manufacturer (who must then report to FDA under manufactured by the sponsor. regulation) or to FDA’s MedWatch program (or In most circumstances in which a serious drug- local health authority if the study is conducted associated AE is suspected, sites are encouraged to outside of the United States). Reporting can be submit supportive data to sponsors, such as facilitated by providing the MedWatch Form laboratory values, vital signs, and examination 3500,15 information regarding the process for results, along with the SAE report form. If the submission, and MedWatch contact information. event is determined to be an AE, the sponsor will For registries that are sponsored or financially include it in the safety database, evaluate it supported in full or in part by a regulated industry internally, and transfer the AE report to the and that study a single product, the most efficient regulatory authorities if required. It should be monitoring system to avoid duplicate reporting is noted that the regulations represent minimum one in which all physicians participating in the requirements for compliance; special registry report all AEs (or SAEs only) directly to circumstances for a particular product may result

282 Chapter 12. Adverse Event Detection, Processing, and Reporting

in additional events being reportable (e.g., are also used. For example, SNOMED-CT expected events of particular interest to (Systematized Nomenclature of Medicine-Clinical regulators). It should not be expected that registry Terms) is used instead of MedDRA in some participants be aware of all the reporting nuances electronic health records. Coding the different associated with a particular product. To the extent verbatim language to preferred terms allows possible, guidance on reporting events of special similar events to be appropriately grouped, creates interest should be provided in the protocol and in consistency among the terms for evaluation, and any safety training. maximizes the likelihood that safety signals will If an external party manages a registry, SAEs be detected. should be submitted to the sponsors as quickly as Sponsors or their designees should review the possible after the registry becomes aware of the accuracy of the coding of verbatim AEs into event. In this situation, the registry is an agent of appropriate terms. If coding is performed by the sponsor, and FDA’s 15-calendar-day reporting someone other than the sponsor, any applicable requirement starts as soon as the event has come to coding conventions associated with the underlying the attention of the registry. (See Section 7 below.) condition or product should be shared. Review of This submission can be accomplished by phone or the coding process should focus on terms that do fax, or by means of automated rules built into the not accurately communicate the severity or vehicle used for data collection (such as automatic magnitude of the AE or possibly mischaracterize triggers that can be designed into electronic data the AE. Review of the coded terms compared with capture programs). For direct regulatory reported verbatim terms should be performed in submissions, the MedWatch Form 3500A16 should order to ensure consistency and accuracy of the AE be used for postapproval reporting for drugs and reporting and to minimize variability of coding of therapeutic biologics unless other means of similar AE terms. Attention to consistency is submission are agreed upon. For vaccines, the especially important, as many different individuals Vaccine Adverse Event Reporting System should may code AEs over time, and this situation be consulted.17 Foreign events may be submitted contributes to variability in the coding process. In on a CIOMS form (the World Health addition to monitoring AEs individually for Organization’s Council for International complete clinical evaluation of the safety data, Organizations of Medical Sciences),8, 18, 19 or a sponsors should consider grouping and analyzing letter can be generated that includes the relevant clinically relevant coded terms that could represent information in narrative format. similar toxicities or syndromes. Combining terms may provide a method of detecting less common 5. Coding and serious events that would otherwise be obscured. However, sponsors should be careful Coding AEs into a standard nomenclature should when combining related terms to avoid amplifying be done by trained experts to ensure accuracy and a weak signal or obscuring important overall consistency. Reporters, patients, health care findings when grouping is overly broad. In providers, and registry personnel should do their addition to monitoring individual AEs, sites and best to capture the primary data clearly, registry personnel should be attentive to toxicities completely, and in as “natural” clinical language as that may cluster into syndromes. possible. Since reporters may use different verbatim terms to describe the same event, it is 6. Adverse Event Management recommended that sponsors apply coding conventions to code the verbatim terms. The In some cases, such as when a safety registry is Medical Dictionary for Regulatory Activities created as a condition of regulatory approval, a (MedDRA®) is customarily used throughout the data safety monitoring board (DSMB), data product development cycle and as part of monitoring committee (DMC), or adjudication pharmacovigilance; however, other coding systems committee may be established with the primary

283 Section III. Operating Registries

role of periodically reviewing the data as they are drug safety surveillance division. By this method, generated by the registry. Such activities are the sponsor provides a separate contact number for generally discussed directly with the regulatory AE reporting (independent of the registry support authorities, such as FDA. These authorities are staff) that places the site in direct contact with typically involved in the design and critique of drug safety personnel. This process minimizes the protocols for postapproval studies. Ultimately, possibility of duplicate AE reports and the registry planning and the registry protocol should potentially complicated reconciliation of two anticipate and clearly delineate the roles, different systems collecting AE information. Use responsibilities, processes, forms, and lines of of this process is critical when dealing with communication for AE reporting for sites, registry products that are available via a registry system as personnel, the DSMB, DMC, or adjudication well as outside of a registry system, and it allows committee if one exists, and the sponsoring sites to have one designated drug safety organization. Documentation should be provided representative for interaction. for definitions and approaches to determining what Sponsors of registries designed specifically for is considered unexpected and possibly related to surveillance of product safety are strongly drug or device exposure. The management of AE encouraged to hold discussions with the regulatory reporting should be clearly specified in the registry authorities when considering the design of the AE protocol, including explanations of the roles, monitoring system. These discussions should be responsibilities, processes, and methods for focused on the purpose of the registry, the “best handling AE reports by the various parties fit” model for AE monitoring, and the timing of conducting the registry, and for performing routine registry updates. With respect to internal followup activities with the site to ensure that operations chosen by the sponsor to support the complete information is obtained. Sponsors who requirements of an AE monitoring system, are stakeholders in a registry should have a anecdotal feedback suggests that health authorities representative of their internal drug safety or expect compliance with the agreed-upon pharmacovigilance group participate in the design requirements. Details regarding implementation and review of the registry protocol and have a role are the responsibility of the sponsor. in the data collection and reporting process (discussed in Chapter 2) to facilitate appropriate It should also be noted that FDA’s Proposed Rule and timely reporting and communication. for Safety Reporting Requirements for Human Drug and Biologics Products (68 FR 12406, For postapproval studies financially sponsored by March 14, 2003) suggests that the responsible manufacturers, the overall company AE point of contact for FDA should be provided for all monitoring systems are usually operated by expedited and periodic AE reports, and preferably, personnel experienced in drug safety (also referred this individual should be a licensed physician. to as pharmacovigilance, regulatory safety, product Although this proposed rule has never been safety, and safety and risk management). If sites finalized, the principle is similar to the Qualified need to report or discuss an AE, they can call the Person for Pharmacovigilance (QPPV) in Europe, contact number provided for the registry, and are whereby a specific, qualified individual is then prompted to press a number if reporting an identified to provide responses to health AE. This number then transfers them to drug authorities, upon request, including those safety surveillance so that they can interact regarding AEs reported via the registry system. directly with personnel in this division and bypass Updated pharmacovigilance regulations issued by the registry coordinating group. These calls may or the European Medicines Agency are expected to may not be tracked by the registry. Alternatively, be implemented in July 2012.20, 21 the registry system can provide instructions to the site on how to report AEs directly to the sponsor’s

284 Chapter 12. Adverse Event Detection, Processing, and Reporting

7. Adverse Event Required guidelines describe standards for expedited reporting5, 22 and provide recommendations for Reporting for Registry periodic safety update reports23 that are generally Sponsors accepted globally.

The reporting requirements of the sponsor directly affect how registries should collect and report AEs. Sponsors that are regulated industries are subject to the requirements shown in Table 12–1. ICH

Table 12–1. Overview of serious adverse event reporting requirements for marketed products

Type of Requirement Drug and Biologics Devices

U.S. postmarketing Primary: 21 CFR 314.80 (drugs), 21 CFR 21 CFR 803.20 regulations 600.80 (biologics) Other: 21 CFR 310.305, 21 CFR 314.98 Required reporting Regulated industries Manufacturer, importer, user facility source Required reports Serious, unexpected, and with a reasonable Death or serious injury; device possibility of being related to drug malfunction exposure (with some exceptions) Alternative reports Not applicable Summary reports (periodic line-listing of reports of well-known events) Timeframe for 15 calendar days for expedited reports 5 workdays, 10 workdays, or 30 calendar reporting days, depending on the source and action required Standard reporting MedWatch 3500A (for mandatory reporting required of a regulated industry) form MedWatch 3500 (for voluntary reporting by health care professionals, consumers, and patients) Web sites http://www.fda.gov/Safety/MedWatch/ http://www.fda.gov/MedicalDevices/ default.htm Safety/ReportaProblem/default.htm#1

Requirements for regulated industries that sponsor notification to the responsible regulatory authority or financially support a registry include expedited within a total of 15 calendar days. Events that do reporting of serious and unexpected AEs made not meet the requirements of expedited reporting known to them via spontaneous reports. For (such as nonserious events or serious events registries, the 15-calendar-day notification applies considered expected or not related) may require if the regulated industry believes there is a submission through inclusion in an appropriate reasonable possibility that the unexpected SAE safety update, such as the New Drug Application was causally related to product exposure. Best or Biologic Licensing Application Annual Report, practices for international reporting are that all Periodic Report, or Periodic Safety Update Report, “affiliates” of a sponsor report serious, unexpected, as applicable.4 In many cases, sponsors are also and possibly related events to the sponsor in a required to provide registry safety updates to the timely fashion, ideally within 2 calendar days; this health authority. Thus, sponsors may coordinate allows the sponsor, in turn, to complete registry safety updates (i.e., determining the date

285 Section III. Operating Registries

for creating the dataset—the data cutoff date) with regulation. Sponsors are encouraged to appoint a the timing of the New Drug Application Annual health care practitioner to this role in order to Report, Periodic Report, Periodic Safety Update ensure appropriate assessment of the Report, or other agreed-upon periodic reporting characteristics of an AE. format. Devices, however, have different reporting When biopharmaceutical or device companies are requirements (see http://www.fda.gov/ not sponsoring, financially supporting, or MedicalDevices/Safety/ReportaProblem/default. participating in a registry in any way, AE reporting htm). In any event, sponsors should discuss safety is dependent upon the “become aware” principle. reporting requirements for their specific registries If any agent or employee of the company receives with the applicable health authorities (such as FDA information regarding an AE report, the agent or and European Medicines Agency) before finalizing employee must document receipt and comply with their registry protocol. internal company policy and regulatory In some cases, a registry sponsor may encourage requirements regarding AE reporting, to ensure sites to systematically report all potential SAEs to compliance with applicable drug and device the sponsor. Given the potential for various regulations. assessments by different sites of the seriousness and relatedness of a particular AE—and therefore, 8. Special Case: Risk inconsistency across sites in the evaluation of a particular AE—this method has certain Evaluation and Mitigation advantages. In addition, expectedness is not always Strategies (REMS) a straightforward assessment, and the expectedness of events can have significant variability depending Under the FDA Amendments Act (2007), FDA on the local approved product labeling. For this established a legally enforceable new framework reason, it is important that this determination be for risk management of products with known made by the sponsor and not the reporter of the safety concerns, called Risk Evaluation and event. Although this approach may result in Mitigation Strategies (REMS).6, 10, 24 The purpose substantially greater demands on the sponsor to of REMS is to ensure that the benefits of a evaluate all reports, it helps ensure compliance and particular drug outweigh the risks. New REMS avoid underreporting. Furthermore, sponsors must programs can be imposed by FDA during clinical make their own assessments regarding the development, as part of the approval process, or at causality of individual solicited events. This any time postapproval, should a new safety signal requirement typically does not affect the need for be identified. Although each REMS is customized reporting, but allows the sponsor to provide its depending on the product and associated safety own evaluation in the full context of the safety issues, potential components include some database. For these reasons, planning for high- combination of— quality and consistent training in AE reporting • A medication guide and/or patient package requirements across sites is the preferred approach insert. Medication guides are informational for a patient registry. packets distributed with some prescription Regardless of who assesses presumed relatedness, drugs, which provide important information to sponsors should be prepared to manage the patients about possible side effects and drug- increased volume of AE reports, and sponsors’ drug interactions. The FDA has indicated the registry staff should be trained to understand situations in which a medication guide is company policy and regulations on AE reporting required to be available and distributed to in order to ensure compliance with local patients.25 A medication guide alone can and regulations. This training includes the ability to frequently does constitute a REMS. identify and evaluate the attributes of each AE and • A communication plan that specifies targeted determine whether the AE should be reported to education and outreach for physicians, the health authority in keeping with local pharmacists, and patients.

286 Chapter 12. Adverse Event Detection, Processing, and Reporting

• Elements to assure safe use (ETASU), in some ETASU system and overall REMS should be cases. ETASU may include restriction of clearly stated (e.g., prevention of in utero exposure prescribing to health care providers with during therapy via routine pregnancy testing), and particular training, experience, and registration forms that document the physician’s certification; dispensing of the drug in and pharmacist’s attestation of their commitment restricted settings; documentation of safe use to requirements of the patient registry system conditions (such as laboratory results or should be completed prior to prescribing or specific patient monitoring); and registries.24 dispensing the product. Unlike the less structured disease or exposure registries discussed above, a restricted-access 9. Reporting Breaches of system associated with an ETASU is designed for Confidentiality or Other Risks approved products that have particular risk-benefit profiles that require more careful controls. The In addition to addressing regulatory purpose of ETASU is to mitigate a certain known responsibilities for reporting adverse events, drug-associated risk by ensuring that product registries must also understand regulatory and access is tightly linked to some preventive and/or ethical requirements and expectations regarding monitoring measure. Examples include systems breaches of confidentiality or the reporting of that monitor laboratory values, such as white blood other risks to patients that may arise during the cell counts during clozapine administration to course of a registry. The Health Information prevent severe leucopenia, or routine pregnancy Technology for Economic and Clinical Health Act testing during thalidomide administration to (HITECH Act) requires HIPAA-covered entities prevent in utero exposure of this known (entities covered by the Health Insurance teratogenic compound. When these programs Portability and Accountability Act of 1996) and include registries, the registries often prospectively their business associates to provide notification collect a battery of information using standardized following a breach of unsecured protected health instruments. information.26 See Chapter 7 for a detailed Data collection under ETASU may carry special discussion of the HITECH Act. State breach AE reporting requirements, and as a result of the notification laws may also apply to registry data. extensive contact with a variety of potential Beyond these legal requirements, registries should sources of safety information (e.g., pharmacists establish clear notification procedures for breaches and patients), care should be taken to identify all of confidentiality or other risks that become known possible routes of reporting. If special during the course of the registry, whether or not requirements exist, they should be made explicit in they are governed by HIPAA or subject to State the registry protocol, with clear definitions of laws. roles, responsibilities, and processes. Training of involved health care providers, such as physicians, References for Chapter 12 nurses, and pharmacists, can be undertaken with written instructions, via telephone or with face-to- 1. Baim DS, Mehran R, Kereiakes DJ, et al. face counseling. Training of these health care Postmarket surveillance for drug-eluting coronary providers should also extend beyond AE reporting stents: a comprehensive approach. Circulation. to the specific requirements of the program in 2006 Feb 14;113(6):891-7. PMID: 16476863. question. Such training may include the intended 2. U.S. Food and Drug Administration. Medical use and associated risk of the product, appropriate Device Reporting. http://www.fda.gov/ patient enrollment, and specific patient monitoring MedicalDevices/Safety/ReportaProblem/default. requirements, including guidelines for product htm. Accessed August 15, 2012. discontinuation and management of AEs, as well as topics to cover during comprehensive counseling of patients. The objectives of the

287 Section III. Operating Registries

3. Gross TP, Witten CM, Uldriks C, et al. A view 14. U.S. Food and Drug Administration. General from the US Food and Drug Administration. In: Instructions for Completing the Internet Frank E. Johnson KSV, ed. The Bionic Human: MedWatch Form. https://www.accessdata.fda.gov/ Health Promotion for People with Implanted scripts/medwatch/medwatch-online.htm. Accessed Prosthetic Devices. Totowa, New Jersey: Humana August 15, 2012. Press, Inc.; 2006:5. p. 61-87. 15. U.S. Food and Drug Administration. MedWatch 4. 21 CFR § 314.80 (2008). Revised April 1, 2011. Form FDA 3500 - Voluntary Reporting. http://www.fda.gov/downloads/AboutFDA/ 5. International Conference on Harmonisation of ReportsManualsForms/Forms/UCM163919.pdf. Technical Requirements for Registration of Accessed September 4, 2013. Pharmaceuticals for Human Use. ICH E2A: Clinical safety data management: definitions and 16. U.S. Food and Drug Administration. Guidance for standards for expedited reporting. 27 October Industry: MedWatch Form FDA 3500A: 1994. http://www.ich.org/fileadmin/Public_Web_ Mandatory Reporting of Adverse Reactions Site/ICH_Products/Guidelines/Efficacy/E2A/ Related to Human Cells, Tissues, and Cellular and Step4/E2A_Guideline.pdf. Accessed September 4, Tissue-Based Products (HCT/Ps). http://www.fda. 2013. gov/BiologicsBloodVaccines/ GuidanceComplianceRegulatoryInformation/ 6. U.S. Food and Drug Administration. Guidance for Guidances/Tissue/ucm074000.htm. Accessed Industry: Establishing Pregnancy Exposure August 15, 2012. Registries. http://www.fda.gov/downloads/Drugs/ GuidanceComplianceRegulatoryInformation/ 17. Vaccine Adverse Event Reporting System. Guidances/ucm071639.pdf. Accessed August 15, http://vaers.hhs.gov. Accessed August 15, 2012. 2012. 18. Council for International Organizations of 7. U.S. Food and Drug Administration. Medical Sciences. CIOMS Form. http://www. Postmarketing Adverse Experience Reporting for cioms.ch/index.php/cioms-form-i. Accessed Human Drug and Licensed Biological Products: September 4, 2013. Clarification of What to Report. http://www.fda. 19. 21 CFR § 314.80(f)(1) (2008). gov/downloads/Drugs/GuidanceCompliance RegulatoryInformation/Guidances/ucm071981. 20. EU Directive 2010/84/EU. 15 December 2010. pdf. Accessed August 15, 2012. http://eur-lex.europa.eu/LexUriServ/LexUriServ. do?uri=OJ:L:2010:348:0074:0099:EN:PDF. 8. 21 CFR § 312.32 (2008). Revised April 1, 2011. Accessed August 15, 2012. 9. 21 CFR § 803 (2008). 21. EU Regulation No 1235/2010. 15 December 2010. 10. Food and Drug Administration Amendments Act http://eur-lex.europa.eu/LexUriServ/LexUriServ. of 2007., Pub. L. No. 110-85 (2007), Title VI. do?uri=OJ:L:2010:348:0001:0016:EN:PDF. Sect. 603. Critical Path Public-Private Accessed August 15, 2012. Partnerships. 22. ICH Topic E2D. Post Approval Safety Data 11. U.S. Food and Drug Administration. Reporting Management. European Medicines Agency; Serious Problems to the FDA. http://www.fda.gov/ May2004. CPMP/ICH/3945/03. http://www.ich. Safety/MedWatch/HowToReport/default.htm. org/fileadmin/Public_Web_Site/ICH_Products/ Accessed August 15, 2012. Guidelines/Efficacy/E2D/Step4/E2D_Guideline. pdf. Accessed August 15, 2012. 12. Dreyer NA, Sheth N, Trontell A, et al. Good practices for handling adverse events detected 23. ICH E2C R1: Clinical Safety Data Management: through patient registries. Drug Information Periodic Updated Safety Reports for Marketed Association Journal. 2008 2008;5(42):421-8. Drugs. European Medicines Agency; Jun1997. CPMP/ICH/288/95. http://www.ema.europa.eu/ 13. U.S. Food and Drug Administration. Guidance for docs/en_GB/document_library/Scientific_ Industry. Good pharmacovigilance practices and guideline/2009/09/WC500002780.pdf. Accessed pharmacoepidemiologic assessment. March 2005. September 4, 2013. http://www.fda.gov/downloads/ RegulatoryInformation/Guidances/UCM126834. pdf. Accessed August 14, 2012.

288 Chapter 12. Adverse Event Detection, Processing, and Reporting

24. U.S. Food and Drug Administration. DRAFT Guidance for Industry: Format and content of proposed risk evaluation and mitigation strategies (REMS), REMS assessments, and proposed REMS modifications. September 2009. http:// www.fda.gov/downloads/Drugs/Guidance ComplianceRegulatoryInformation/Guidances/ UCM184128.pdf. Accessed August 15, 2012. 25. U.S. Food and Drug Administration. Guidance: Medication Guides – Distribution Requirements and Inclusion in Risk Evaluation and Mitigation Strategies (REMS). http://www.fda.gov/ downloads/Drugs/GuidanceCompliance RegulatoryInformation/Guidances/UCM244570. pdf. Accessed August 15, 2012. 26. 74 F.R. 42740 (August 24, 2009).

289

Chapter 13. Analysis, Interpretation, and Reporting of Registry Data To Evaluate Outcomes

1. Introduction used for registry data are presented, addressing how conclusions may be drawn from the data and Registries have the potential to produce databases what caveats are appropriate. The chapter also that are an important source of information describes how timelines for data analysis can be regarding health care patterns, decisionmaking, built in at registry inception and how to determine and delivery, as well as the subsequent association when the registry data are complete enough to of these factors with patient outcomes. Registries, begin analysis. for example, can provide valuable insight into the safety and/or effectiveness of an intervention or 2. Hypotheses and Purposes the efficiency, timeliness, quality, and patient centeredness of a health care system. The utility of the Registry and applicability of registry data rely heavily on While it may be relatively straightforward to the quality of the data analysis plan and its users’ develop hypotheses for registries intended to ability to interpret the results. Analysis and evaluate safety and effectiveness, not all registries interpretation of registry data begin with a series have specific, testable, or simple hypotheses. of core questions: Disease registries commonly have aims that are • Study purpose: Were the objectives/hypotheses primarily descriptive, such as describing the predefined or post hoc? typical clinical features of individuals with a • Patient population: Who was studied? disease, variations in phenotype, and the clinical progression of the disease over time (i.e., natural • Data quality: How were the data collected, history). These registries play a particularly reviewed, and verified? important role in the study of rare diseases. • Data completeness: How were missing data In the case of registries where the aim is to study handled? the associations between specific exposures and • Data analysis: How were the analyses chosen outcomes, prespecification of the study and performed? methodology and presence or absence of a priori While registry data present many opportunities for hypotheses or research questions may affect the meaningful analysis, there are inherent challenges acceptance of results of studies derived from to making appropriate inferences. A principal registry data. The many possible scenarios are well concern with registries is that of making inferences illustrated by examples at the theoretical extremes. without regard to the quality of data, since quality On one extreme, a study may evolve out of a clear standards have not been previously well and explicit prespecified research question and established or consistently reported. In some hypothesis. In such a study, there may have been registries, comparison groups may not be robustly preliminary scientific work that laid the conceptual defined, and information provided about the foundation and plausibility for the proposed study. external validity of a registry sample is often The investigators fully articulate the objectives and limited. These factors must be considered when analytic plan before embarking on any analysis. making inferences based on analyses of registry The outcome is clearly defined and the statistical data.1 approach documented. Secondary analyses are This chapter explains how analysis plans are identified and may be highlighted as hypothesis constructed for registries, how they differ generating. The investigators have no prior depending on the registry’s purpose, and how knowledge of analyses in this database that would registry design and conduct can affect analysis and bias them in the formulation of their study interpretation. The analytic techniques generally objective. The study is conducted and published

291 Section III. Operating Registries

regardless of the result. The paper states clearly population, five populations, each of which is a that the objective and hypothesis were subset of the preceding population, need to be prespecified. For registries intended to support considered, along with how well each population national coverage determinations with data represents the preceding population. These five collection as a condition of coverage, the specific subpopulations are shown in Figure 13–1. coverage decision question may be specified a The target population is defined by the study’s priori as the research question in lieu of a formal purpose. To assess the appropriateness of the target hypothesis. population, one must ask the question, “Is this At the other extreme, a study may evolve out of an really the population that we need to know about?” unexpected observation in a database in the course For example, the target population for a registry of of doing analyses for another purpose. A study oral contraceptive users would include women of could also evolve from a concerted effort to childbearing age who could become pregnant and discover associations—for example, as part of a are seeking to prevent pregnancy. Studies often large effort to understand disease causation. In miss important segments of the population in an such a study, the foundation for the study is effort to make the study population more developed post hoc, or after making the homogeneous. For example, a study to assess a observation. Because of the way in which the medical device used to treat patients for cardiac observation was found, the rationale for the study arrhythmias that defines only men as its target is developed retrospectively. The paper publishing population would be less informative than it could this study’s results does not clearly state that the be, because the device is designed for use in both objective and hypothesis were not prespecified. men and women. Of course, many examples fall between these The accessible population is defined using extremes. An investigator may suspect an inclusion criteria and exclusion criteria. The association for many variables but find the inclusion criteria define the population that will be relationship for only one of them. The investigator used for the study and generally include decides to pursue only the positive finding and geographic (e.g., hospitals or clinics in the New develop a rationale for a study or grant. The England region), demographic, disease-specific, association was sought, but it was sought along and temporal (e.g., specification of the included with associations for many other variables and dates of hospital or clinic admission), as well as outcomes. other criteria. Conversely, the exclusion criteria Thus, while there is substantial debate about the seek to eliminate specific patients from study and importance of prespecified hypotheses,2, 3 there is may be driven by an effort to assure an adequate- general agreement that it is informative to reveal sized population of interest for analysis. The same how the study was developed. Transparency in goals may be said of inclusion criteria, since it is methods is needed so that readers may know difficult to separate inclusion from exclusion whether these studies are the result of hypotheses criteria (e.g., inclusion of adults aged 18 and older developed independently of the study database, or vs. exclusion of children younger than 18). whether the question and analyses evolved from The accessible population may lose experience with the database and multiple representativeness to the extent that convenience iterations of exploratory analyses. Both types of plays a part in its determination, because people studies have value. who are easy to enroll in the registry may differ in some critical respects from the population at large. 3. Patient Population Similarly, to the extent that homogeneity plays a part in determining the accessible population, it is The purpose of a registry is to provide information less likely to be representative of the entire about a specific patient population to which all population because certain population subgroups study results are meant to apply. To determine how will be excluded. well the study results apply to the target

292 Chapter 13. Analysis, Interpretation, and Reporting of Registry Data To Evaluate Outcomes

Factors to be considered in assessing the accessible The reason for using an intended population rather population’s representativeness of the target than the whole accessible population for the study population include all the inclusion and exclusion is simply a matter of convenience and practicality. criteria mentioned above. One method of The issues to consider in assessing how well the evaluating representativeness is to describe the intended population represents the accessible demographics and other key descriptors of the population are similar to those for assessing how registry study population and to contrast its well the accessible population represents the target composition with patients with similar population. The main difference is that the characteristics who are identified from an external intended population may be specified by a database, such as might be obtained from health sampling scheme, which often tries to strike a insurers, health maintenance organizations, or the balance among representativeness, convenience, U.S. Surveillance Epidemiology and End Results and budget. If the intended population is a random (SEER) cancer registries.4 sample of the accessible population, it may be However, simple numerical/statistical reasonably assumed that it will represent the representativeness is not the main issue. accessible population; however, for many, if not Representativeness should be evaluated in the most, registries, a complete roster of the accessible context of the purpose of the study—that is, population does not exist. More commonly, the whether the study results can reasonably be intended population is compared with the generalized or extrapolated to other populations of accessible population in terms of pertinent interest outside of those included in the accessible variables. population. (See Case Example 26.) For example, To the extent that convenience or other design suppose that the purpose of the study is to assess (e.g., stratified random sample) is used to choose the effectiveness of a drug in U.S. residents with the intended population, one must consider the diabetes. If the accessible population includes no extent to which the sampling of the accessible children, then the study results may not apply to population—by means other than random children, since children often metabolize drugs sampling—has decreased the representativeness of very differently from adults. the intended population. For example, suppose On the other hand, consider the possibility that the that, for the sake of convenience, only patients accessible population is generally drawn from a who attend clinic on Mondays are included in the geographically isolated region, whereas the target study. If patients who attend clinic on Mondays are population may be the entire United States or the similar in every relevant respect to other patients, world. In that case, the accessible population is not that may not constitute a limitation. But if Monday geographically representative of the target patients are substantially different from patients population, but that circumstance would have little who attend clinic on other days of the week (e.g., or no impact on the representativeness of the study well-baby clinics are held on Mondays) and if findings to the target population if the action of the those differences affect the outcome that is being drug (or its delivery) does not vary geographically studied (e.g., proportion of baby visits for “well (which we would generally expect to be the case, babies”), then that sampling strategy would unless pertinent racial/genetic or dietary factors substantially alter the interpretations from the were involved). Therefore, in this example, the registry and would be considered a meaningful lack of geographical representativeness would not limitation. affect interpretation of results.

293 Section III. Operating Registries

Figure 13–1. Patient populations fewer side effects but that is more costly. Inclusion in the actual population may be influenced by prescribing practices governed by a health insurer. Target Population For example, if a new drug is approved for The population to which the study reimbursement only for patients who have “failed” findings are meant to apply treatment with other anti-inflammatory products, the resulting actual population will be systematically different from the target population of potential anti-inflammatory drug users. The Accessible Population actual population may be refractory to treatment or may have more comorbidities (e.g., gastrointestinal Subset of the target population who problems), and may be specifically selected for are specifically defined and available treatment beyond the intention of the study- for study specified inclusion criteria. In fact, registries of newly introduced drugs and devices may often include patients who are different from the ultimate target population. Indended Population Finally, the analytic population includes all those Members of the accessible patients who meet the criteria for analysis. In some population who are sampled cases, it becomes apparent that there are too few according to the reistry design cases of a particular type, or too few patients with certain attributes, such that these subgroups do not contribute enough information for meaningful analysis. Patients may also be excluded from the Actual Population analysis population because their conditions are so People who actually participate rare that to include them could be considered a in registry breach of patient confidentiality. Analytic populations are also created to meet specific needs. For example, an investigator may request a data set that will be used to analyze a subset of the registry population, such as those who had a specific Analytic Population treatment or condition. People who meet the criteria A related issue is that of early adopters,5 in which for analysis practitioners who are quick to use a novel health care intervention or therapy differ from those who use it only once it is well established. For example, The extent to which the actual population is not a registry of the use of a new surgical technique fully representative of the intended population is may initially enroll largely academic physicians generally a matter of real-world issues that prevent and only much later enroll community-based the initial inclusion of study subjects or adequate surgeons. If the outcomes of the technique differ followup. In assessing representativeness, one between the academic surgeons (early adopters) must consider the likely underlying factors that and community-based surgeons (later adopters), caused those subjects not to be included in the then the initial results of the registry may not analysis of study results and how that might affect reflect the true effectiveness of the technique in the interpretations from the registry. For example, widespread use. Patients selected for treatment consider a study of a newly introduced medication, with a novel therapy may also differ with regard to such as an anti-inflammatory drug that is thought factors such as severity or duration of disease and to be as effective as other products and to have prior treatment history, including treatment

294 Chapter 13. Analysis, Interpretation, and Reporting of Registry Data To Evaluate Outcomes

failures. For example, patients with more severe or For example, suppose the research question late-stage disease who have failed other treatments addresses the comparative effectiveness of two might be more likely to use a newly approved treatments for a given disease using an existing product that has shown efficacy in treating their registry. To be meaningful, the registry should condition. Later on, patients with less severe have accurate, well-defined, and complete disease may start using the product. information, including potential confounding and Finally, patients who are included in the analytic effect-modifying factors; population characteristics population for a given analysis of registry data of those with the specified disease; exposures may also be subject to selection or inclusion (whether patients received treatment A or B); and criteria (admissibility criteria), and these may patient outcomes of interest. Confounding factors affect interpretation of the resulting analyses. are variables that influence both the exposure (See Chapter 18.) For example, if only patients (treatment selection) and the outcome in the who remain enrolled and attend followup visits analyses. These factors can include patient factors through 2 years after study initiation are included (age, gender, race, socioeconomic factors, disease in analysis of adherence to therapy, it is possible or severity, or comorbid illness); provider factors likely that adherence among those who remain (experience, skills); and system factors (type of enrolled in the study and have multiple followup care setting, quality of care, or regional effects). visits will be different from adherence among While it is not possible to identify all confounding those who do not. Differential loss to followup, factors in planning a registry, it is desirable to give whereby patients who are lost may be more likely serious thought to what will be important and how to experience adverse outcomes, such as mortality, the necessary data can be collected. While effect than those who remain under observation, is a modification is not a threat to validity, it is related issue that may lead to biased results. important to consider potential effect modifiers for (See Chapter 3.) data collection and analysis in order to evaluate whether an association varies within specific subgroups.6 Analysis of registries requires 4. Data Quality Issues information about such variables so that the confounding covariates can be accounted for, using In addition to a full understanding of study design one of several analytic techniques covered in and methodology, analysis of registry events and upcoming sections of this chapter. In addition, as outcomes requires an assessment of data quality. described in Chapter 3, eligibility for entry into the One must consider whether most or all important registry may be restricted to individuals within a covariates were collected, whether the data were certain range of values for potential confounding complete, and whether the problem of missing data factors in order to reduce the effects of these was handled appropriately, as well as whether the factors. Such restrictions may also affect the data are accurate. generalizability of the registry. 4.1 Collection of All Important 4.2 Data Completeness Covariates Assuming that a registry has the necessary data While registries are generally constructed for a elements, the next step is to ensure that the data particular purpose or purposes, registry are complete. Missing data can be a challenge for information may be collected for one purpose any registry-based analysis. Missing data include (e.g., provider performance feedback) and then situations in which a variable is directly reported used for another (e.g., addressing a specific as missing or unavailable, a variable is clinical research question). When using an “nonreported” (i.e., the observation is blank), the available database for additional purposes, one reported data may not be interpretable, or the value needs to be sure that all the information necessary must be imputed to be missing because of data to address a specific research question was inconsistency or out-of-range results. Before collected in a manner that is sufficient to answer analyzing a registry database, the database should the question.

295 Section III. Operating Registries

be “cleaned” (discussed in Chapter 11, Section While pragmatically it is difficult to determine 2.5.), and attempts should be made to obtain as whether data are MCAR or MAR, there are, much missing data as realistically possible from nonetheless, several means of managing missing source documents. Inconsistent data (e.g., a “yes” data within an analysis. For example, a complete answer to a question at one point and “no” to the case strategy limits the analysis to patients with same question at another) and out-of-range data complete information for all variables. This is the (e.g., a 500-year-old patient) should be corrected default strategy used in many standard analytic when possible. Finally, the degree of data packages (e.g., SAS, Cary, NC). A simple deletion completeness should be summarized for the of all incomplete observations, however, is not researcher and eventual consumer of analyses from appropriate or efficient in all circumstances, and it the registry. Detailed examples of sources of may introduce significant bias if the deleted cases incomplete data are described in Chapter 18. are substantively different from the retained, complete cases (i.e., not MCAR). In observational 4.3 Missing Data studies with prospective, structured data The intent of any analysis is to make valid collection, missing data are not uncommon, and inferences from the data. Missing data can threaten the complete case strategy is inefficient and not this goal both by reducing the information yield of generally used. For example, patients with diabetes the study and, in many cases, by introducing bias. who were hospitalized because of inadequate A thorough review of types of missing data with glucose control might not return for a scheduled examples can be found in Chapter 18. Briefly, the followup visit at which HbA1c was to be first step is to understand which data are missing. measured. Those missing values for HbA1c would The second step is to understand why the data are probably differ from the measured values because missing (e.g., missing item-response or right of the reason for which they were missing, and censoring). Finally, missing data fall into three they would be categorized as MNAR. In an classic categories of randomness:7 example of MAR, the availability of the results of certain tests or measurements may depend on what • Missing completely at random (MCAR): is covered by patients’ health insurance (a known Instances where there are no differences value), since registries do not typically pay for between subjects with missing data and those testing. Patients without this particular with complete data. In such random instances, measurement may still contribute meaningfully to missing data only reduce study power without the analysis. In order to include patients with introducing bias. missing data, one of several imputation techniques • Missing at random (MAR): Instances where may be used to estimate the missing data. missing data depend on known or observed Imputation is a common strategy in which average values but not unmeasured data. In such cases, values are substituted for missing data using accounting for these known factors in the strategies such as unconditional and conditional analysis will produce unbiased results. mean, multiple hot-deck, and expectation • Missing not at random (MNAR): Here, missing maximum, among others.7, 8 For data that are data depend on events or factors not measured captured at multiple time points or repeated by the researcher and thus potentially introduce measures, investigators often “carry forward” a bias. last observation. However, such a technique can be To gain insight into which of the three categories problematic if early dropouts occur and a response of missing data are in play, one can compare the variable is expected to change over time or when distribution of observed variables for patients with the effect of the exposure (or treatment) is specific missing data to the distribution of those intermittent. Worst-case imputation is another variables for patients for whom those same data means of substitution in which investigators test are present. the sensitivity of a finding by substituting a worst-case value for all missing results. While this

296 Chapter 13. Analysis, Interpretation, and Reporting of Registry Data To Evaluate Outcomes

is conservative, it offers a lower bound to an and handle missing data as well as how to association rather than an accurate assessment. explicitly report on data completeness to facilitate One particular imputation method that has interpretation of study results. For more received significant attention in recent analyses has information on other specific types of missing data been termed multiple imputation. Rubin first issues (e.g., left truncation), please see Chapter 18. proposed the idea to impute more than one value for a missing variable as a means of reflecting the 4.4 Data Accuracy and Validation uncertainty around this value.9 The general While observational registry studies are usually strategy is to replace a missing value with multiple not required to meet U.S. Food and Drug values from an approximate distribution for Administration and International Conference on missing values. This produces multiple complete Harmonisation standards of Good Clinical Practice data sets for analysis from which a single summary developed for clinical trials, sponsors and contract finding is estimated. research organizations that conduct registry studies There are several issues concerning how are responsible for ensuring the accuracy of study prognostic models for decisionmaking can be data to the extent possible. Detailed plans for site influenced by data completeness and missing monitoring, quality assurance, and data data.10 Burton and Altman reviewed 100 verification should be developed at the beginning multivariable cancer prognostic models published of a study and adhered to throughout its lifespan. in seven leading cancer journals in 2002. They Chapter 11 discusses in detail approaches to data found that the proportion of complete cases was collection and quality assurance, including data reported in only 39 studies, while the percentage management, site monitoring, and source data missing for important prognostic variables was verification. reported in 52 studies. Comparison of complete Ensuring the accuracy and validity of data and cases with incomplete cases was provided in 10 programming at the analysis stage requires studies, and the methods used to handle missing additional consideration. The Office of data were summarized in 32 studies. The most Surveillance and Epidemiology (OSE) of the Food common techniques used for handling missing and Drug Administration’s Center for Drug data in this review article were (a) complete case Evaluation and Research uses the manual analysis (12), (b) dropping variables with high Standards of Data Management and Analytic numbers of missing cases from model Process in the Office of Surveillance and consideration (6), and (c) using some simple Epidemiology for analyses of databases conducted author imputation rule (6). Only one study within OSE; the manual addresses many of these reported using multiple imputation. The reviewers issues and may be consulted for further elaboration concluded that there was room for improvement in on these topics.14 Topics addressed that pertain to the reporting and handling of missing data within ensuring the accuracy of data just before and registry studies during analysis include developing a clear Readers interested in learning more about methods understanding of the data at the structural level of for handling missing data and the potential for bias the database and variable attributes; creating are directed to other useful resources by Greenland analytic programs with careful documentation and and Finkle,11 Hernán and colleagues,12 and Lash, an approach to variable creation and naming Fox, and Fink.13 conventions that is straightforward and, when possible, consistent with the Clinical Data It is important to keep in mind that the impact of Interchange Standards Consortium initiative; and data completeness will differ, depending on the complete or partial verification of programming extent of missing data and the intended use of the and analytic data set creation by a second analyst. registry. It may be less problematic with regard to descriptive research than research intended to For more detail about validation substudies, please support decisionmaking. For all registries, it is see Chapter 18. important to have a strategy for how to identify

297 Section III. Operating Registries

5. Data Analysis that can be attributed to a specific exposure, and it may be used to indicate the impact of a particular This section provides an overview of practical exposure at a population level. The standard considerations for analysis of data from a registry. textbooks cited here have detailed discussions As the name suggests, a descriptive study focuses regarding epidemiologic and statistical methods on describing frequency and patterns of various commonly used for the various analyses supported elements of a patient population, whereas an by registries.6, 16, 17,18, 19 analytical study focuses on examining associations For analytical studies of data derived from between patients or treatment characteristics and observational studies such as registries, it is health outcomes of interest (e.g., comparative important to consider the role of confounding. effectiveness). Although those planning a study try to collect as Statistical methods commonly used for descriptive much data as possible to address known purposes include those that summarize confounders, there is always the chance that information from continuous variables (e.g., mean, unknown confounders will affect the interpretation median) or from categorical variables (e.g., of analyses derived from observational studies. It proportions, rates). Registries may describe a is important to consider the extent to which bias population using incidence (the proportion of the (systematic error stemming from factors that are population that develops the condition over a related to both the decision to treat and the specified time interval) and prevalence (the outcomes of interest [confounders]) could have proportion of the population that has the condition distorted the results. For example, selective at a specific point in time). Another summary prescribing (confounding by indication) results estimate that is often used is an incidence rate. The when people with more severe disease or those incidence rate (also known as absolute risk) takes who have failed other treatments are more likely to into account both the number of people in a receive newer treatments; these patients are population who develop the outcome of interest systematically different from other patients who and the person-time at risk, or the length of time may be treated with the product under study. contributed by all people during the period when Misclassification in treatment can result from the they were in the population and the events were patient’s incorrect recall of dose, or poor adherence counted. or treatment compliance. Other types of bias include detection bias20 (e.g., when comparison For studies that include patient followup, an groups are assessed at different points in time or important part of the description of study conduct by different methods), selective loss to followup in is to characterize how many patients are “lost,” or which patients with the outcomes of most interest drop out, during the course of the registry, at what (e.g., sickest) may be more likely to drop out of point they are lost, and if they return. Lasagna one treatment group than another, and plots are one convenient method to visually assess performance bias (e.g., systematic differences in missing data over time when conducting a care other than the intervention under study, such longitudinal analysis.15 Figure 13–2 illustrates key as a public health initiative promoting healthy points of information that provide a useful lifestyles directed at patients who receive a description of losses to followup and study particular class of treatment). dropouts. Confounding may be evaluated using stratified For analytical studies, the association between a analysis, multivariable analysis, sensitivity risk factor and outcome may be expressed as analyses, and simple or quantitative bias attributable risk, relative risk, odds ratio, or hazard analysis.12 Appropriate methods should be used to ratio, depending on the nature of the data collected, adjust for confounding. For example, if an the duration of the study, and the frequency of the exposure or treatment varies over time and the outcome. Attributable risk, a concept developed in confounding variable also varies over time, the field of public health and preventive medicine, traditional adjustment using conventional is defined as the proportion of disease incidence

298 Chapter 13. Analysis, Interpretation, and Reporting of Registry Data To Evaluate Outcomes

multivariable modeling will introduce selection of interest. Such groupings may be accounted for bias. Marginal structural models use inverse in analysis through use of analytic methods probability weighting to account for time- including (ANOVA), and dependent confounding without introducing hierarchical or multilevel modeling.36-39 21 selection bias. The extensive information and Heterogeneity of treatment effect is also an large sample sizes available in some registries also important consideration for comparative support use of more advanced modeling effectiveness research as the effect of a treatment techniques for addressing confounding by may vary within subgroups of heterogeneous indication, such as the use of propensity scores to patients.40 Stratification on the propensity score create matched comparison groups, or for has been used to identify heterogeneity of stratification or inclusion in multivariable risk 22-25 treatment effect and may identify clinically modeling. New methods also include the meaningful differences between subgroups. high-dimensional propensity score (hd-PS) for adjustment using administrative data.26 The uptake For economic analyses, the analytic approaches of these approaches in the medical literature in often encountered are cost-effectiveness analyses recent years has been extremely rapid, and their and cost-utility studies. To examine cost- application to analyses of registry data has also effectiveness, costs are compared with clinical been broad. Examples are too numerous for a few outcomes measured in units such as life selections to be fully representative, but registries expectancy or years of disease avoided.41 Cost- in nearly every therapeutic area, including utility analysis, a closely related technique, cancer,27 cardiac devices,28 organ compares costs with outcomes adjusted for quality transplantation,29 and rare diseases,30 have of life (utility) using measures known as quality- published the results of analyses incorporating adjusted life years. Since most new interventions approaches based on propensity scores. As noted are more effective but also more expensive, in Chapter 3, instrumental variable methods another analytic approach examines the present opportunities for assessing and reducing incremental cost-effectiveness ratio and contrasts the impact of confounding by indication,31-33 but that to the willingness to pay. (Willingness-to-pay verification of the assumptions are important to analyses are generally conducted on a country-by- ensure that an instrument is valid.34 Violations in country basis, since various factors relating to the instrumental variable assumptions or the use of national health insurance practices and cultural a weak instrument will lead to results more biased issues affect willingness to pay.) The use of than those from conventional methods.35 While a registries for cost-effectiveness evaluations is a variety of methods have been developed to address fairly recent development, and consequently, the confounding, particularly confounding by methods are evolving rapidly. More information indication, residual confounding may still be about economic analyses can be found in standard present even after adjustment; therefore, these textbooks.42-47 methods may not fully control for unmeasured It is important to emphasize that cost-effectiveness confounding.35 For specific examples of the analyses, much like safety and clinical application of these methods, please see Chapter effectiveness analyses, require collection of 18. Information bias, such as misclassification, and specific data elements suited to the purpose. selection bias are also threats to the validity of our Although cost-effectiveness-type analyses are findings and examples can be found in Chapter 18. becoming more important and registries can play a For further information on how to quantify bias, key role in such analyses, registries traditionally please see Lash, Fox, and Fink.13 have not collected much information on quality of Groupings within a study population, such as life or resource use that can be linked to cost patients seen by a single clinician or practice, data.48 To be used for cost-effectiveness analysis, residents of a neighborhood, or other “clusters,” registries must be developed with that purpose in may themselves impact or predict health outcomes mind.

299 Section III. Operating Registries

Figure 13–2. The flow of participants into an analysis

Potential participants assessed for eligibility (n=...)

Excluded (n = ...) Ineligible n = Reasons... n =

Eligible (n=...)

Did not consent (n = ...) Refused n = Other reasons... n =

Consent to participate (n=...)

Only required if numbers Losses after consent (n = ...) consenting are not the Reasons... n = same as the numbers at baseline Numbers participating at baseline data collection (n = ...)

Losses after follow-up (n = ...) Reasons... n =

Only required if Numbers participating at nth >1 follow-up wave/s of data collection (n = ...)

Numbers participating at final wave of data collection (n = ...)

Tooth L, Ware R, Bain C. Quality of reporting of observational longitudinal research. Am J Epidemiol 2005; 161(3):280–8. Reprinted with permission. Copyright restrictions apply. By permission of Oxford University Press on behalf of The Johns Hopkins Bloomberg School of Public Health.

300 Chapter 13. Analysis, Interpretation, and Reporting of Registry Data To Evaluate Outcomes

5.1 Developing a Statistical Analysis 5.1.2 Preliminary Descriptive Analysis To Assist Plan SAP Development 5.1.1 Need for a Statistical Analysis Plan During SAP development, one particular aspect of a registry that is somewhat different from a It is important to develop a statistical analysis plan randomized controlled study is the necessity to (SAP) that describes the analytical principles and understand the “shape” of the data collected in the statistical techniques to be employed in order to study by conducting a simple stratified analysis.15 address the primary and secondary objectives, as This may be crucial for a number of reasons. specified in the study protocol or plan. Generally, the SAP for a registry study intended to support Given the broad inclusion criteria that most decisionmaking, such as a safety registry, is likely registries tend to propose, there might be a wide to be more detailed than the SAP for a descriptive distribution of patients, treatment, and/or outcome study or health economics study. A registry may characteristics. The distribution of age, for require a primary “master SAP” as well as example, may help to determine if more detailed subsequent, supplemental SAPs. Supplemental analyses should be conducted in the “oldest old” SAPs might be triggered by new research age group (80 years and older) to help understand questions emerging after the initial master SAP health outcomes in this subgroup that might be was developed or might be needed because the different from those of their younger counterparts. registry has evolved over time (e.g., additional data Unless a registry is designed to limit data collected, data elements revised). Although the collection to a fixed number of regimens, the study evolving nature of data collection practices in population may experience many “regimens,” some registries poses challenges for data analysis considering the combination of various dose and interpretation, it is important to keep in mind levels, drug names, frequency and timing of that the ability to answer questions emerging medication use (e.g., acute, chronic, intermittent), during the course of the study is one of the and sequencing of therapies. The scope and advantages (and challenges) of a registry. In the complexity of these variations constitute one of the specific case of long-term rare-disease registries, most challenging aspects of analyzing a registry, many of the relevant research questions of interest since treatment is given at each individual cannot be defined a priori but arise over time as physician’s discretion. Grouping of treatment into disease knowledge and treatment experience regimens for analysis should be done carefully, accrue. Supplemental SAPs can be developed only guided by clinical experts in that therapeutic area. when enough data become available to analyze a The full picture of treatment patterns may become particular research question. At times, the method clear only after a sizable number of patients have of statistical analysis may have to be modified to been enrolled. Consequently, the treatment accommodate the amount and quality of data definition in an SAP may be refined during the available. To the extent that the research question course of a study. Furthermore, there may be and SAP are formulated before the data analyses occasions where a particular therapeutic regimen are conducted and results are used to answer is used in a much smaller number of patients than specific questions or hypotheses, such anticipated, so that specific study objectives supplemental analysis retains much of the intent of focusing on this group of patients might become prespecification rather than being wide-ranging unfeasible. Also, the registry might have enrolled exploratory analyses (sometimes referred to as many patients who would normally be excluded “fishing expeditions”). The key to success is to from a clinical trial because of significant provide sufficient details in the SAP that, together contraindications related to comorbidity or with the study protocol and the case report forms, concomitant medication use. In this case, the SAP describe the overall process of the data analysis may need to define how these patients will be and reporting. analyzed (either as a separate group or as part of the overall study population) and how these

301 Section III. Operating Registries

different approaches might affect the interpretation can be considered. The timing of such analyses of the study results. may be influenced by regulatory requirements. There is a need to evaluate the presence of Second, it may be of interest to examine treatment potential sources of bias and, to the extent feasible, practices or health outcomes during the study to use appropriate statistical measures to address capture any emerging trends. Finally, it may also such biases. For example, the bias known as be important to provide intermediate or periodic confounding by indication49 results from the fact analysis to document progress, often as a that physicians do not prescribe medicine at requirement for continued funding. random: the reason a patient is put on a particular While it is useful to conduct such periodic regimen is often associated with their underlying analysis, careful planning should be given to the disease severity and may, in turn, affect treatment process and timing. The first questions are whether outcome. (See Chapter 18 for more detailed a sufficient number of patients have been enrolled discussion and examples.) To detect such a bias, and whether a sufficient number of events have the distribution of various prognostic factors at occurred. Answers to both questions can be baseline is compared for patients who receive a estimated based on the speed of enrollment and treatment of interest and those who do not. A rate of patient retention, as well as the expected related concept is channeling bias, in which drugs incidence rate of the event of interest. The second with similar therapeutic indications are prescribed issue is whether sufficient time has elapsed after to groups of patients who may differ with regard to the initial treatment with a product so that it is factors influencing prognosis.50 To detect such a biologically plausible for events to have occurred. bias, registry developers and users must document (For example, some events, such as site reactions the characteristics of the treated and untreated to injections, can be observed after a relatively participants and either demonstrate their short duration, compared with events like cancers, comparability or use statistical techniques to adjust which may have a long induction or latency.) If for differences where possible. (Additional there are too few patients or insufficient time has information about biases often found in registries elapsed, premature analyses may lead to the is detailed in Chapter 3, Section 10.) In addition to inappropriate conclusion that there is no such biases, analyses need to account for factors occurrence of a particular event. Similarly, that are interrelated, also known as effect uncommon events, occurring by random chance in modifiers.15 The presence of effect modification a limited sample, may be incorrectly construed as may also be identified after the data are collected. a safety signal. However, it is inappropriate to All of these issues should be taken into account in delay analysis so long that an opportunity might be an SAP, based on understanding of the patient missed to observe emerging safety outcomes. population in the registry. Investigators should use sound clinical and epidemiological judgment when planning an 5.2 Timing of Analyses during the intermediate analysis and, whenever possible, use Study data from previous studies to help to determine the Unlike a typical clinical trial, registries, especially feasibility and utility of such an analysis. those that take several years to complete, may When planning the timing of the analysis, it may conduct intermediate analyses before all patients be helpful to consider substudies if emerging have been enrolled and/or all data collection has questions require data not initially collected. been completed. Such midcourse analyses may be Substudies often involve data collection based on undertaken for several reasons. First, many of biological specimens or specific laboratory these registries focus on serious safety outcomes. procedures. They may, for example, take the form For such safety studies, it is important for all of nested case-control studies. In other situations, parties involved to actively monitor the frequency a research question may be applicable only to a of such events at regular predefined intervals so subset of patients, such as those who become that further risk assessment or risk management pregnant while in the study. It may also be

302 Chapter 13. Analysis, Interpretation, and Reporting of Registry Data To Evaluate Outcomes

desirable to conduct substudies among patients in may include infection, sensitivity reactions, a selected site or patient group to confirm the cancer, organ rejection, and mortality. validity of study measurement. In such instances, a Endpoints must be precisely defined at the data supplemental SAP may be a useful tool to describe collection and analysis stages. Are the study the statistical principles and methods. data on all-cause mortality or cause-specific mortality? Is information available on 5.3 Factors To Be Considered in the pathogen-specific infection (e.g., bacterial vs. Analysis viral)? (See Case Example 27.) Are there Registry results are most interpretable when they competing risks?54 are specific to well-defined endpoints or outcomes • Covariates: As with all observational studies, in a specific patient population with a specific comparative effectiveness research requires treatment status. Registry analyses may be more careful consideration, collection, and analysis meaningful if variations of study results across of important confounding and effect modifying patient groups, treatment methods, or subgroups of variables. For medication exposures, are dose, endpoints are reported. In other words, analysis of duration, and calendar time under a registry should explicitly provide the following consideration? Directed acyclic graphs (DAGs) information: can be useful tools to illustrate how the • Patient: What are the characteristics of the exposure (or treatment), outcome and 55, 56 patient population in terms of demographics, covariates are related. such as age, gender, race/ethnicity, insurance • Time: For valid analysis of risk or benefit that status, and clinical and treatment characteristics occurs over a period of time following therapy, (e.g., past history of significant medical detailed accounting for time factors is required. conditions, disease status at baseline, and prior For exposures, dates of starting and stopping a treatment history)? treatment or switching therapies should be • Exposure (or treatment): Exposure could be recorded. For outcomes, the dates when therapeutic treatment such as medication or followup visits occur, and whether or not they surgery; a diagnostic or screening tool; lead to a diagnosis of an outcome of interest, behavioral factors such as alcohol, smoking are required in order to take into account how habits, and diet; or other factors such as genetic long and how frequently patients were predisposition or environmental factors. What followed. Dates of diagnosis of outcomes of are the distributions of the exposure in the interest, or dates when patients complete a population? Is the study objective specific to screening tool or survey, should be recorded. At any one form of treatment? Is a new user the analysis stage, results must also be design being used?51 Does the exposure described in a time-appropriate fashion. For definition (index and reference group) and example, is an observed risk consistent over analysis avoid immortal-time bias?52 Are there time (in relation to initiation of treatment) in a repeated measures or is the exposure long-term study? If not, what time-related risk intermittent? measures should be reported in addition to or instead of cumulative risk? When exposure • Endpoints (or outcomes): Outcomes of interest status changes frequently, what is the method of may encompass effectiveness or comparative capturing the population at risk? Many effectiveness, the benefits of a health care observational studies of intermittent exposures 53 intervention under real-world circumstances, (e.g., use of nonsteroidal antiinflammatory and safety—the risks or harms that may be drugs or pain medications) use time windows associated with an intervention. Examples of of analysis, looking at events following first use effectiveness outcomes include survival, of a drug after a prescribed interval (e.g., 2 disease recurrence, symptom severity, quality weeks) without drug use. Different analytic of life, and cost-effectiveness. Safety outcomes

303 Section III. Operating Registries

approaches may be required to address issues effective therapy and those who do not (e.g., of patients enrolling in a registry at different underlying disease severity or contraindications), times and/or having different lengths of or it may not be feasible to maintain a long-term observation during the study period. cohort of patients who are not treated with such a • Potential for bias: Successful analysis of medication. It is known that external information observational studies also depends to a large about treatment practices (such as scientific extent on the ability to measure and analytically publications or presentations) can result in address the potential for bias. Refer to Chapter physicians changing their practice, such that they 3, Section 10 for a description of potential no longer prescribe the previously accepted sources of bias. Directed acyclic graphs can standard of care. There may be a systematic also be useful for understanding and identifying difference between physicians who are early the source of bias.55, 56 Details and examples of adopters and those who start using the drug or quantification of bias can be found in Chapter device after its effectiveness has been more widely 18. For details on how to quantify potential accepted. Early adopters may also share other bias, see the textbook by Lash, Fox, and Fink.13 practices that differentiate them from their later- adopting colleagues.5 5.3.1. Choice of Comparator In the absence of a good internal comparator, one An example of a troublesome source of bias is the may have to leverage external comparators to choice of comparator. When participants in a provide critical context to help interpret data cohort are classified into two or more groups revealed by a registry. An external or historical according to certain study characteristics (such as comparison may involve another study or another treatment status, with the “standard of care” group database that has disease or treatment as the comparator), the registry is said to have an characteristics similar to those of registry subjects. internal or concurrent comparator. The advantage Such data may be viewed as a context for of an internal comparator design is that patients anticipating the rate of an event. One widely used are likely to be more similar to each other, except comparator is the U.S. SEER cancer registry data, for their treatment status, than patients in because SEER provides detailed annual incidence comparisons between registry subjects and rates of cancer stratified by cancer site, age group, external groups of subjects. When defining the gender, and tumor staging at diagnosis. SEER comparator group, it is important not to introduce represents 28 percent of the U.S. population.57 A immortal time bias.52 In addition, consistency in procedure for formalizing comparisons with measurement of specific variables and in data external data is known as standardized incidence collection methods make the comparison more rate or ratio;15 when used appropriately, it can be valid. Internal comparators are particularly useful interpreted as a proxy measure of risk or relative for treatment practices that change over time. risk. Comparative effectiveness studies may often necessitate use of an internal comparator in order Use of an external comparator, however, may to maximize the comparability of patients present significant challenges. For example, SEER receiving different treatments within a given study, and a given registry population may differ from and to ensure that variables required for each other for a number of reasons. The SEER multivariable analysis are available and measured data cover the general population and have no in an equivalent manner for all patients to be exclusion criteria pertaining to history of smoking analyzed. or cancer screening, for example. On the other hand, a given registry may consist of patients who Unfortunately, it is not always possible to have or have an inherently different risk of cancer than the sustain a valid internal comparator. For example, general population, resulting from the registry’s there may be significant medical differences having excluded smokers and others known to be between patients who receive a particularly

304 Chapter 13. Analysis, Interpretation, and Reporting of Registry Data To Evaluate Outcomes

at high risk of developing a particular cancer. Such may be used in sensitivity analyses or bias a registry would be expected to have a lower analyses to determine the robustness of a registry’s overall incidence rate of cancer, which, if SEER findings. Sensitivity analysis refers to a procedure incidence rates are used as a comparator, may used to determine how robust the study result is to complicate or confound assessments of the impact alterations of various parameters. If a small of treatment on cancer incidence in the registry. parameter alteration leads to a relatively large Regardless of the choice of comparator, similarity change in the results, the results are said to be between the groups under comparison should not sensitive to that parameter. Sensitivity and bias be assumed without careful examination of the analyses may be used to determine how the final study patients. Different comparator groups may study results might change when taking into result in very different inferences for safety and account those lost to followup. A simple effectiveness evaluations; therefore, analysis of hypothetical example is presented in Table 13–1. registry findings using different comparator groups

Table 13–1. Hypothetical simple sensitivity analysis

Impact of Loss to Followup on Incidence Rates per 1,000 in a Study of 1,000 Patients in a Registry

Assuming a 10% Assuming a 30% Various Assumptions of the Observed Incidence Rate Loss to Followup Loss to Followup

Assuming that the incidence of patients lost to followup is X times the rate of incidence estimated in those who stayed in the registry: X=0.5 106 94 X=1 111 110 X=2 122 143 X=3 156 242

Table 13–1 illustrates the extent of change in the being exposed and the other being nonexposed. In incidence rate of a hypothetical outcome assuming that case, the impact of loss to followup on the varying degrees of loss to followup, and relative risk could be estimated by using sensitivity differences in incidence between those for whom analysis. More examples are included in Chapter there is information and those for whom there is 18. no information due to loss to followup. In the first 5.3.2 Patient Censoring example, where 10 percent of the patients are lost to followup, the estimated incidence rate of At the time of a registry analysis, events may not 111/1,000 people is reasonably stable; it does not have occurred for all patients. For these patients, change too much when the (unknown) incidence in the data are said to be censored, indicating that the those lost to followup changes from 0.5 times the observation period of the registry was stopped observed to 5 times the observed, with the before all events occurred (e.g., mortality). In corresponding incidence rate that would have been these situations, it is unclear when the event will observed ranging from 106 to 156 per 1,000. On occur, if at all. In addition, a registry may enroll the other hand, when the loss to followup increases patients until a set stop date, and patients entered to 30 percent, the corresponding incidence rates into the registry earlier will have a greater that would have been observed range from 94 to probability of having an event than those entered 242. This procedure could be extended to a study more recently because of the longer followup. An that has more than one cohort of patients, with one important assumption, and one that needs to be

305 Section III. Operating Registries

assessed in a registry, is how patient prognosis survival analyses59 and issues of recurrent varies with the time of entrance into the registry. outcomes and repeated measures, with or without This issue may be particularly problematic in missing data,60 in longitudinal cohort studies. registries that assess innovative (and changing) Other texts address a range of regression and therapies. Patients and outcomes initially observed nonregression approaches to analysis of case- in the registry may differ from patients and control and cohort study designs61 that may be outcomes observed later in the registry timeframe, applied to registries. either because of true differences in treatment options available at different points in time, or 7. Interpretation of Registry because of the shorter followup for people who entered later. Patients with censored data, however, Data contribute important information to the registry Interpretation of registry data is needed so that the analysis. When possible, analyses should be lessons from the registry can be applied to the planned so as to include all subjects, including target population and used to change future health those censored before the end of the followup care and improve patient outcomes. Proper period or the occurrence of an event. One method interpretation of registry data allows users to of analyzing censored data to estimate the understand the precision of the observed risk or conditional probability of the event occurring is to incidence estimates, to evaluate the hypotheses use the Kaplan-Meier method.58 In this method, tested in the current registry, and often also to for each time period, the probability is calculated generate new hypotheses to be examined in future that those who have not experienced an event registries or in randomized controlled trials. If the before the beginning of the period will still not purpose of the registry is explicit, the actual have experienced it by the end of the period. The population studied is reasonably representative of probability of an event occurring at any given time the target population, the data quality monitored, is then calculated from the product of the and the analyses performed so as to reduce conditional probabilities of each time interval. potential biases, then the interpretation of the For information about right censoring and left registry data should allow a realistic picture of the truncation, please see Chapter 18. quality of medical care, the natural history of the disease studied, or the safety, effectiveness, or 6. Summary of Analytic value of a clinical evaluation. Each of these topics needs to be discussed in the interpretation of the Considerations registry data, and potential shortcomings should be In summary, a meaningful analysis requires careful explored. Assumptions or biases that could have consideration of study design features and the influenced the outcomes of the analyses should be nature of the data collected. Most typical highlighted and separated from those that do not epidemiological study analytical methods can be affect the interpretation of the registry results. The applied, and there is no one-size-fits-all approach. use of a comparator of the highest reasonably Efforts should be made to carefully evaluate the possible quality is integral to the proper presence of biases and to control for identified interpretation of the analysis. potential biases during data analysis. This requires Interpretation of registry results may also be aided close collaboration among clinicians, by comparisons with external information. epidemiologists, statisticians, study coordinators, Examples include rates, or prevalence, of the and others involved in the design, conduct, and outcomes of interest in other studies and different interpretation of the registry. data sources (taking into account reasons why they A number of biostatistics and epidemiology may be similar or different). Such comparisons can textbooks cover in depth the issues raised in this put the findings of registry analyses within the section and the appropriate analytic approaches for context of previous study results and other addressing them—for example, “time-to-event” or pertinent clinical and biological considerations as

306 Chapter 13. Analysis, Interpretation, and Reporting of Registry Data To Evaluate Outcomes

to the validity and generalizability of the results. recommendations concerning continued use and/or Once analyzed, registries provide important adaptation of the registry and to evaluate patient feedback to several groups. First analysis and safety. The final group consists of the end users of interpretation of the registry will demonstrate the registry output, such as patients or other health strengths and limitations of the original registry care consumers, health services researchers, health design and will allow the registry developers to care providers, and policymakers. These are the make needed design changes for future versions of people for whom the data were collected and who the registry. Another group consists of the study’s may use the results to choose a treatment or sponsors and related oversight/governance groups, intervention, to determine the need for additional such as the scientific committee and data research programs to change clinical practice, to monitoring committee. (Refer to Chapter 2, develop clinical practice guidelines, or to Section 2.6 for more information on registry determine policy. Ideally, all three user groups governance and oversight.) Interpretation of the work toward the ultimate goal of each registry— analyses allows the oversight committees to offer improving patient outcomes.

307 Section III. Operating Registries

Case Examples for Chapter 13

Case Example 26. Using registry data to As a large, observational, prospective registry, evaluate outcomes by practice ESCF collected data on a large number of patients Description The Epidemiologic Study of from a range of participating sites. At the time of Cystic Fibrosis (ESCF) Registry the outcomes study, the registry was estimated to was a multicenter, encounter- have data on over 80 percent of CF patients in the based, observational, United States, and it collected data from more postmarketing study designed to than 90 percent of the sites accredited by the U.S. monitor product safety, define Cystic Fibrosis Foundation. Because the registry clinical practice patterns, explore contained a representative population of CF risks for pulmonary function patients, the registry database offered strong decline, and facilitate quality potential for analyzing the association between improvement for cystic fibrosis practice patterns and outcomes. (CF) patients. The registry Proposed Solution collected comprehensive data on In designing the study, the team decided to pulmonary function, compare CF sites using lung function (i.e., FEV1 microbiology, growth, [forced expiratory volume in 1 second] values), a pulmonary exacerbations, common surrogate outcome for respiratory CF-associated medical studies. Data from 18,411 patients followed in conditions, and chronic and 194 care sites were reviewed, and 8,125 patients acute treatments for children and from 132 sites (minimum of 50 patients per site) adult CF patients at each visit to were included. Only sites with at least 10 patients the clinical site. in a specified age group (ages 6–12, 13–17, and Sponsor Genentech, Inc. 18 or older) were included for evaluation of that Year Started 1993 age group. For each age group, sites were ranked in quartiles based on the median FEV1 value at Year Ended Patient enrollment completed in each site. The frequency of patient monitoring 2005; followup complete. and use of therapeutic interventions were No. of Sites 215 sites over the life of the compared between upper and lower quartile sites registry after stratification for disease severity. No. of Patients 32,414 patients and 832,705 Results encounters recorded Substantial differences in lung health across Challenge different CF care sites were observed. Within-site tended to be consistent across the three Although guidelines for managing cystic fibrosis age groups. Patients who were cared for at patients have been widely available for many higher- sites had more frequent years, little is known about variations in practice monitoring of their clinical status, measurements patterns among care sites and their associated of lung function, and cultures for respiratory outcomes. To determine whether differences in pathogens. These patients also received more lung health existed between groups of patients interventions, particularly intravenous antibiotics attending different CF care sites, and to determine for pulmonary exacerbations. The study whether these differences were associated with concluded that frequent monitoring and increased differences in monitoring and intervention, data use of appropriate medications in the on a large number of CF patients from a wide management of CF are associated with improved variety of CF sites were necessary. outcomes.

308 Chapter 13. Analysis, Interpretation, and Reporting of Registry Data To Evaluate Outcomes

Case Example 26. Using registry data to For More Information evaluate outcomes by practice (continued) Johnson C, Butler SM, Konstan MW. et al. Key Point Factors influencing outcomes in cystic fibrosis: a Stratifying patients by quartile of lung function, center-based analysis. Chest. 2003;123:20–7. age, and disease severity allowed comparison of Padman R, McColley SA, Miller DP. et al. Infant practices among sites and revealed practice care patterns at Epidemiologic Study of Cystic patterns that were associated with better clinical Fibrosis sites that achieve superior childhood lung status. The large numbers of patients and sites function. Pediatrics. 2007;119:E531–7. allowed for sufficient information to create meaningful and informative stratification, and resulted in sufficient information within those strata to reveal meaningful differences in site practices.

Case Example 27. Using registry data to study showing that it reduces the rate of RSV patterns of use and outcomes hospitalizations. To capture postlicensure patient Description The Palivizumab Outcomes demographic outcome information, the Registry was designed to manufacturer wanted to create a prospective study characterize the population of that identified infants receiving palivizumab. The infants receiving prophylaxis for objectives of the study were to better understand respiratory syncytial virus the population receiving the prophylaxis for RSV (RSV) disease, to describe the disease and to study the patterns of use and patterns and scope of the use of hospitalization outcomes. palivizumab, and to gather data Proposed Solution on hospitalization outcomes. A multicenter registry study was created to Sponsor MedImmune, LLC collect data on infants receiving palivizumab Year Started 2000 injections. No control group was included. The registry was initiated during the 2000–2001 RSV Year Ended 2004 season. Over 4 consecutive years, 256 sites across No. of Sites 256 the United States enrolled infants who had received palivizumab for RSV under their care, No. of Patients 19,548 infants provided that the infant’s parent or legally Challenge authorized representative gave informed consent for participation in the registry. Data were RSV is the leading cause of serious lower collected by the primary health care provider in respiratory tract disease in infants and children the office or clinic setting. The registry was and the leading cause of hospitalizations limited to data collection related to subjects’ usual nationwide for infants under 1 year of age. medical care. Infants were enrolled at the time of Palivizumab was approved by the U.S. Food and their first injection, and data were obtained on Drug Administration (FDA) in 1998 and is palivizumab injections, demographics, and risk indicated for the prevention of serious lower factors, as well as on medical and family history. respiratory tract disease caused by RSV in pediatric patients at high risk of RSV disease. Followup forms were used to collect data on Two additional large retrospective surveys subsequent palivizumab injections, including conducted after FDA approval studied the dates and doses, during the RSV season. effectiveness of palivizumab in infants, again Compliance with the prescribed injection

309 Section III. Operating Registries

Case Example 27. Using registry data to study receiving palivizumab prophylaxis for RSV in a patterns of use and outcomes (continued) large nationwide cohort of infants from a Proposed Solution (continued) geographically diverse group of practices and clinics. The registry data also showed that the use schedule was determined by comparing the of palivizumab was mostly consistent with the number of injections actually received with the 2003 guidelines of the American Academy of number of expected doses, based on the month Pediatrics for use of palivizumab for prevention that the first injection was administered. Infants of RSV infections. As the registry was conducted who received their first injection in November prospectively, nearly complete demographic were expected to receive five injections, whereas information and approximately 99 percent of infants receiving their first injection in February followup information was captured on all enrolled would be expected to receive only two doses infants, an improvement compared with through March. Data were also collected for all previously completed retrospective studies. enrolled infants hospitalized for RSV and were directly reported to an onsite registry coordinator. Key Point Testing for RSV was performed locally, at the A simple stratified analysis was used to describe discretion of the health care provider. Adverse the characteristics of infants receiving injections events were not collected and analyzed separately to help prevent severe RSV disease. Infants in the for purposes of this registry. Palivizumab is registry had a low hospitalization rate, and these contraindicated in children who have had a data support the effectiveness of this treatment previous significant hypersensitivity reaction to outside of a controlled clinical study. Risk factors palivizumab. Cases of anaphylaxis and for RSV hospitalizations were described and anaphylactic shock, including fatal cases, were quantified by presenting the number of infants reported following initial exposure or re-exposure with RSV hospitalization as a percentage of all to palivizumab. Other acute hypersensitivity enrolled infants who were hospitalized. These reactions, which might have been severe, were data supported an analysis of postlicensure also reported on initial exposure or re-exposure to effectiveness of RSV prophylaxis, in addition to palivizumab. Adverse reactions occurring greater describing the patient population and usage than or equal to 10 percent and at least 1 percent patterns. more frequently than placebo are fever and rash. For More Information In postmarketing reports, cases of severe thrombocytopenia (platelet count <50,000/ Leader S, Kohlhase K. Respiratory syncytial microliter) and injection site reactions were virus-coded pediatric hospitalizations, 1997-1999. reported. Ped Infect Dis J. 2002;21(7):629–32. Results Frogel M, Nerwen C, Cohen A. et al. Prevention of hospitalization due to respiratory syncytial From September 2000 through May 2004, the virus: Results from the Palivizumab Outcomes registry collected data on 19,548 infants. The Registry. J Perinatol. 2008;28:511–7. analysis presented injection rates and hospitalization rates for all infants by month of American Academy of Pediatrics—Committee on injection and by site of first dose (pediatrician’s Infectious Disease. Red Book 2003: Policy office or hospital). The observed number of Statement: Revised indications for the use of injections per infant was compared with the palivizumab and respiratory syncytial virus expected number of doses based on the month the immune globulin intravenous for the prevention first injection was given. Over 4 years of data of respiratory syncytial virus infections. collection, less than 2 percent (1.3%) of enrolled Pediatrics. 2003;112:1442-6. infants were hospitalized for RSV. This analysis confirmed a low hospitalization rate for infants

310 Chapter 13. Analysis, Interpretation, and Reporting of Registry Data To Evaluate Outcomes

References for Chapter 13 12. Hernan MA, Hernandez-Diaz S, Werler MM, et al. Causal knowledge as a prerequisite for 1. Sedrakyan A, Marinac-Dabic D, Normand SL, et confounding evaluation: an application to birth al. A framework for evidence evaluation and defects epidemiology. Am J Epidemiol. 2002 Jan methodological issues in implantable device 15;155(2):176-84. PMID: 11790682. studies. Med Care. 2010 Jun;48(6 Suppl):S121-8. 13. Lash TL, Fox MP, Fink AK. Applying quantitative PMID: 20421824. bias analysis to epidemiologic data. New York, 2. Cole P. The hypothesis generating machine. NY: Springer; 2009. Epidemiology. 1993 May;4(3):271-3. 14. U.S. Food and Drug Administration. Office of PMID: 8512992. Surveillance and Epidemiology, Center for Drug 3. Yusuf S, Wittes J, Probstfield J, et al. Analysis and Evaluation and Research. Standards for Data interpretation of treatment effects in subgroups of Management and Analytic Processes in the Office patients in randomized clinical trials. JAMA. 1991 of Surveillance and Epidemiology (OSE). March Jul 3;266(1):93-8. PMID: 2046134. 3, 2008. http://www.fda.gov/downloads/ AboutFDA/ReportsManualsForms/ 4. National Cancer Institute. Surveillance StaffPoliciesandProcedures/ucm082060.pdf. Epidemiology and End Results. http://seer.cancer. Accessed August 15, 2012. gov. Accessed August 27, 2012. 15. Swihart BJ, Caffo B, James BD, et al. Lasagna 5. Schneeweiss S, Gagne JJ, Glynn RJ, et al. plots: a saucy alternative to spaghetti plots. Assessing the comparative effectiveness of newly Epidemiology. 2010 Sep;21(5):621-5. marketed medications: methodological challenges PMID: 20699681. PMCID: 2937254. and implications for drug development. Clin Pharmacol Ther. 2011 Dec;90(6):777-90. 16. Hennekens CH, Buring JE, Mayrent SL. PMID: 22048230. Epidemiology in medicine. 1st ed. Boston: Little, Brown and Company; 1987. 6. Rothman K, Greenland S, Lash TL, eds. Modern Epidemiology. 3rd ed. New York: Lippincott 17. Kleinbaum DG, Kupper LL, Miller KE, et al. Williams & Wilkins; 2008. Applied and other multivariable methods. Belmont, CA: Duxbury 7. Little RJA, Rubin DB. Statistical analysis with Press; 1998. missing data. New York: John Wiley & Sons; 1987. 18. Aschengrau A, Seage G. Essentials of epidemiology in public health. Sudbury, MA: 8. Barzi F, Woodward M. Imputations of missing Jones & Bartlett; 2003. values in practice: results from imputations of serum cholesterol in 28 cohort studies. Am J 19. Rosner B. Fundamentals of biostatistics. 5th ed. Epidemiol. 2004 Jul 1;160(1):34-45. Boston: Duxbury Press; 2000. PMID: 15229115. 20. Higgins J, Green S.The Cochrane Collaboration. 9. Rubin DB. Multiple imputations in sample The Cochrane Handbook for Systematic Reviews surveys - a phenomenological Bayesian approach Of Interventions. 2006. http://www.cochrane.org/ to nonresponse. Proceedings of the Section on sites/default/files/uploads/Handbook4.2.6Sep2006. Survey Research Methods, American Statistical pdf. Accessed August 15, 2012. Association 1978. pp. 20-34. 21. Robins JM, Hernan MA, Brumback B. Marginal 10. Burton A, Altman DG. Missing covariate data structural models and causal inference in within cancer prognostic studies: a review of epidemiology. Epidemiology. 2000 Sep;11(5): current reporting and proposed guidelines. Br J 550-60. PMID: 10955408. Cancer. 2004 Jul 5;91(1):4-8. PMID: 15188004. 22. Mangano DT, Tudor IC, Dietzel C, et al. The risk PMCID: 2364743. associated with aprotinin in cardiac surgery. 11. Greenland S, Finkle WD. A critical look at N Engl J Med. 2006 Jan 26;354(4):353-65. methods for handling missing covariates in PMID: 16436767. epidemiologic regression analyses. Am J Epidemiol. 1995 Dec 15;142(12):1255-64. PMID: 7503045.

311 Section III. Operating Registries

23. Cepeda MS, Boston R, Farrar JT, et al. about/annualmtg08/090908slides/Brookhart.htm. Comparison of logistic regression versus Accessed August 14, 2012. propensity score when the number of events is low 32. Angrist JD, Imbens GW, Rubin DB. Identification and there are multiple confounders. Am J of causal effects using instrumental variables. Epidemiol. 2003 Aug 1;158(3):280-7. Journal of the American Statistical Association. PMID: 12882951. 1996;91(434):444-55. 24. Sturmer T, Joshi M, Glynn RJ, et al. A review of 33. Brookhart MA, Rassen JA, Schneeweiss S. the application of propensity score methods Instrumental variable methods in comparative yielded increasing use, advantages in specific safety and effectiveness research. settings, but not substantially different estimates Pharmacoepidemiol Drug Saf. 2010 compared with conventional multivariable Jun;19(6):537-54. PMID: 20354968. methods. J Clin Epidemiol. 2006 May;59(5): PMCID: 2886161. 437-47. PMID: 16632131. PMCID: 1448214. 34. Brookhart MA, Schneeweiss S. Preference-based 25. Glynn RJ, Schneeweiss S, Sturmer T. Indications instrumental variable methods for the estimation for propensity scores and review of their use in of treatment effects: assessing validity and pharmacoepidemiology. Basic Clin Pharmacol interpreting results. Int J Biostat. Toxicol. 2006 Mar;98(3):253-9. 2007;3(1):Article 14. PMID: 19655038. PMID: 16611199. PMCID: 1790968. PMCID: 2719903. 26. Schneeweiss S, Rassen JA, Glynn RJ, et al. 35. Hernan MA, Robins JM. Instruments for causal High-dimensional propensity score adjustment in inference: an epidemiologist’s dream? studies of treatment effects using health care Epidemiology. 2006 Jul;17(4):360-72. claims data. Epidemiology. 2009 Jul;20(4):512-22. PMID: 16755261. PMID: 19487948. PMCID: 3077219. 36. Merlo J, Chaix B, Yang M, et al. A brief 27. Reeve BB, Potosky AL, Smith AW, et al. Impact of conceptual tutorial of multilevel analysis in social cancer on health-related quality of life of older epidemiology: linking the statistical concept of Americans. J Natl Cancer Inst. 2009 Jun clustering to the idea of contextual phenomenon. 16;101(12):860-8. PMID: 19509357. J Epidemiol Community Health. 2005 PMCID: 2720781. Jun;59(6):443-9. PMID: 15911637. 28. Brodie BR, Stuckey T, Downey W, et al. Outcomes PMCID: 1757045. with drug-eluting stents versus bare metal stents in 37. Holden JE, Kelley K, Agarwal R. Analyzing acute ST-elevation myocardial infarction: results change: a primer on multilevel models with from the Strategic Transcatheter Evaluation of applications to nephrology. Am J Nephrol. New Therapies (STENT) Group. Catheter 2008;28(5):792-801. PMID: 18477842. Cardiovasc Interv. 2008 Dec 1;72(7):893-900. PMCID: 2613435. PMID: 19016465. 38. Diez-Roux AV. Multilevel analysis in public health 29. Shuhaiber JH, Kim JB, Hur K, et al. Survival of research. Annu Rev Public Health. 2000;21: primary and repeat lung transplantation in the 171-92. PMID: 10884951. United States. Ann Thorac Surg. 2009 Jan;87(1):261-6. PMID: 19101309. 39. Leyland AH, Goldstein H. Multilevel modeling of health statistics. Chichester, UK: John Wiley & 30. Grabowski GA, Kacena K, Cole JA, et al. Dose- Sons, LTD; 2001. response relationships for enzyme replacement therapy with imiglucerase/alglucerase in patients 40. Varadhan R, Segal JB, Boyd CM, et al. A with Gaucher disease type 1. Genet Med. 2009 framework for the analysis of heterogeneity of Feb;11(2):92-100. PMID: 19265748. treatment effect in patient-centered outcomes research. Journal of clinical epidemiology. 31. Brookhart, M. Alan. Instrumental Variables for 2013;66(8):818-25. Comparative Effectiveness Research: A Review of Applications. Slide Presentation from the AHRQ 41. Palmer AJ. Health economics--what the 2008 Annual Conference (Text Version); nephrologist should know. Nephrol Dial Rockville, MD: Agency for Healthcare Research Transplant. 2005 Jun;20(6):1038-41. and Quality; Jan, 2009. http://www.ahrq.gov/ PMID: 15840678.

312 Chapter 13. Analysis, Interpretation, and Reporting of Registry Data To Evaluate Outcomes

42. Neumann PJ. Opportunities and barriers. Using 52. Suissa S. Immortal time bias in observational cost-effectiveness analysis to improve health care. studies of drug effects. Pharmacoepidemiol Drug New York, NY: Oxford University Press; 2004. Saf. 2007 Mar;16(3):241-9. PMID: 17252614. 43. Edejer TT-T, Baltussen R, Adam T, et al. Making 53. Haynes RB, Sackett DL, Guyatt GH, et al. Clinical choices in health: WHO guide to cost- epidemiology. 3rd ed. New York: Lippincott effectiveness analysis. Geneva: World Health Williams & Wilkens; 2005. Organization; 2004. 54. Andersen PK, Geskus RB, de Witte T, et al. 44. Drummond M, Stoddart G, Torrance G. Methods Competing risks in epidemiology: possibilities for the economic evaluation of health care and pitfalls. Int J Epidemiol. 2012 Jun;41(3):861- programmes. 3rd ed. New York: Oxford University 70. PMID: 22253319. PMCID: 3396320. Press; 2005. 55. Greenland S, Pearl J, Robins JM. Causal diagrams 45. Muennig P. Designing and conducting cost- for epidemiologic research. Epidemiology. 1999 effectiveness analyses in medicine and health care. Jan;10(1):37-48. PMID: 9888278. New York: John Wiley & Sons, LTD; 2002. 56. Hernan MA, Hernandez-Diaz S, Robins JM. 46. Haddix AC, Teutsch SM, Corso PS. Prevention A structural approach to selection bias. effectiveness: a guide to decision analysis and Epidemiology. 2004 Sep;15(5):615-25. economic evaluation. New York: Oxford PMID: 15308962. University Press; 2003. 57. National Cancer Institute. “SEER: Surveillance 47. Gold MR, Siegel JE, Russell LB, et al. Cost- Epidemiology and End Results.” http://seer.cancer. effectiveness in health and medicine: the Report gov/about/factsheets/SEER_brochure.pdf. of the Panel on Cost-Effectiveness in Health and Accessed August 15, 2012. Medicine. New York: Oxford University Press; 58. Bland JM, Altman DG. Survival probabilities (the 1996. Kaplan-Meier method). BMJ. 1998 Dec 48. Raftery J, Roderick P, Stevens A. Potential use of 5;317(7172):1572. PMID: 9836663. routine databases in health technology assessment. PMCID: 1114388. Health Technol Assess. 2005 May;9(20):1-92, 59. Kleinbaum DG, Klein M. : a self- iii-iv. PMID: 15899148. learning text. 2nd ed. New York: Springer; 2005. 49. Salas M, Hofman A, Stricker BH. Confounding by 60. Twisk JWR. Applied longitudinal data analysis for indication: an example of variation in the use of epidemiology – a practical guide. Cambridge, UK: epidemiologic terminology. Am J Epidemiol. 1999 Cambridge University Press; 2003. Jun 1;149(11):981-3. PMID: 10355372. 61. Newman SC. Biostatistical methods in 50. Petri H, Urquhart J. Channeling bias in the epidemiology. New York: John Wiley & Sons, interpretation of drug effects. Stat Med. 1991 LTD; 2001. Apr;10(4):577-81. PMID: 2057656. 51. Ray WA. Evaluating medication effects outside of clinical trials: new-user designs. Am J Epidemiol. 2003 Nov 1;158(9):915-20. PMID: 14585769.

313

Chapter 14. Modifying and Stopping Registries

1. Introduction substantive impact on the ongoing conduct of the registry. Less ambitious transitions (e.g., changes Most, if not all registries, should undergo periodic in data elements on preexisting case report forms) critical evaluation by key stakeholders to ensure are not specifically covered here; however, parts of that the objectives are being met. When registry this section (e.g., data analysis) are pertinent to objectives are no longer being met or when clinical such transitions. or other changes affect the registry (e.g., changes While the considerations for a major registry in treatment practices, introduction of a new transition are similar to those for the launch of a therapy), the registry may need to be adapted or to new registry, there are several distinguishing stop collecting new data. This chapter is divided features. First, a registry transition is facilitated by into two sections. The first section describes an existing registry and the collective experience possible reasons for a registry transition and issues of conducting that registry. The existing registry that should be considered in planning and can essentially serve as the starting point for implementing a transition. The second section creating a prototype of the revision. The planning discusses factors that may lead to the and design of the registry transition should also determination to stop a patient registry. Case benefit from lessons learned in operating the Examples 28, 29, 30, and 31 describe a variety of existing version of the registry. What has worked registry transitions. well, and what has been problematic? What challenges have been encountered at every level, 2. Registry Transitions from staff entering data at the participating sites to the analyst creating reports? Indeed, one or more A wide variety of factors may drive the decision to of these issues may be contributing factors in the proceed with a registry transition. For example, a decision to proceed with the registry transition. registry may need to transition to a new technology Even if this is not the case, the transition provides platform to remain functional for its participants, an opportunity to address these issues. Registry or a registry that was designed to study the natural transitions also present unique challenges distinct history of a disease for which there was no from the development of a new registry. In effective treatment may change its purpose when a particular, transferring data collected in an existing new product or therapy becomes available in the registry to the revised registry (i.e., data migration) market. Other scenarios in which a transition may can be a complex and resource-intensive process. be necessary include changes in funding sources (See Case Example 29.) and stakeholders (e.g., funding for a government- sponsored registry may end, resulting in transition Despite these differences, the steps in the to private ownership, such as to a professional execution of a major registry transition are association) or the introduction of new regulatory analogous to those involved in the launch of a new requirements (e.g., a registry may need to be registry. Therefore, the section is organized in adapted to fulfill a postmarketing commitment). accordance with the general framework for Because many different factors may contribute to a developing a new registry, with a planning and registry’s transition, transitions are highly variable design phase, an implementation phase to carry in scope and resource requirements. out the project plan, and an assessment of the potential impact on data management and analysis. This section focuses on issues that are of particular significance in a major registry transition, defined 2.1 Planning and Design as a change in the purpose, sponsor, and/or technology platform, all of which will have a The planning and design of a registry transition begins with an assessment phase, in which the

315 Section III. Operating Registries

need for a transition is considered. Articulating the • Sponsor/funding organization representative: purpose(s), determining if a major registry Ensures that the team has the resources transition is an appropriate means of achieving the necessary to complete the project and keeps the purpose(s), and assessing the feasibility of a sponsor apprised of any issues that may affect registry transition are important considerations, as the timeline or budget for the transition. such projects require a significant commitment of • Project manager: Accountable for all aspects of resources and have associated risks. Chapter 2, the transition, including timely escalation of Section 2.2 describes the assessment phase for a issues for resolution. new registry, much of which is directly relevant to • Clinical expert: Provides guidance on changes the consideration of a major registry transition. If that affect the clinical content of the registry the assessment leads to a decision to move (e.g., changes in purpose and data collection), forward, then the planning and design of the and provides input on data migration, as transition can proceed with the creation of a needed. project charter, formation of a transition team, and development of a comprehensive project plan that • Epidemiologist/biostatistician: Provides encompasses governance, ethical and legal issues, guidance on changes that affect the study and technology considerations. design and analysis plans (e.g., changes in purpose, data collection, data management, and 2.1.1 Creating a Project Charter data migration). The creation of a project charter is often a useful • Data management expert: Provides guidance starting point in planning a transition. A project on changes that affect data collection, data charter typically includes the following storage, or data quality assurance. information: • Legal/ethical expert: Provides guidance on how • Overview of the transition changes affect the legal and ethical construct of • Purpose/justification for the transition the registry (e.g., contract with funding source(s), contracts with participating sites, • Goals and objectives of the transition contracts with vendors, and data sharing • Business case for the transition (if applicable) agreements), and identifies any ethical issues • Identification of major stakeholders (e.g., need for institutional review board review, • Assumptions and constraints (organizational, changes to informed consent documents, need environmental, and external) to re-consent participants). • Potential risks • Other representatives: Depending on the nature of the transition, other representatives may be • Milestones/deliverables or high-level timeline included on the transition team, such as (1) a • Budget principal investigator or study coordinator from • Transition team members a participating site with experience entering data into the registry to provide guidance on • References to source documents, if applicable feasibility and burden of data entry, (2) a (e.g., new clinical practice guidelines) technical expert to help guide a transition to a 2.1.2 Forming a Transition Team new technology platform, (3) a patient advocate The next step is to assemble the transition team, to provide the patient’s perspective, and/or which will be responsible for planning and (4) representatives of other databases that are implementing the registry transition. It is linked to the registry (e.g., a related registry or important to include key stakeholders and to think substudy). broadly about the talent and expertise needed to Once the transition team has been assembled, it is accomplish a successful transition. In general, the critical to achieve consensus on the rationale and transition team should include the following the overarching goal(s) for the registry transition. members: Open discussion at this stage may identify

316 Chapter 14. Modifying and Stopping Registries

unanticipated barriers, which can be addressed relevant to a registry transition. Some additional proactively in the transition planning. Gaining the considerations are addressed below. full support of the transition team will increase the • Scientific advisory board governance during likelihood of a successful registry transition. the transition: Many registries have scientific 2.1.3 Developing a Project Plan advisory boards that oversee the conduct of the registry. These boards may also play a role in The next step for the transition team is to develop governance during a registry transition and a detailed project plan encompassing timeline and provide external perspective for the budget. The transition project plan should be considerations and future objectives for a thoughtful, complete, and realistic. As with all registry transition. Membership of the scientific projects of this magnitude and complexity, advisory board should be reviewed to ensure disagreement among stakeholders over scope, cost the key stakeholders that are involved in the overruns, and time delays may occur. These transition are represented. During the registry predictable issues should be anticipated as much as transition, the scientific advisory board can also possible, and risk mitigation strategies considered. act as an advocate of change by publicly The project plan should also consider other supporting the transition and helping to engage sources of risk specific to the transition (e.g., and motivate clinicians at the participating unexpected issues with technology compatibility, centers. External stakeholders, such as patient delays in obtaining institutional review board advocacy groups and regulatory agencies/ approval, and disputes related to ownership health authorities, may also be informed of the issues). Chapter 2 provides more information on transition and, depending on the goals of the project planning considerations. transition, potentially enlisted as additional The project plan should also address staffing public advocates for the registry transition. issues. The transition may require new expertise • Governance of data access: Registry transitions and skills that alter staffing requirements. Training will also require revisiting the registry’s data existing employees or hiring appropriately skilled access policies and procedures. If a data access personnel may be necessary. Planning for committee is already in place, the committee additional workload on the registry staff during the should be charged with (1) determining how actual transition is also an important consideration, changes in the registry will affect the policies as they may be operating and supporting the and procedures for accessing data, and existing registry while working on the transition to (2) reviewing the operational plan for executing the modified registry. analysis plans with respect to the registry Other issues that should be considered in transition transition. Furthermore, if the transition planning relate to governance, ethical concerns, involves a change in registry stakeholders, the legal matters, data collection, and technology. procedures for conducting analyses and These issues are discussed in more detail in the developing publications should be re-examined. following sections. New stakeholders may need to be involved in the prioritization of analysis plans, conduct of 2.1.4 Governance Issues analyses, and/or the review of scientific Nearly all registry transitions will require an abstracts and manuscripts. internal and external governance structure to 2.1.5 Ethical and Legal Issues manage and approve changes, whether the transition relates to the scientific objectives of the The major ethical and legal issues for registries registry, technology changes, or data access. The focus on data privacy, patient confidentiality, and transition team is one important component of the ownership of and access to the data. These issues, governance structure. Chapter 2, Section 2.6 covered comprehensively in Chapter 7, should also reviews the governance considerations for the be carefully considered during a registry transition. planning of a new registry, many of which are It is important to note that interpretations of the

317 Section III. Operating Registries

pertinent laws and regulations are numerous and (2) direct contact with patients to obtain new data, varied, leading to inconsistent application among (3) collection of biological samples or linkage of institutions, which may affect multicenter existing specimens to registry data, (4) the use of registries.1 Hence, input from legal counsel and data from deceased participants, or (5) linkage of regulatory authorities should be sought when the participant’s data to other databases. If the planning a registry transition. Some common legal planned registry modifications involve patients for or ethical concerns that may arise during registry whom the feasibility of obtaining consent would transitions are reviewed here. require unreasonable burden or situations where the consenting process would potentially introduce An early step in the registry transition planning 2-4 process is consideration of the need for an unacceptable level of bias, discussions with institutional review board (IRB)/ethics committee local IRBs/ECs should be undertaken to see if the (EC) review. If the purpose of the registry is consent can be waived. Chapter 8 discusses these unchanged and no new data are being collected, issues in more detail. IRB/EC review may not be necessary—subject to 2.1.6 Data Collection ethical guidelines and the requirements of the A major component of the registry transition individual IRBs/ECs. However, IRB/EC review project plan should be a thorough evaluation of would likely be required in certain transitions; for current and future data collection needs. The example, if new data will be collected through project plan should allocate time for contact with patients, if the new data that will be epidemiologists and clinical experts to jointly collected includes identifiable personal review the current registry case report form (CRF). information, or if the data will be used in a It is of paramount importance that the relevance of different manner than previously communicated to the current set of data elements is reviewed, in patients (45 CFR §46.102(f)). light of what is known about new hypotheses to be Especially when the purpose of the registry has tested. During this review, some data elements may changed, a registry transition may involve deemed irrelevant and may not be required moving extending the followup period of the initial cohort. forward. When considering the collection of In these circumstances, recontacting patients or additional covariates and outcomes, particular using their identifiers may be necessary to collect attention must be given to balancing the scientific the longer-term data. For example, a cardiac assist relevance of the new data elements with the device registry may have been established initially logistical burden on participating centers. to determine perioperative safety. However, new Additional considerations may arise if a registry safety concerns associated with longer term transition involves one of the following specific implantation may prompt a change in the purpose circumstances: of the registry. Medical records, death indices, and patient interviews may be required in order to • Collection of Biological Samples: Biobanks, collect the longer-term followup data. This new defined as facilities that store biological data collection effort would likely require IRB/EC material (e.g., serum, genomic material, review. pathology specimens) from humans, are increasingly popular additions to registries.5 Consideration should also be given to whether any The addition of a biobank raises many changes will be required in the informed consent logistical issues, which are outside the scope of process (e.g., obtaining revised consents from this chapter. However, it should be noted that existing subjects, obtaining new consents for the addition of a biobank will likely require registries that do not currently have such consents). changes in the informed consent. Some If consent was obtained for registry participation biobanks have used general consents to cover initially, re-consenting may be needed, especially future analyses of the biological material and when the registry transition will result in integration into the registry, but there is (1) longer or otherwise different followup than significant concern about these broad consent what was originally agreed to by patients,

318 Chapter 14. Modifying and Stopping Registries

documents. Some commentaries on this issue transition, particularly one involving a change in have suggested that such broad consents are stakeholders, a careful review of agreements or more appropriate when limited to a specific contracts should be performed to determine if disease entity, thereby allowing for studies modifications are needed. In some cases, the examining diagnosis, mechanisms of disease, registry transition may involve moving data from risk factors, and treatment outcomes.6-8 one platform to another. Hence, data ownership Chapter 8 discusses these issues in more detail. may need to be clarified. For example, a • Pediatric Registries: If a registry enrolls professional organization may determine that the pediatric participants and the registry transition vendor maintaining its registry is performing involves extending the followup period, below expectations and may select a new vendor to consideration should be given to whether house and run the registry. Depending on the terms participants need to give consent when they of the prior agreements, it may or may not be reach an eligible age. This is particularly possible to import the historical data into the new important for those registries that plan to add a vendor’s system. biobank or link to other databases as part of the Registry data are often collected using electronic transition process. There is considerable debate or paper CRFs that may have intellectual property regarding the ethics of parents enrolling their protections, including copyright, trademark, and children in research studies. More discussion patent. If continued use of these forms is planned, on this topic can be found in Chapter 7. It is measures should be taken to ensure that the also important to note that for all registries, the appropriate permissions for use are still applicable right to withdraw is inherent;8, 9 see Chapter 8. when the registry transitions. • National to International Registry: Some 2.1.8 Data Access registry transitions may extend the geographic scope of a registry. For example, a U.S.-based In addition to data ownership, ongoing data access registry may add participating sites in Europe. is an important consideration. The new and When the registry scope extends beyond ongoing registry stakeholders should consider national borders, additional ethical and legal whether the previous stakeholders should have concerns must be addressed. Each country may access to the previously collected data as well as to have different legislation and restrictions for the data collected in the future. Federal and the collection and processing of subject academic stakeholders may need to execute information and its use for research. Adequate technology transfer agreements (e.g., material time and additional resources to investigate transfer agreements) or other contractual these requirements should be factored into the agreements in order to access the data. project plan. Moreover, if Federal funds are 2.1.9 Changes in Funding used in the registry transition, additional steps may be involved in the expansion of the Registry transitions may also include changes in registry. In particular, some registries may be funding. For example, a registry that was initially collecting data on vulnerable international funded through a government grant may be populations for which additional privacy transitioned to a professional association or protection safeguards may be necessary. industry partner. When funding sources change, Federal guidelines for performing international the role of the funding entities should be clearly research should be consulted as part of the delineated to ensure that there is no real or planning process. perceived threat to privacy or data confidentiality. 2.1.7 Data Ownership and Licensing In some cases, a change in funding may require contract modifications in anticipation of potential A number of scenarios exist in which ownership of conflicts between the new stakeholders and the registry materials must be delineated, including the remaining stakeholders. For example, industry ownership of the interface, platform, may elect to partially fund a registry that is also infrastructure, and data. During a registry

319 Section III. Operating Registries

receiving Federal funding from a regulatory obligations. Establishing the escrow account would agency. Contracts may need to be modified to increase the cost of the initiative for the sponsor, clearly delineate how each set of funds will be the vendor, or both, and should be considered spent. The new Chapter 24 on public-private when planning the transition. In addition, contracts partnerships provides more information on these should contain explicit clauses that guarantee the issues. As with all contracts involving Federal transmission of data to a new vendor when the funds, attention should be given to regulations contract expires or if the vendor defaults on the governing their appropriate use. Additionally, contract. changes in funding may alter the locus of legal 2.1.11 Technology Considerations rights and obligations. It is important to have unambiguous conversations with stakeholders and A registry designed to collect long-term followup associated contractual agreements that clearly data will inevitably undergo technology changes. delineate the rights of the funding entities. Platforms for electronic data capture (EDC) may be upgraded (for example, through version updates When data are transferred from one owner/sponsor within a system), or the registry sponsor may to another, the liability associated with the select a different third-party vendor to host the protection of subjects’ information should be EDC system. Upgrading the EDC system and clarified. Consideration should be given to technology platform may enable more frequent indemnification clauses in data transfer data entry from participating centers, rather than agreements. Oftentimes, the data transfer annual or semi-annual data reporting under agreements detail that the new sponsor of the previous technology environments. Such changes registry will accept all liability for use of the data have implications for training plans for previously collected by the transferring sponsor. participating centers (see Section 2.2.2, Training). The data transfer agreement should also contain a Technology considerations relevant to linkage of a clause that the new sponsor agrees to use the data registry to an electronic health (medical) records properly. In these circumstances, the liability (EHR/EMR) database or other database and for would be assumed by the new sponsor if a breach collection of patient-reported outcomes are of information occurred whereby subject-level covered in Chapters 15 through 18. In transitioning information was relayed to an outside party. If the to a new registry technology platform, it is new sponsor is a Federal entity, however, there are important to clearly define software requirements regulations that prohibit the Federal Government in order to avoid design flaws, which are costly to from indemnifying others (e.g., Anti-Deficiency correct after project completion. Soliciting input Act). from various stakeholders (e.g., data entry 2.1.10 Contracts With Vendors personnel, clinical experts, data analysts) may be Issues may arise with vendors (including helpful to validate the proposed design of the new inadequate performance of duties, loss of financial registry. The proposed design should be presented solvency, or escalating cost of renewing the to them in an easy-to-understand format (e.g., a contract), necessitating a transition to a new prototype) rather than a detailed requirements vendor. In light of these potential outcomes, it is document, which may be more difficult to necessary to draft contracts that consider these comprehend. Setting aside time for user- scenarios and contain provisions to address them. acceptance testing or pilot testing may also be For example, if a registry is being transitioned to a useful to identify issues before the transition is new, fledgling company, consideration should be complete. given to establishing an escrow account for the One of the earliest and most important decisions in registry. This account would cover the cost of transitioning to a new technology platform is ensuring that the data remain accessible to the whether to develop the platform in-house or use an sponsoring body. Moreover, it would prevent the external vendor. Each approach has advantages registry from being part of the estate if the and disadvantages. The in-house approach requires company is unable to meet its contractual personnel with the appropriate expertise and the

320 Chapter 14. Modifying and Stopping Registries

infrastructure to support such a project. turn to external vendors. Selecting a registry Development tools widely used by software vendor is an important strategic decision for an companies should be employed, if possible, to organization, particularly for sponsors who mitigate the risk of experiencing shortages of anticipate operating the registry for many years. qualified personnel for ongoing support and Some factors that should be considered in selecting maintenance of the application. Organizations that a registry vendor are outlined in Table 14–1. do not have the internal resources and expertise to develop a registry application in-house usually

Table 14–1. Considerations in selecting a registry vendor

• Develop detailed requirements for the new registry before issuing a request for proposals. The requirements may be modified later to align with the vendor’s framework for development, but having complete requirements early in the process will allow for a more accurate timeline and cost estimate. • Gather as much information as possible about the potential vendor by contacting existing clients and asking detailed questions about communication, timelines, budget, and post-release support. • Ask an independent expert to evaluate and analyze the technology platforms and technology expertise of the potential vendor. • Ask the potential vendor to be specific with their cost estimates. Avoid vendors that cannot provide concrete estimates. • Know the hosting and maintenance fees of the existing registry and compare them to the hosting and maintenance estimates from the potential vendor. • Assess the security policies and procedures established by the vendor and ensure that they comply with the industry standards and best-of-breed practices. • Assess the ability and willingness of the potential vendor to transfer registry data (both to transfer historical data into their registry application and transfer the data out from their registry application if the registry changes vendors in the future). • Learn about the vendor’s experience in importing data from other sources of medical information using standard interfaces (e.g., Health Level Seven [HL7], Clinical Data Interchange Standards Consortium [CDISC]), and also about their ability to build custom interfaces. A list of existing and emerging standards in the field is maintained by U.S. Food and Drug Administration. • Consider the vendor’s international experience, including translation and help-desk support, if pertinent to the planned transition. • Discuss policies related to data access, including how participating sites can access their own data and how the registry team can obtain data sets for analysis.

Once a vendor has been selected and the features Another technology consideration is transitioning of their technology platform are known, it is personnel involved in data entry at participating important to assess the hardware, software, and sites from an existing registry to the new registry. browser configurations at the participating sites, as This requires an analysis of security levels in order these may affect performance of the registry to transfer users to the appropriate permission application. It is also important to ensure that the level in the revised registry. In some cases, users participating sites have access to the optimal can be transferred electronically from the existing configurations on which the application has been to the new registry application, but in other cases, tested and validated. Requesting a technology they must be added manually. The transition team contact person at each of the participating sites must develop a plan for accomplishing the transfer may be helpful to facilitate working through these that minimizes the effort at the participating sites, issues during the transition. but ensures that only valid users can access the

321 Section III. Operating Registries

registry at the appropriate permission level. Of 2.2.2 Training note, a registry transition provides an opportunity The development and implementation of a robust to assess the activity level of users at the training program prior to the registry transition participating sites and their ongoing need to access will facilitate the rollout of the revised registry and the registry. improve the quality of data collected. Training A final technology consideration pertinent to a needs will vary according to the scope of the transition relates to the closeout of an existing registry transition. For example, a technological registry. Generally, the closeout should be change that affects the user interface, functionality, scheduled well after the anticipated launch of the and/or organization of the data elements will likely new registry, as timelines on such complex require extensive training, whereas a transition projects are often delayed. The existing registry related to a change in purpose with minimal may also be useful in validating successful data impact on data entry should require less training. migration into the new registry. When developing a training program, the key elements of adult learning theory10 should be kept 2.2 Implementation in mind, and several questions should be Once a transition plan has been developed and the addressed: decision has been made to move ahead, it is • Who is the intended audience? Determining the important to communicate with stakeholders about audience will have a significant impact on the the plans, train registry participants on the changes design and implementation of the training and support them through the launch, and assess program. For example, internal staff training the impact of the transition on data management will differ from that of external registry and analysis activities. participants, and the training program for 2.2.1 Communication clinicians will likely differ from that designed for data entry personnel. Communicating with all stakeholders is critical • What are the learning objectives? The learning during a registry transition. The transition team objectives should drive the development of the should develop a communication plan that defines curriculum. What do the people involved in the who is responsible for communicating what and to registry need to know to be successful during whom. The frequency and mode of communication and after the transition? The focus should be on should be established with a particular sensitivity what will change and why, and what the impact to key stakeholders. Since the registry transition of the changes on registry participants will be. will likely disrupt workflow at the participating sites, communicating the rationale for the change, • What information is needed to meet the the timeline, and the impact on users is important. learning objectives? High-level overviews and Any change in expectations or incentives for detailed documents are useful to help participation should be fully explained. It is participants with varying levels of interaction important to anticipate and respond to questions with the registry understand the changes. The and concerns from participating sites, knowing creation of a reference guide that clearly that change can lead to stress and anxiety. In most describes what changes were made and why circumstances, the communication plan will focus each change was made will be extremely on retaining participating sites through the helpful to some registry participants. transition. However, a registry transition provides • What are the best mechanisms for an opportunity to evaluate participating centers to disseminating the information? People respond decide whether all of them should be retained. A differently to various learning environments transition may also be an ideal time to recruit and techniques. Depending on the size of the additional sites. registry, training may be offered in various ways, some of which are described below:

322 Chapter 14. Modifying and Stopping Registries

–– Conference calls can be effective for smaller they may uncover problems with the revised groups and allow for open discussion. registry that have been missed during testing. Such –– Webinars can be useful when larger groups problems may require immediate attention not only are involved and the training activity from support personnel, but also from the includes visual presentation. developers of the registry application. At some defined point in time (e.g., 1 to 3 months after –– Face-to-face meetings are frequently launch), a broader analysis of all of the questions effective since the learner is less likely to be and comments from participants may be helpful in distracted. prioritizing any further changes to the registry. –– One-on-one training sessions are usually well received, since the training can be 2.2.4 Data Management customized to the individual learner. Technological changes may require changes to the However, this approach is costly. database/data warehouse used to store the registry –– User’s guides, manuals, FAQs, and other data. Database or data warehouse transfers are documents can be posted on a Web site, or complex processes that involve a number of steps, hard-copy materials can be distributed to including creating a new database layout, mapping participants. the legacy data to the new database layout, and • What is the best approach to ensure that transferring the data with rigorous quality controls learning has occurred? It is important to to ensure that the transfer is successful. Database confirm that the training program is successful, transfers also need to be conducted in accordance in order to avoid issues with retention and data with any regional IRB or EC approvals to ensure quality after the transition launch. Learning that the privacy of any patient-level data is assessment approaches include tests (e.g., the maintained. The size and complexity of the completion of a sample data collection form or registry as well as the extent of the changes in the other task), surveys, and direct feedback. CRFs will determine the complexity of the data Feedback from the learning assessments should mapping process. The data fields known to users of be incorporated into the training program, as the registry might be collected in different contexts needed. Pilot testing may also be useful for (e.g., with added specificity or new dependencies refining and strengthening the training program between data elements on CRFs), and these before launch. differences must be considered in the data mapping process. Relatively small changes in the 2.2.3 Supporting Participants Through the wording of a question on the CRF, or creating an Registry Launch additional category on an existing item In addition to a robust training program, sufficient (e.g., expanding categories of ethnicities) may personnel and resources should be assigned to introduce ambiguities in the mapping of the respond to input and inquiries from registry existing data set to the new environment. In other participants following the launch of the revised instances, significant changes to the definition of registry. Accessibility of the support team is very an outcome variable will typically require review important during this critical period of the and adjudication of prior cases to establish transition. Planning for the registry launch should longitudinal consistency across the data set (for delineate how users can submit questions or further detail, see Section 2.3.3. below). For these concerns (e.g., by email or by calling a support reasons, input and evaluation of the impact of the desk), who will be the first responders, and how migration on future registry outcome analyses complex issues will be escalated for further from subject matter experts, including evaluation. Many straightforward questions epidemiologists and clinical experts, along with (e.g., problems logging on) can be resolved documentation of decision rules that were quickly and efficiently. However, it is important to established during the epidemiological and clinical carefully assess all input from participants since review, will be needed in the data mapping process. The effort and expense involved in the

323 Section III. Operating Registries

data migration is often underestimated and document the rationale for adding, modifying, or adequate time must be allocated during the project deleting data fields, so that this information can be planning and in establishing timelines. Despite communicated to stakeholders and registry careful attention to detail, this activity often participants. Second, careful consideration should becomes an iterative process, with data mapping, be given to the future impact of changes. Certain data importation, and quality control checks that changes may make it difficult to link prior data lead to corrections in the data mapping, re- sets with the new data sets. For example, adopting importation of the data, et cetera. a new, broader definition may mean that data can Many practical issues should be considered when only be linked in one direction, as shown in Table transferring a database. First, it is important to 14–2.

Table 14–2. Impact of definition changes on data linkage

Old Definition New Definition Linkage Direction

Death: A mortality that occurred in Death: a mortality that occurred Deaths in the old data set fall within the hospital within 30 days of the within 30 days of the procedure, the parameters of the new definition. procedure. whether in the hospital or not. However, deaths according to the new definition would not necessarily apply to the old definition, since they include mortalities post- hospitalization.

When making changes to the data structure, the changes can present enormous challenges to the following questions should be considered: continuity and validity of the analyses. The issues • Will existing queries (i.e., questions raised by a range from the handling of new data elements to data manager and issued back to the the introduction of selection bias or recall bias if participating centers regarding a data entry the cohort definition evolves during the transition. issue) need to be rewritten for the new data set? 2.3.1 Changes in Cohort Definition • Will existing reports (e.g., percent of patients A registry transition may involve a change in the with a laboratory value above a certain number) inclusion or exclusion criteria for patient need to be revised for the new dataset? participation, thus shifting the definition of the • Will more server space be needed to house the study cohort. These changes can occur under a data? number of scenarios, such as if the registry moves • How can the impact of the changes on the from a disease-based cohort (i.e., no inclusion processes affected by the new data structures be criteria for receiving a particular treatment) to minimized? focusing on a cohort of patients with the disease who receive a specific therapy or class of therapies It is also important to determine what metadata (i.e., inclusion criteria now requires patients to be (e.g., long name, short name, data type/data receiving a treatment). Cohort definitions may also format, and permissible values) are important to change based on geography (e.g., if a registry capture for each field and how the transition will transitions from a national to a global catchment affect the metadata. area). This introduces the possibility of geographic differences in disease severity or treatment 2.3 Data Analysis patterns, which may require thorough A registry transition may introduce many data documentation of baseline clinical status in order analysis considerations that require the input of to stratify or perform covariate adjustment, if epidemiologists and/or biostatisticians. Transitions necessary. that involve new hypotheses or technological

324 Chapter 14. Modifying and Stopping Registries

Other changes in the cohort definition may occur if interested in collecting data on heart failure as an the registry transitions from having broad outcome may find variation in how the definition participation by centers to a limited set of centers of heart failure is applied across contributing (e.g., physicians who are associated with large centers. While the refinement of definitions for specialty care clinics). A registry transition that data elements occurs, analyses on the outcome results in such a change in the cohort definition variables may still take place. However, methods has the potential to introduce selection bias into of quantitative sensitivity analysis may be the registry by focusing the enrollment and necessary to understand the degree to which ongoing followup of subjects on a potentially more misclassification of variables may introduce bias severely affected group of patients. As enrollment into the analytic results. Results of source and followup occur, epidemiologists should be document verification efforts can be used as inputs actively involved to assess whether selection bias into quantitative sensitivity analysis to directly has been introduced. Comparisons of demographic estimate the sensitivity and specificity of the and baseline clinical features of subjects before outcome variable. and after the transition may be sufficient to assess 2.3.3 Impact on Existing Cases of an Outcome the degree of bias introduced and to understand which factors or variables can be considered for A registry transition may lead to redefining an stratification or covariate adjustment. Advanced outcome in order to increase sensitivity and methodologies such as comorbidity indices or specificity. For example, a registry that has been propensity score analyses may be necessary to collecting data about the onset of Parkinson’s adequately adjust for the changes in the cohort disease as an outcome measure may transition to over time. more stringent inclusion and exclusion criteria. Although this may result in increased validity of 2.3.2 Introducing New Data Elements the outcome, the statistical power of the analyses As scientific advances improve the understanding from the registry may be compromised, as there of a particular disease or new treatments become will likely be fewer patients meeting the case available, new hypotheses will likely be formed. In definition going forward. Patients who have order to test new hypotheses, it may be necessary already been identified in the registry as cases may to add data elements and/or refine the definition of require re-evaluation (and possibly re- existing data elements. When adding new data adjudication) to determine if their clinical scenario elements, common data elements and validated fulfills the revised selection criteria. instruments should be used when possible. (See Figure 14–1 shows the potential impact of a Chapter 4.) it may be necessary to validate the new change in an outcome (e.g., case definition of data elements, through source document Parkinson’s disease) following a registry transition. verification of the original medical records, Note that the smaller cohort size following the laboratory tests, or diagnostic reports. Results of registry transition may reduce statistical power and source document verification may show there are cases that met original case definition may require discrepancies in the accuracy of new data elements re-evaluation. being captured. For example, investigators

325 Section III. Operating Registries

Figure 14–1. Potential impact of a change in outcome

Cases requiring All patients in the registry re-evaluation with Parkinson’s disease

Patients in the registry intially identified with Parkinson’s disease on case report form

Patients who fulfill revised definition of Parkinson’s disease following registry transition

2.3.4 Impact of Patient-Reported Outcomes Table 14–3 illustrates the possible consequences of Registries frequently include patient-reported transitioning from a disease-based registry to a outcomes, such as those reported on the SF-36® focus on patients with the disease exposed to a health survey, or activities of daily living. It is particular therapy. Prior to the transition, the risk important to note and characterize whether the of the outcome among exposed and unexposed type of patients who are reporting self-assessments patients was similar. Following the transition, there to the registry and their disease severity or are more exposed patients, and, for the purposes of outcome status are changing over time because of illustrating the impact of bias, assume awareness the transition. If such instruments are introduced of the registry transition results in exposed patients during a registry transition, patients may begin to preferentially reporting onset of a particular preferentially recall events, which can lead to a outcome. Because of this preferential report, the bias in the outcomes. In addition, if the registry risk is approximately 25 percent greater among the transitions from a purely disease-based registry to exposed as compared with before the transition. a therapy- or product-based registry, patients who The apparent risk ratio is now 1.46 comparing become aware of this change may begin to report exposed to unexposed. their health status more or less favorably.

326 Chapter 14. Modifying and Stopping Registries

Table 14–3. Possible consequences of a change in registry focus

Exposed Unexposed Before Transition Cases with specific patient-reported outcome 70 50 Total patients 450 375 Cumulative incidence per 100 15.6 13.3 Risk ratio 1.17 Following the Transition* Cases with specific patient-reported outcome 175 50 Total patients 900 375 Cumulative incidence per 100 19.4 13.3 Apparent risk ratio 1.46 * The emphasis on enrolling patients who have been exposed to the therapy leads to an apparent 25 percent increase in the incidence of cases among the exposed.

2.3.5 Comparative Effectiveness Analysis hypotheses of interest may no longer be testable A registry may transition from a disease-based from the standpoint of statistical power. cohort to one that is focused on specific Alternatively, consideration of statistical power for treatment(s) in order to establish comparative newly specified hypotheses following the transition effectiveness studies between multiple treatments. may provide an assessment of the extent of A greater emphasis on baseline covariate data may enrollment and followup required for robust future be required in this situation, and epidemiologists analyses. should be involved to identify the key variables 2.4 Summary of Registry Transition that would account for differences in disease Considerations severity among the treatment groups in order to mitigate biases such as confounding by indication. Many registries will undergo a major transition at Epidemiologists must also be involved in planning some point in their life cycle, most often related to the statistical analysis, which may require a change in purpose, sponsor, and/or technology matching or other multivariate statistical platform. A major registry transition is a complex techniques. and resource-intensive process with associated risks. Careful and comprehensive planning will 2.3.6 Biostatistics and Statistical Power maximize the probability of success. However, Statistical power must be considered in registry unexpected challenges may still occur during the transitions that lead to changes in the size of the implementation phase. The transition team should cohort and/or the extent of followup. For example, be prepared to react to circumstances as they arise a transition that focuses the registry on a smaller and modify the project plan accordingly. This number of participating centers may diminish the chapter has reviewed the steps involved in the number of new enrollees, but provide the benefit of execution of a registry transition, including the an extended length of followup. The transition may planning and design, implementation, subsequent eventually provide a greater number of exposed impact on data management and analysis issues. patients who develop the outcome(s) of interest. Table 14–4 presents a checklist of key issues that Biostatisticians should be involved in assessing the may be helpful to readers who are considering a impact of changes in cohort accrual on statistical major registry transition. precision of the analyses. Previously specified

327 Section III. Operating Registries

Table 14–4. Checklist of key considerations for a registry transition

Planning and Design Phase 1. Determine if a registry transition is appropriate and feasible. a. Has the purpose of the transition been clearly articulated? b. Is a transition an appropriate means of achieving the purpose? c. Is the transition feasible from a resource perspective? 2. Organize a transition team. a. Has a transition team been assembled with all necessary areas of expertise? b. Is the team in agreement on the rationale for and goals of the transition? c. If applicable, how will partner organizations be involved in the transition planning and design? 3. Develop a transition project plan. a. Does the project plan cover timeline, budget, and staffing? b. Have “lessons learned” from operating the current registry been considered and addressed in the transition plan, as necessary? c. Have major risks been identified and risk mitigation strategies considered? 4. Engage advisory boards and other stakeholders. a. Is the scientific advisory board in agreement with the rationale for and goals of the transition? b. Are any changes needed to the scientific advisory board to ensure that appropriate areas of expertise for the transition are represented? c. Will changes to the data access policies and procedures be necessary? 5. Consider legal and ethical issues. a. Will the changes require review/approval by an IRB or EC? b. Do the changes require informed consent, or does the existing informed consent form need to be updated? c. Is the registry expanding to collect data in new countries? If so, what additional ethical and legal considerations must be addressed? d. Are any changes needed to existing contracts or agreements? 6. Assess the potential impact of technology changes. a. Does the transition involve changing to a new technology? If so, have the hardware, software, and browser configurations been assessed at participating sites to ensure that the new technology will perform well? b. Is there a plan for transferring personnel (usernames/passwords) from the previous system to the new system? c. Will a new registry vendor be selected? If so, have potential vendors been thoroughly assessed (see Figure 14–1)? Implementation 1. Share information on the transition with registry participants and stakeholders. a. Is there a communication plan that clearly defines who should communicate what information to whom and at what time? b. Who will answer questions about the transition?

328 Chapter 14. Modifying and Stopping Registries

Table 14–4. Checklist of key considerations for a registry transition (continued)

2. Train registry participants and support them through the launch. a. Have training plans been developed for registry staff and participants? b. Is there sufficient registry staff to carry out training for participants? c. Have registry materials (e.g., user guides, data definitions) been updated? d. Has a plan been developed to support participants after launch of the revised registry? Data Management and Data Analysis 1. Develop a plan for data migration. a. Is data mapping or migration necessary? b. Are the timeline and budget sufficient for data migration, which is often an iterative, complex process? c. Is there a clear rationale for adding, modifying, or deleting each data field? d. Have the implications of changes to the data structure been carefully considered? 2. Determine how the transition may affect data analyses. a. Did the transition change the definition of the study cohort? If so, has the potential for selection bias or recall bias been assessed? b. Have new or modified data elements been reviewed to determine if participants are reporting this information correctly? c. Have outcome measures been redefined? Will existing cases of the outcome require re-adjudication? d. If comparative effectiveness research is planned, will additional baseline covariates be needed for the analyses? e. Will the transition affect the statistical precision of the analyses?

3. Planning for the End of a 3.1 When Should a Patient Registry Patient Registry End? 3.1.1 Stopping an Experiment Once a registry is in place, how long should it The principles regarding rules for stopping a study continue? What are reasonable decision criteria for mostly stem from the need to consider stopping an stopping data collection? This section considers experiment. Because experiments differ from the issues related to stopping a patient registry registries in crucial ways, it is important to study and suggests some guidelines. Although the distinguish between the issues involved in stopping specific answers to these questions will vary from an experimental study and in stopping a study to study, general types of considerations can nonexperimental study. In an experiment, the be identified. The discussion here focuses on patient’s treatment is determined by the study registries intended to assess specific safety or protocol, which typically involves random effectiveness outcomes rather than those intended assignment to a treatment regimen. In a to assess health care operations such as continuous nonexperimental study, patients are treated quality improvement. See also Case Example 28. according to the treatment protocol devised by their own clinicians, typically uninfluenced by the study. In a randomized trial of a new therapeutic

329 Section III. Operating Registries

agent or a field trial for a vaccine, the size of the alternative view, referred to as “clinical equipoise,” study population is ordinarily set in the study is that equipoise can be achieved at the group protocol, based on assumptions about the expected level, with the enthusiasm of some investigators or hypothesized results and the study size needed for the prospects of the study intervention being to reach a reasonable scientific conclusion. balanced by the skepticism of others.14 Whichever Ordinarily this planned study size is based on interpretation of equipoise is adopted, most power calculations, which require as input the investigators agree that if equipoise becomes criteria for statistical significance, the untenable as study results accumulate, the study anticipated, the baseline occurrence rate of the should be stopped to avoid depriving some study study outcome, and the relative size of the study participants of a potential benefit relative to what arms. Because of inherent problems in relying on other participants receive. 11, 12 statistical significance for inference, the study For an advisory board to decide to stop a study size preferably will be planned around estimation early, there must be solid evidence of a difference of effect and the desired level of precision. In a between the groups before the planned study study intended to provide some reassurance about endpoint is reached. Such stopping decisions are the safety of an agent, the study size may be usually based on ethical concerns, as scientific planned to provide a specific probability that the considerations would seldom dictate an early stop upper confidence bound of a conventional to a study that had been planned to reach a specific confidence interval measuring an adverse effect size. Advisory boards must base stopping would be less than some specified value, given a decisions on analyses of accumulating study data, postulated value for the effect itself (such as no which are usually formally presented at regular effect). In the latter situation, if no effect is meetings of the review board. Statistical concerns anticipated, a power calculation is not only have been raised about biases that can arise from unreasonable but is not even possible, whereas repeated analyses of accumulating data.15 To offset planning a study on the basis of precision of these concerns, many experiments are planned estimation is always possible and always with only a limited number of interim analyses, reasonable. and the interpretation of study results takes into Stopping an experiment earlier than planned is an account the number of interim analyses. important decision typically made by an advisory 3.1.2 Stopping a Fixed-Length Nonexperimental group, such as a data safety and monitoring board, Study which is constituted to monitor study results and make decisions about early stopping. In a Like experiments, most nonexperimental studies biomedical experiment, the investigator has a also have a fixed time for their conduct and a greater ethical obligation than in a planned size that reflects goals analogous to those nonexperimental study to safeguard the well-being in experimental studies. Nevertheless, the ethical of study participants. This is because the concerns that motivate stopping an experiment investigator is administering an intervention to before its planned completion do not have a direct study participants that is expected to affect the counterpart in nonexperimental studies. probability that study participants will experience Nonexperimental studies have ethical concerns, one or more specific health outcomes. but they relate to issues such as data privacy, intrusive questioning, or excessive inducements Equipoise is a widely accepted (but, unfortunately, for participation rather than to concerns about not universally accepted) ethical precept regarding intervention in the lives of the participants. human biomedical experimentation.13 Equipoise Although it is theoretically reasonable that an requires that at the outset of the study, the investigator could choose to stop a investigator has a neutral outlook regarding which nonexperimental study for ethical reasons, those of the study groups would fare better. A strict reasons would presumably relate to ethical interpretation of equipoise requires each of the problems that were discovered in the course of the study investigators to be in a state of equipoise. An study but were unrecognized at the outset rather

330 Chapter 14. Modifying and Stopping Registries

than to an early conclusion regarding the study ended registry that focuses on a set of specific goal. For example, the investigator in a endpoints (congenital malformations) among a nonexperimental study could learn from an interim subset of patients (pregnant women) taking a class analysis that the association between the exposure of medications (antiepileptic drugs).16 It has no and the outcome under study was much stronger fixed stopping point. than anticipated. Unlike the experimental setting, Measuring the frequency of rare endpoints however, the investigator in a nonexperimental demands large study sizes. Therefore, a monitoring study is not administering the exposure to any of system that includes rare endpoints may have to the study subjects and thus has no responsibility to run for a long while before the accumulated data the study subjects regarding their exposure. will be informative for low-frequency events. On The discovery of an ethical problem during the the other hand, the lower the frequency of an conduct of a nonexperimental study is therefore adverse event, even one with serious possible but extremely rare. Because the findings consequences, the smaller is the public health from an interim analysis should not lead to problem that a relative excess of such events would discontinuation of a nonexperimental study, there represent. is little motivation to conduct interim analyses for Traditional surveillance systems are intended to nonexperimental studies that have been planned continue indefinitely because they are intended to with a fixed size and period of execution. If there monitor changes in event frequency over time. For is some considerable time value to the findings, example, surveillance systems for epidemic such as to inform regulatory action, it might be infectious diseases provide early warning about worthwhile to conduct an interim analysis to get an outbreaks and help direct efforts to contain such early appraisal of study findings. Unless there is an outbreaks. In contrast, a patient registry is not a appropriate outlet for releasing interim findings, true surveillance system, since most are not however, it is possible that early findings will not intended to provide an early warning of a change circulate beyond the circle of investigators. In most in outcome frequency. Rather, most patient circumstances, such analyses are hard to justify in registries are intended to compile data on light of the fact that they are based on a smaller outcomes associated with novel treatments, to amount of data than was judged appropriate when supplement the sparse data usually available at the the study was planned; thus the originally planned time that these treatments are considered for analysis based on all the collected data will still approval by regulatory agencies. For example, a need to be conducted. Unless there is a clear public regulatory agency might mandate a patient registry health case to publicize interim results, journal as a condition of approval to supplement safety policies that require that data to be published have information that was submitted during the not been previously published may inhibit any application process. release of preliminary findings to news media or to journals. How long should such a registry continue? Although it is not possible to supply a general 3.1.3 Stopping an Open-Ended Study answer to this question, there is little reason to Although patient registries may be undertaken support a registry continuing indefinitely unless with a fixed length or size, or both, based on study there is a suspicion that the treatments or treatment goals relating to specific safety or efficacy effects will change over time. Otherwise, the time hypotheses, many such studies are begun as should come when the number of patients studied open-ended enterprises without a planned stopping suffices to answer the questions that motivated the point. For example, patient registries without registry. The Acyclovir Pregnancy Registry, which specific hypotheses may be undertaken to monitor began in 1984, was stopped in 1999. Its advisory the safety of patients receiving a novel therapy. committee concluded: “The [Acyclovir Pregnancy] The Antiepileptic Drug Pregnancy Registry, Registry findings to date do not show an increase established in 1997, is an example of an open- in the number of birth defects identified among the prospective reports [of exposures to acyclovir]

331 Section III. Operating Registries

when compared with those expected in the general certain frequency. For example, the goal could be population. In addition, there is no pattern of to establish a specified level of confidence that defects among prospective or retrospective unexplained hepatic necrosis in the 3 months acyclovir reports. These findings should provide following drug exposure occurs in less than 1 some assurance in counseling women following patient in 1,000. Alternatively, the goal might be to prenatal exposure [to acyclovir].”17 The consensus provide a more precise estimate of the frequency was that additional information would not add of a previously identified risk, such as anaphylaxis. materially to the information that already had been Ideally, this goal should be formulated in specific collected, and thus the registry was closed down. numeric terms. With specific goals, the registry To avoid uncertainty about the fate of an open- can have a planned target and will not be open ended study, it would be sensible to formulate a ended. specific goal that permits a satisfactory conclusion If a registry study does not have a single or very to data collection. Such a goal might be, for limited set of primary objectives, a stopping point example, the observation of a minimum number of will be more challenging to plan and to justify. specific adverse events of some type. Even better Even so, with measurable goals for some would be to plan to continue data collection until endpoints, it will be possible to determine whether the upper bound of a confidence interval for the the registry has achieved a core purpose, indicating rate or risk of the key outcome falls below some a reasonable stopping point. Conversely, a registry threshold or until the lower bound falls above a that fails to meet measurable goals and appears to threshold. Analogous stopping guidelines could be be unable to meet them in a reasonable time is also formulated for registry studies that are designed a candidate to be stopped. For example, if the with a built-in comparison group. registry faces unexpectedly low patient accrual, it should be stopped, as was done with the 3.2 Decisions on Stopping and Registry Observational Familial Adenomatous Polyposis Goals Registry Study in Patients Receiving Celecoxib.18 Ideally, stopping decisions ought to evaluate data This study enrolled only 72 patients in 4 years, out from a registry against its stated goals. Thus, the of a planned 200 during 5 years. Another reason to registry protocol or charter should include one or consider stopping is incomplete or poor-quality more specific and measurable endpoints against information. Poor-quality data are of particular which to judge whether the project should continue concern when the data regard sensitive or illegal or stop. Without that guidance, any decision to behavior, such as self-reported information on discontinue a registry may appear arbitrary and sexual practices.19 Decisions about stopping a will be more readily subject to political registry because of low enrollment or inadequate considerations. In cases where there are no information are made simpler with clearly stated measurable endpoints to use in making the goals regarding both features of the study. The decision, it is important that any final reports or criteria for useful quantity and quality of publications linked to the registry include a clear information should be specified at the outset. How discussion of the reasons for stopping it. well the study meets the criteria can be assessed periodically during data collection. Registry goals will vary according to the motivation for undertaking the project and the A registry may outlive the question it was created source of funding. Product-specific registries may to answer. For example, if use of the product is be created as postapproval regulatory superseded by another treatment, the questions that commitments. For products about which there are drove the creation of the registry may no longer be limited preapproval safety data, the wish for relevant, in which case it may best be retired (see additional comfort about the product’s safety Case Example 45). For medical devices, for profile can be translated into a measurable goal. example, newer technology is continuously Such a goal might be to exclude the occurrence of replacing the old, although safety issues for older life-threatening or fatal drug-related events at a technology may motivate continuing a registry of

332 Chapter 14. Modifying and Stopping Registries

an outmoded technology. A related issue arises 3.3 What Happens When a Registry when the question of interest evolves as data Ends? collection proceeds. Stopping or continuing the Stopping a registry might mean ceasing all registry depends on whether it can address the information collection and issuing a final report. changing goal or goals. That, in turn, depends on An intermediate decision that falls short of a full whether the governance of the registry provides stop might involve ceasing to accrue new patients adequate flexibility to refocus the registry in a new while continuing to collect information on existing direction. participants. This step may be useful if the registry The decision to stop a registry may also depend on goals are in the process of changing. If a registry is mundane considerations such as cost or staffing. to be stopped, the archiving rules should be For long-running registries, eventually the value of checked and followed, so that those who need to new information may face diminishing returns. consult the data for questions not fully addressed Some registries have central core staff, deeply in reports or publications can get their answers committed to the registry, who serve as its later, provided that the charter of the registry historical memory. Departure of such individuals allows it. Following German reunification in 1990, can cripple the registry’s function, and a decision it was determined that the East German National to stop may be appropriate. Similarly, a cohort of Cancer Registry, which had received detailed engaged investigators may disperse over time or reports on 2 million cancer cases from 1954 to lose interest in the registry. Funding sources may 1990, was in violation of West German privacy dry up, making it impossible for the registry to laws, and the data were quarantined. In the more function at a level that justifies its continued usual case, orderly archiving of the data in existence. anticipation of later access should be part of the A thorny question concerns how a registry can close-down procedure, in a manner consistent with 21 continue with altered ownership or governance. the charter under which the data were collected. Suppose a registry is formed with multiple A slightly different scenario occurs when the stakeholders, and one or more withdraws for the registry has a single sponsor whose purposes have reasons described above. For example, when the been achieved or determined to be unachievable, implantable cardioverter defibrillator (ICD) and the sponsor decides to end the registry. Is there registry was formed, it came about in response to a an obligation to patients or participating providers CMS Coverage with Evidence Development to continue the registry because some value (e.g., decision. The Heart Rhythm Society and the quality improvement, data for other comparisons) American College of Cardiology developed the can still be derived? It is difficult to argue that the registry with funding from industry to help sponsor has an ongoing financial responsibility institutions meet the need for registry participation once the registry has achieved or failed to achieve for payment purposes, and they layered quality its primary purpose, especially if this has been improvement and research goals onto that spelled out in the protocol and informed consent. mandate.20 The resulting registry was rapidly Yet one can argue that, to the extent that it is integrated into more than 2,000 institutions in the feasible and affordable to engage other United States. If CMS determines that the ICD stakeholders in discussions of potential registry is no longer needed for its purposes, the transitioning of the registry to other owners, this registry must determine if it will continue as a approach should be encouraged. Nontrivial issues quality improvement program and whether to add of data ownership, property, confidentiality, and other stakeholders and funding sources or patient privacy would need to be satisfactorily participation drivers (such as manufacturers, addressed to make such transitions possible, and insurers, or other government agencies such as therefore it is always best to consider this FDA). possibility early on in registry planning. Both the National Registry of Myocardial Infarction, sponsored by Genentech, Inc., and the OPTIMIZE-

333 Section III. Operating Registries

HF registry, sponsored by GlaxoSmithKline, 3.4 Summary of Considerations for successfully completed transitions to other Planning for the End of a Registry organizations (American College of Cardiology Experimental studies, such as clinical trials or field and American Heart Association, respectively) trials, come with a high ethical burden of when those registries were concluded, providing responsibility, which includes periodically their participating hospitals with the ability to reevaluating the ethical basis for continuing the continue the quality improvement efforts begun trial in the light of interim results. Consequently, under those registries.22, 23 trials require interim analyses and data safety There is no clear ethical obligation to participants monitoring boards, which decide whether the to continue a registry that has outlived its scientific study should be stopped for ethical reasons. In usefulness. In fact, altering the purpose of a nonexperimental studies, there is much less registry would be complicated unless the original motivation to conduct interim analyses because registry operators were interested in doing so. For there is no ethical motivation to do so. There is instance, if a registry is to be transferred, then it also no reason to appoint a data safety monitoring should be a restricted transfer (presumably a gift) board, although any study could appoint an to ensure that the permissions, terms, and external advisory board. If nonexperimental conditions under which it was compiled continue studies are planned to be of fixed length or fixed to be satisfied. The participants should be notified study size, they can be conducted as planned and should determine if they will continue without interim analyses, unless the time value of participation and allow their data to be used for an early, interim analysis is important enough to this new purpose. compensate for the added cost of conducting it and There are a few potential reasons to consider the tentativeness of the findings, which are based preserving registry data once the registry on only a subset of the planned study data. developers have determined that it should end. Even if a patient registry is undertaken as an One reason is that the data may be capable of open-ended project without a fixed endpoint, it producing a recognized public health benefit that need not continue forever. Unlike true surveillance will continue if the registry does. Another situation efforts, patient registries of novel therapies are not may be that the registry has historical importance, intended to monitor changes in occurrence rates such as a registry that tracks the outbreak of a over time. Rather, they are conducted to assemble novel infectious disease that may provide insight enough data to evaluate associations that could not into the transmission of the disease, if not now, be evaluated with the limited data available at the then sometime in the future. Longitudinal time of new product approval. Therefore, collections of data may also be useful for reasonable goals should be set for the amount of hypothesis generation. information to be collected in such registries, In creating a registry, the investigators should plan based on specific endpoints of interest. These goals what will happen to data when the registry ends. If can and should be cast in specific terms regarding a public health benefit might be realized from data quality, study enrollment, and precision of the registry data, then archiving of registry data is a estimates of specific measures that the registry is potential answer. Decisions must be made by the intended to describe. registry owners in careful consideration of other stakeholders, potential costs, and privacy and security concerns.

334 Chapter 14. Modifying and Stopping Registries

Case Examples for Chapter 14

Case Example 28. Determining when to stop reported cardiovascular defects, it was not an open-ended registry possible to determine the credibility of the Description The Bupropion Pregnancy potential signal using registry data alone. Further, Registry was an observational the sample size was not adequate to reach exposure-registration and definitive conclusions regarding the absolute or followup study to monitor relative risk of any specific birth defects in prenatal exposure to bupropion women using bupropion during pregnancy (as the and detect any major teratogenic registry was powered only to examine the rate of effect. birth defects overall) and was unlikely to achieve its goal as structured. Sponsor GlaxoSmithKline The advisory committee recommended a study to Year Started 1997 expedite the accumulation of pregnancy outcome Year Ended The registry closed to new data among women exposed to bupropion during enrollments on November 1, pregnancy. In response, a large, claims-based, 2007, and continued to follow retrospective cohort study was conducted. This existing cases through March 31, study enrolled 1,213 women exposed in the first 2008. trimester and did not confirm a consistent pattern of defects (Cole et al., 2007). The prevalence of No. of Sites Not applicable cardiovascular defects associated with first- No. of Patients 1,597 trimester exposure to bupropion was 10.7 per 1,000 infants. Challenge Results Bupropion, an antidepressant with the potential for prenatal exposure, was labeled with a The advisory committee reviewed the evidence pregnancy category C by the U.S. Food and Drug and concluded that the signal did not represent an Administration (FDA) due to prior animal data. increased risk. The committee recommended The manufacturer established a prospective discontinuation of the registry based on findings pregnancy registry to monitor pregnancy from the retrospective cohort and 10 years of exposures to bupropion for any potential surveillance through the registry. The committee increased risk of congenital anomalies. Because took the position that sufficient information had the purpose of the registry was postmarketing accumulated to meet the scientific objective of safety surveillance, the duration of the registry the registry. The high lost-to-followup rate was was open ended. The registry had collected data also taken into consideration. The registry closed on more than 1,500 exposed pregnant women to new enrollments on November 1, 2007, and over 10 years when a potential signal suggestive continued to follow existing cases through March of a bupropion-related increase in cardiovascular 31, 2008. birth defects emerged. Key Point Proposed Solution In a registry without a specified end date or target The advisory committee reviewed the registry size, it is important to periodically review the data to assess the potential signal. However, due registry data to determine if the registry has met to the potential bias from the large percentage of its scientific objectives and to ensure that the cases lost to followup (35.8%), retrospective registry purpose is still relevant. reports, and incomplete descriptions of the

335 Section III. Operating Registries

Case Example 28. Determining when to stop The Bupropion Pregnancy Registry. Final an open-ended registry (continued) Report: September 1, 1997–March 31, 2008. For More Information Issued August 2008. http://pregnancyregistry. gsk.com/bupropion.html. Accessed June 6, 2012. Cole JA, Modell JG, Haight BR. et al. Bupropion in pregnancy and the prevalence of congenital Alwan S, Reefhuis J, Botto LD, et al. Maternal malformations. Pharmacoepidemiol Drug Safety. use of bupropion and risk for congenital heart 2007;16:474–84. defects. Am J Obstet Gynecol. 2010 Jul;203(1):52.e1-6. Epub 2010 Apr 24.

Case Example 29. Challenges in transitions evolved through multiple iterations, including the and changes in data collection most recent transition in 2010. This registry Description The Cystic Fibrosis Foundation transition, prompted by the implementation of a (CFF) Patient Registry is a new technology platform and the need to update rare-disease registry that collects and expand the data collection fields, imposed data from clinical visits, operational challenges on the CFF working group hospitalizations, and care tasked with the registry transition. Challenges episodes to track national trends included modifying data collection fields, in morbidity and mortality, mapping historical data, and ensuring usability assess the effectiveness of for the sites. treatments, and drive quality Proposed Solution improvement in care for patients Planning for the latest transition began in 2007, with cystic fibrosis (CF). when CFF began searching for a new vendor with Sponsor Cystic Fibrosis Foundation an FDA-compliant (21 CFR, Part 11) Web Year Started 1986 platform. After selecting a vendor, CFF began the process of evaluating all data fields in the registry. Year Ended Ongoing Six working groups of subject matter experts No. of Sites 110 CFF-accredited care centers were convened to review data fields from the in the United States existing registry and make suggestions for improvements, additions, and deletions (in the No. of Patients More than 26,000 areas of genetics, pulmonology, microbiology, Challenge cystic fibrosis–related diabetes, transplantation, and infant care/ambiguous diagnosis). CFF staff The CFF Patient Registry collects information on created an online prototype of the registry that more than 26,000 patients with cystic fibrosis allowed reviewers, including registry physicians who receive care at one of over 110 CFF- and subject matter experts, to see and comment accredited care centers across the United States. on all data fields and functions planned for the The registry collects demographic and diagnostic new registry platform. The recommendations of information, lung function, nutritional status, these groups were vetted by CFF staff in order to respiratory microbiology, and other indicators of balance data entry burden for the care center staff disease progression, as well as prescribed and usefulness for future research. medications and hospitalizations. Registry data dating back to 1986 are used to track national After selecting data fields for the new version of trends in morbidity and mortality, to assess the the registry, focus shifted to designing online case effectiveness of treatments, and to drive quality report forms and mapping historical data into the improvement. The CFF Patient Registry has new registry. For example, the old registry

336 Chapter 14. Modifying and Stopping Registries

Case Example 29. Challenges in transitions Results and changes in data collection (continued) Three months after the successful addition of new Proposed Solution (continued) fields and data migration to the new registry, a platform collected information on 81 distinct survey was distributed to all sites to solicit user genotype mutations in addition to an “other, feedback and identify areas for further change. specify:” option. The genetics working group The response rate was high, with 70 percent of expanded the list of available mutation variables users providing feedback on their experiences to 269, and data previously entered into the with the new platform. Responders ranged from “other, specify:” field were mapped forward into new users to experienced sites, and results these new mutation variables. Once mapping was indicated that sites were still gaining familiarity complete, there were several iterative migrations with the new registry. Based on the survey of historical data into the new registry platform. responses, several modifications to the platform CFF staff carefully audited the import process were made to improve the functionality of the and identified errors prior to the launch of the live system. registry. Key Point During a preliminary test period, users were able The transition of the CFF Patient Registry to a to log in to the new registry, populated with new technology platform illustrates the “dummy” data, and become acquainted with the importance of a well-planned registry transition. new functionality, format, and questions in the Early allocation of human and financial registry. During a second test period, users resources, collaboration with sites, and a realistic viewed their actual patient data in the new timeline allowed for stepwise development. This registry, and helped to identify and resolve approach minimized disruption to users and several minor errors in data migration. When the ensured the integrity of the data migration registry went live in April 2010, users were process. supported with online training manuals and For More Information videos, FAQ documents, Webinars, and live support. www.cff.org

337 Section III. Operating Registries

Case Example 30. Transitioning from startup Initial funding for the registry was provided in to ongoing registry funding with public and 2009 through a grant from the National Institutes private partners of Health (NIH), which allowed the registry to be Description Rheumatoid Arthritis launched, patients to be enrolled, and data Comparative Effectiveness collection to begin. However, the grant Research (RACER) is a disease mechanism was for a finite period of 2 years, and registry designed to assess university researchers saw value in continuing comparative and cost data collection. Researchers sought to continue effectiveness of existing and registry operations and disseminate registry data new biologic drug therapies for and results to interested parties. rheumatoid arthritis. Proposed Solution Sponsor University of Pittsburgh; Prior to the end of NIH funding, the registry, National Institutes of Health; which now had data on more than 1,000 patients, Genentech, Inc. began discussions with Genentech, Inc, a Year Started 2010 biotechnology company with two marketed biologics for rheumatoid arthritis. Initially, the Year Ended Ongoing discussions focused on data sharing and acquiring No. of Sites 4 clinics in the Pittsburgh area access to the registry data set with an intent to collaborate and generate real-world effectiveness No. of Patients More than 1,000 of rheumatoid arthritis treatments. Although Challenge Genentech was not mandated by a regulatory agency to collect postmarketing data on their RACER was established by researchers at the product, they expressed interest in being able to University of Pittsburgh to determine relative collect this data for scientific purposes with effectiveness of existing versus new and minimal investment in redundant or duplicative expensive biologic therapies for rheumatoid registry infrastructure. Genentech valued the arthritis; compare biologic therapies in terms of work that RACER had done already, especially mechanistic and biomarker measurements to EHR integration, the collection of PROs and predict optimal treatment; and test whether quality-of-life measures, and the potential for practical treat-to-target strategies can improve nested substudies within the larger registry treatment management of the disease. Clinical database. outcomes assessed include disease activity, quality of life, function and work productivity, Discussions between the registry and Genentech and biomarker and mechanistic outcomes turned to possible models for sustaining registry including C-reactive protein, rheumatoid factor, operations while granting Genentech access to and cyclic citrullinated peptide auto-antibody the valuable comparative effectiveness data being levels. collected by the registry. There is no protocol-mandated visit schedule, and Results providers determine the visit schedule appropriate Plans are now in place to fully transition the for each patient. Registry data are extracted from funding of the registry to Genentech after the the site’s electronic health record (EHR) system original NIH contract has ended. Genentech will and supplemented with patient-reported outcome continue to have full access to registry data on an (PRO) measures and some physician-reported ongoing basis, but will have an arms-length measures at each visit. The laboratory values are relationship to the operations of the registry. The measured using a blood sample collected at each researchers and registry staff at the University of visit. Pittsburgh will retain full control of registry operations and ownership of the registry data set.

338 Chapter 14. Modifying and Stopping Registries

Case Example 30. Transitioning from startup Soejima M, Patel AM, Goudeau D, et al. Effect to ongoing registry funding with public and of disease modifying anti-rheumatic Drugs private partners (continued) (DMARDs) on anti-CCP2 and anti-citrullinated Results (continued) protein antibody (ACPA) levels during longitudinal assessments in rheumatoid arthritis The University of Pittsburgh researchers and patients. Arthritis Rheum. 2011 Genentech will meet regularly to collaborate on Oct;63(10):S146-7. analyses, manuscripts, and abstracts. Work is currently being planned on a protocol for a nested Soejima M, Zhijie Z, Jones DM, et al. Prognostic substudy to evaluate the relative efficacy of the and diagnostic significance of autoantibodies to four clinically relevant treatment options in citrullinated proteins (ACPA) in patients with a patients failing their first tumor necrosis factor scleroderma-rheumatoid arthritis (SSc-RA) (TNF) antagonist treatment. overlap syndrome. Arthritis Rheum. 2011 Oct;63(10):S332-3. Key Point Lupash D, Patel AM, Amity CL, et al. Registries can benefit from public and private Comparison of the Patient-Based Routine partners to secure funding during both the startup Assessment of Patient Index 3 in usual care of and maintenance phases of the registry. Members rheumatoid arthritis to the Physician-Based of a public-private partnership may join and leave Disease Activity Score-28 Joint Count and as funding becomes available and shared interests Clinical Disease Activity Index. Arthritis intersect. This partnership enables effective Rheum. 2011 Oct;63(10):S834. public-private collaboration to advance science using observational research. Patel AM, Amity CL, Frydrych LM, et al. Comparison of rheumatoid arthritis patient For More Information characteristics from randomized controlled trials Patel AM, Amity CL, Frydrych LM, et al. The to a registry designed for rheumatoid arthritis 2010 rheumatoid arthritis criteria versus the 1987 comparative effectiveness research. Arthritis rheumatoid arthritis criteria: Will the real criteria Rheum. 2011 Oct;63(10):S837-8. please stand up! Arthritis Rheum. 2011 Oct;63(10):S48-9.

339 Section III. Operating Registries

Case Example 31. Modifying a registry due to Proposed Solution changes in standards of care The sponsor had selected a steering committee Description The GOLD reGISTry was a with engaged key opinion leaders who provided prospective, multicenter, 5-year guidance for the study and encouraged flexibility global disease registry designed in study design to allow for potential changes. In to collect information on patients 2009, the steering committee convened and with advanced and localized determined that the registry would begin gastrointestinal stromal tumors. collecting data on patients with localized GIST, in The registry collected diagnostic, addition to those with advanced disease who were treatment, and outcomes already enrolled in the registry. The study team information in order to identify drafted a protocol amendment to include the and compare practice patterns localized GIST population and allowing worldwide and assist assessment of physician adherence to new clinical practitioners in making treatment guidelines published by the European Society of decisions as standards of care Medical Oncology the same year. The data evolved. management and statistical analysis plans were Sponsor Novartis Oncology also revised to allow for the incorporation of the new data. Year Started 2007 Significant efforts were then directed at site Year Ended 2012 engagement, including abstract submissions and No. of Sites More than 200 publicity through the key opinion leaders. The registry also maintained site interest through No. of Patients 1653 interim study summaries presented at professional Challenge congresses. The sponsor had limited monitoring resources available to accommodate the new When it was launched in 2007, the 5-year GOLD patient population, so study designers developed a reGISTry enrolled only patients with advanced plan that used remote monitoring and training, gastrointestinal stromal tumors (GIST). This reserving onsite visits for research-naïve sites or population was of interest to researchers because for-cause audits. This allowed monitors to focus standards of care for advanced GIST were not as on those sites that required more training and clearly defined and widely used as the standard of allowed these sites to gain clinical research care for localized GIST, which was complete experience in an observational study. surgical excision. The sponsor expected that the outcomes data collected from advanced GIST Results patients would be valuable in helping to refine The registry enrolled 1,653 patients in the two standards of care for these patients. populations within four years: more than 1,000 In 2008 and 2009, Gleevec®/Glivec® (imatinib with advanced GIST, and more than 500 with mesylate) received FDA and European Medicines localized GIST. The steering committee played an Agency (EMA) approval for adjuvant use in important role in the recruitment and retention of localized GIST after tumor resection. This sites, highlighting the importance of the study approval, combined with emerging clinical trial through publications and interim summaries data, prompted new interest in collecting presented at scientific and professional congresses diagnostic, treatment, and outcomes information throughout the enrollment period. As the planned from patients with localized GIST. 5-year study period has been completed, the sponsor is now in the process of locking the registry database in preparation for final analyses.

340 Chapter 14. Modifying and Stopping Registries

Case Example 31. Modifying a registry due to tumour: a randomised, double-blind, placebo- changes in standards of care (continued) controlled trial. Lancet. 2009 Mar Key Point 28;373(9669):1097-104. Changes in standard of care can significantly Casali PG, Jost L, Reichardt P, et al. impact the design of a study as new treatments Gastrointestinal stromal tumours : ESMO clinical are approved or new patient populations become recommendations for diagnosis, treatment and of interest. Registry developers should anticipate follow-up. Ann Oncol. 2009 May;20 Suppl 4: that such changes might occur, and should 64-7. consider what aspects of the registry could be Chacon M, Reichardt P, Gu J, et al. The GOLD most impacted. A steering committee well reGISTry: A global observational registry regarded in the field and knowledgeable about the collecting longitudinal data on patients with disease and treatment can provide significant advanced GIST—Second annual summary. J Clin guidance during registry transitions and keep Oncol (Meeting Abstracts). May 2010; 28 (15 sites engaged as the changes are implemented. Suppl.): 10092. For More Information DeMatteo RP, Ballman KV, Antonescu CR, et al. Adjuvant imatinib mesylate after resection of localised, primary gastrointestinal stromal

References for Chapter 14 6. Hauser RM, Weinstein M, Pool R, et al., eds. Conducting Biosocial Surveys: Collecting, 1. Nosowsky R, Giordano TJ. The Health Insurance Storing, Accessing, and Protecting Biospecimens Portability and Accountability Act of 1996 and Biodata. Washington, DC: National (HIPAA) privacy rule: implications for clinical Academies Press; 2010. research. Annu Rev Med. 2006;57:575-90. 7. Hofmann B. Broadening consent--and diluting PMID: 16409167. ethics? J Med Ethics. 2009 Feb;35(2):125-9. 2. Littenberg B, MacLean CD. Passive consent for PMID: 19181887. clinical research in the age of HIPAA. J Gen 8. Ries NM. Growing up as a research subject: Intern Med. 2006 Mar;21(3):207-11. ethical and legal issues in birth cohort studies PMID: 16637821. PMCID: 1828090. involving genetic research. Health Law J. 3. Al-Shahi R, Warlow C. Using patient-identifiable 2007;15:1-41. PMID: 19702179. data for observational research and audit. BMJ. 9. Ries NM, LeGrandeur J, Caulfield T. Handling 2000 Oct 28;321(7268):1031-2. PMID: 11053151. ethical, legal and social issues in birth cohort PMCID: 1118832. studies involving genetic research: responses from 4. Tu JV, Willison DJ, Silver FL, et al. studies in six countries. BMC Med Ethics. Impracticability of informed consent in the 2010;11:4. PMID: 20331891. PMCID: 2859353. Registry of the Canadian Stroke Network. N Engl 10. Knowles M. Adult Learning. In: Craing R, ed. The J Med. 2004 Apr 1;350(14):1414-21. ASTD Training and Development Handbook. New PMID: 15070791. York: McGraw-Hill; 1996. p. 253-64. 5. Truyers C, Kellen E, Arbyn M, et al. The use of 11. Rothman K, Greenland S, Lash TL, eds. Modern human tissue in epidemiological research; ethical Epidemiology. 3rd ed. New York: Lippincott and legal considerations in two biobanks in Williams & Wilkins; 2008. Belgium. Med Health Care Philos. 2010 May;13(2):169-75. PMID: 19936964.

341 Section III. Operating Registries

12. Rothman KJ, Johnson ES, Sugano DS. Is 19. Eng TR, Butler WT. The hidden epidemic: flutamide effective in patients with bilateral confronting sexually transmitted diseases. orchiectomy? Lancet. 1999 Apr 3;353(9159):1184. Institute of Medicine (U.S.) Committee on PMID: 10210003. Prevention and Control of Sexually Transmitted Diseases. Washington, DC: National Academies 13. Freedman B. Equipoise and the ethics of clinical Press; 1997. research. N Engl J Med. 1987 Jul 16;317(3): 141-5. PMID: 3600702. 20. Hammill S, Phurrough S, Brindis R. The National ICD Registry: now and into the future. Heart 14. Weijer C, Shapiro SH, Cranley Glass K. For and Rhythm. 2006 Apr;3(4):470-3. PMID: 16567298. against: clinical equipoise and not the uncertainty principle is the moral underpinning of the 21. House of Commons Select Committee on Science randomised controlled trial. BMJ. 2000 Sep and Technology. Science and Technology Sixth 23;321(7263):756-8. PMID: 10999914. Report. July 27, 2000. Section on Cancer PMCID: 1127868. Research: Epidemiology and Publich Health Research. http://www.publications.parliament.uk/ 15. McPherson K. Statistics: the problem of pa/cm199900/cmselect/cmsctech/332/33208. examining accumulating data more than once. N htm#a39. Accessed September 10, 2013. Engl J Med. 1974 Feb 28;290(9):501-2. PMID: 4589874. 22. New Program to Help Hospitals Improve Care for Health Failure Patients Saves Lives [press release]. 16. The antiepileptic drug registry. http://www. http://www.prnewswire.com/news-releases/ aedpregnancyregistry.org/. Accessed August 15, new-program-to-help-hospitals-improve-care-for- 2012. heart-failure-patients-saves-lives-54170737.html. 17. Acyclovir Pregnancy Registry and Valacyclovir Accessed August 28, 2012. Pregnancy Registry Interim Report, December 23. “American College Of Cardiology Foundation’s 1997. Glaxo Wellcome, RTP, NC 27709; as NCDR Creates Network To Measure Patient referenced on Web page titled, “GlaxoSmithKline Care.” Medical News Today. www. Pregnancy Registries.”. http://pregnancyregistry. medicalnewstoday.com/articles/55798.php. gsk.com/acyclovir.html. Accessed August 15, Accessed July 2, 2012. 2012. 18. ClinicalTrials.gov. Observational Familial Adenomatous Polyposis Registry Study In Patients Receiving Celecoxib Compared to Control Patients. http://clinicaltrials.gov/ct2/show/ NCT00151476. Accessed August 15, 2012.

342

U.S. Department of Registries for Evaluating Patient Outcomes: A User’s Guide Health and Human Services

Agency for Healthcare Research and Quality 540 Gaither Road Rockville, MD 20850

Volume 1

Registries for Evaluating Patient Outcomes: A User’s Guide

Third Edition

AHRQ Pub. No. 13(14)-EHC111 April 2014 ISBN: 978-1-58763-432-1