Data Quality Monitoring and Surveillance System Evaluation
Total Page:16
File Type:pdf, Size:1020Kb
TECHNICAL DOCUMENT Data quality monitoring and surveillance system evaluation A handbook of methods and applications www.ecdc.europa.eu ECDC TECHNICAL DOCUMENT Data quality monitoring and surveillance system evaluation A handbook of methods and applications This publication of the European Centre for Disease Prevention and Control (ECDC) was coordinated by Isabelle Devaux (senior expert, Epidemiological Methods, ECDC). Contributing authors John Brazil (Health Protection Surveillance Centre, Ireland; Section 2.4), Bruno Ciancio (ECDC; Chapter 1, Section 2.1), Isabelle Devaux (ECDC; Chapter 1, Sections 3.1 and 3.2), James Freed (Public Health England, United Kingdom; Sections 2.1 and 3.2), Magid Herida (Institut for Public Health Surveillance, France; Section 3.8 ), Jana Kerlik (Public Health Authority of the Slovak Republic; Section 2.1), Scott McNabb (Emory University, United States of America; Sections 2.1 and 3.8), Kassiani Mellou (Hellenic Centre for Disease Control and Prevention, Greece; Sections 2.2, 2.3, 3.3, 3.4 and 3.5), Gerardo Priotto (World Health Organization; Section 3.6), Simone van der Plas (National Institute of Public Health and the Environment, the Netherlands; Chapter 4), Bart van der Zanden (Public Health Agency of Sweden; Chapter 4), Edward Valesco (Robert Koch Institute, Germany; Sections 3.1 and 3.2). Project working group members: Maria Avdicova (Public Health Authority of the Slovak Republic), Sandro Bonfigli (National Institute of Health, Italy), Mike Catchpole (Public Health England, United Kingdom), Agnes Csohan (National Centre for Epidemiology, Hungary), Yves Dupont (Scientific Institute of Public Health, Belgium), Irena Klavs (National Institute of Public Health, Slovenia), Anna Kurchatova (National Centre of Infectious and Parasitic Diseases, Bulgaria), Mathias Leroy (Scientific Institute of Public Health, Belgium), David Mercer (World Health Organization, Regional Office for Europe), Zsuzsanna Molnar (National Centre for Epidemiology, Hungary), Pierre Nabeth (World Health Organization), Elvira Rizzuto (National Institute of Health, Italy), Malgorzata Sadkowska (National Institute of Public Health, Poland), Gudrun Sigmundsdottir (Centre of Health Security and Communicable Disease Prevention, Island), Ewa Staszewska (National Institute of Public Health, Poland), Reinhild Strauss (Federal Ministry of Health, Austria). Contributing ECDC experts: Cristian Avram, Arnold Bosman, Sergio Brusin, Bruno Ciancio, Denis Coulombier, Isabelle Devaux, Erika Duffell, Ana-Belen Escriva, Rodrigo Filipe, Graham Fraser, Keith Hodson, Frantiska Hruba, Charles Johnston, Marius Mag, Vladimir Prikazsky, Carmen Varela-Santos. Acknowledgements We would also like to thank the following individuals for their help, input and feedback: Elliot Churchill, Larisa Fedarushchanka, Lisa Ferland, Carmen Emily Hazim (all Public Health Practice, LLC; United States of America), Christopher Williams (Public Health England, United Kingdom), Michaela Dierke (Robert Koch Institute, Germany). Suggested citation: European Centre for Disease Prevention and Control. Data quality monitoring and surveillance system evaluation – A handbook of methods and applications. Stockholm: ECDC; 2014. Stockholm, September 2014 ISBN 978-92-9193-592-5 doi 10.2900/35329 Catalogue number TQ-04-14-829-EN-N © European Centre for Disease Prevention and Control, 2014 Reproduction is authorised, provided the source is acknowledged ii TECHNICAL DOCUMENT Data quality monitoring and surveillance system evaluation Contents Abbreviations .............................................................................................................................................. vii 1 Introduction ...............................................................................................................................................1 1.1 Background ........................................................................................................................................1 1.1.1 Monitoring data quality .................................................................................................................1 1.1.2 Evaluating surveillance systems .....................................................................................................2 1.2 Surveillance attributes .........................................................................................................................2 1.2.1 Completeness and validity .............................................................................................................2 1.2.2 Sensitivity, specificity, positive predictive value and negative predictive value .................................... 3 1.2.3 Timeliness ...................................................................................................................................5 1.2.4 Usefulness ...................................................................................................................................5 1.2.5 Representativeness ......................................................................................................................5 1.2.6 Other surveillance attributes ..........................................................................................................6 1.2.7 Development of data quality indicators ...........................................................................................6 1.3 Description of a surveillance system ......................................................................................................7 1.3.1 Surveillance objectives ..................................................................................................................8 1.3.2 List of diseases under surveillance and case definitions .................................................................... 9 1.3.3 Data sources and data flow ......................................................................................................... 10 1.3.4 Surveillance networks ................................................................................................................. 11 1.3.5 Populations under surveillance ..................................................................................................... 12 1.3.6 Geographic coverage .................................................................................................................. 12 1.3.7 Types of surveillance .................................................................................................................. 12 1.3.8 Specification of the information to be reported .............................................................................. 13 1.3.9 Reporting format ........................................................................................................................ 14 1.3.10 Data entry ............................................................................................................................... 14 1.3.11 System architecture .................................................................................................................. 15 1.4 References ....................................................................................................................................... 16 2 Monitoring data quality .............................................................................................................................. 17 2.1 Conceptual framework ....................................................................................................................... 17 2.1.1 Underreporting and under-ascertainment ..................................................................................... 17 2.1.2 Setting up a data quality monitoring system ................................................................................. 19 2.2 Monitoring completeness ................................................................................................................... 23 2.2.1 Describe the system ................................................................................................................... 23 2.2.2 Plan the monitoring process ........................................................................................................ 25 2.2.3 Measure internal completeness and summarise results .................................................................. 26 2.2.4 Interpret the results of the measurement ..................................................................................... 27 2.2.5 Make suggestions for remediation ................................................................................................ 28 2.3 Monitoring internal validity ................................................................................................................. 32 2.3.1 Describe the system ................................................................................................................... 32 2.3.2 Plan the monitoring process ........................................................................................................ 33 2.3.3 Measure internal validity and summarise the results ...................................................................... 34 2.3.4 Interpret the results of the measurement ..................................................................................... 36 2.3.5 Make suggestions for remediation ................................................................................................ 37 2.3.6 Questions