Assessing Quality of Admin Data for Use in Censuses
Total Page:16
File Type:pdf, Size:1020Kb
UNITED NATIONS ECONOMIC COMMISSION FOR EUROPE Guidelines for Assessing the Quality of Administrative Sources for Use in Censuses Prepared by the Conference of European Statisticians Task Force on Assessing the Quality of Administrative Sources for Use in Censuses United Nations Geneva, 2021 Preface The main purpose of this publication is to provide the producers of population and housing censuses with guidance on how to assess the quality of administrative data for use in the census. The Guidelines cover the practical stages of assessment, from working with an administrative data supplier to understand a source, its strengths and limitations, all the way to the receipt and analysis of the actual data. The Guidelines cover key quality dimensions on which an assessment is made, using various tools and indicators. For completeness the Guidelines also include information about the processing and output stages of the census, with respect to the use of administrative sources. The publication was prepared by a Task Force established by the Conference of European Statisticians (CES), composed of experts from national statistics offices, and coordinated by the United Nations Economic Commission for Europe (UNECE). ii Acknowledgements These Guidelines were prepared by the UNECE Task Force on Assessing the Quality of Administrative Sources for Use in Censuses, consisting of the following individuals: Steven Dunstan (Chair), United Kingdom Paula Paulino, Portugal Katrin Tschoner, Austria Dmitrii Calincu, Republic of Moldova Christoph Waldner, Austria Valentina Istrati, Republic of Moldova Josée Morel, Canada Marina Pérez Julián, Spain Lionel Espinasse, France Alberto Salcedo, Spain Stefan Dittrich, Germany Şebnem Beşe-Canpolat, Turkey Tobias Kahlenberg, Germany Muharrem Gürleyen Gök, Turkey Thomas Körner, Germany Mehmet Şaban Ucari, Turkey Ingeborg Vorndran, Germany Louisa Blackwell, United Kingdom Sheelagh Bonham, Ireland Adriana Castaldo, United Kingdom Brendan Murphy, Ireland Sara Correia, United Kingdom Alaa Atrash, Israel Sara Haylock, United Kingdom Yael Feinstein, Israel Charlotte Hillyard, United Kingdom Gerardo Gallo, Italy Jack Sim, United Kingdom Donatella Zindato, Italy Stephan Tietz, United Kingdom Snezana Remikovic, Montenegro Clare Watson, United Kingdom Eric Schulte Nordholt, Netherlands Marina Wright, United Kingdom Christine Bycroft, New Zealand Tom Mule, United States of America Abby Morgan, New Zealand Eduard Jongstra, UNFPA Janusz Dygaszewicz, Poland Diana Beltadze, Eurostat Krzysztof Woźnica, Poland Sorina Vâju, Eurostat João Farrajota, Portugal Ian White, Independent expert Sandra Lagarto, Portugal Fiona Willis-Núñez, UNECE The Guidelines were developed and agreed upon by the entire Task Force. Each chapter was drafted by a team under the leadership of one or more individuals, as follows: • Census methodologies and uses of administrative data for censuses: Sara Correia and Ian White • Quality framework: Sara Correia and Sorina Vâju • Source Stage: Sorina Vâju, Josée Morel, Diana Beltadze and Steven Dunstan • Data Stage: Christoph Waldner, Tobias Kahlenberg and Sara Correia • Process Stage: Sara Haylock, Abby Morgan, Adriana Castaldo and Steven Dunstan • Output Stage: Sara Correia, Marina Pérez, Sandra Lagarto and Steven Dunstan. The Task Force extends particular thanks to the United Kingdom’s Office for National Statistics for the invaluable contributions of many of its staff to these final Guidelines, most notably Steven Dunstan and Sara Correia who ensured that the Task Force met its goals and who brought this product to its completion. iii Contents Chapter 1. Introduction...................................................................................................... 1 1.1 Background....................................................................................................... 1 1.2 Use of administrative data in censuses ............................................................ 2 1.3 Key risks to quality ........................................................................................... 2 1.4 Scope and structure of the Guidelines ............................................................. 3 1.5 Summary of recommendations ....................................................................... 5 Chapter 2. Census methodologies and uses of administrative data for censuses ............ 7 2.1 Census methodologies ..................................................................................... 7 2.2 Uses of administrative data ............................................................................. 9 2.3 Types of administrative sources..................................................................... 12 Chapter 3. Quality framework ......................................................................................... 16 3.1 Quality and Error in Censuses ........................................................................ 16 3.2 Measuring Quality .......................................................................................... 18 3.3 Stages of Quality Assessment ........................................................................ 19 3.4 Quality Dimensions ........................................................................................ 19 3.5 Feasibility Research ........................................................................................ 24 Chapter 4. Source Stage ................................................................................................... 29 4.1 Source quality dimensions ............................................................................. 29 4.2 Tools and indicators ....................................................................................... 30 4.3 Recommendations ......................................................................................... 42 4.4 Case studies .................................................................................................... 43 Chapter 5. Data Stage ...................................................................................................... 49 5.1 Data quality dimensions ................................................................................. 49 5.2 Tools and indicators ....................................................................................... 51 5.3 Recommendations ......................................................................................... 57 5.4 Case studies .................................................................................................... 58 Chapter 6. Process Stage .................................................................................................. 64 6.1 Record linkage ................................................................................................ 64 6.2 Statistical registers and the ‘signs of life’ methodology ................................ 67 6.3 Enumeration of population units: administrative data-based models .......... 69 6.4 Conflict resolution/decision between sources .............................................. 71 6.5 Editing and Imputation................................................................................... 72 6.6 Recommendations ......................................................................................... 73 6.7 Case studies .................................................................................................... 74 Chapter 7. Output Stage .................................................................................................. 83 7.1 Output quality dimensions ............................................................................. 83 7.2 Further tools and processes ........................................................................... 86 7.3 Case studies .................................................................................................... 91 Chapter 8. Conclusions and recommendations ............................................................... 94 8.1 Recommendations ......................................................................................... 94 8.2 Areas for further development ...................................................................... 96 iv List of boxes Box 1: Feasibility research in Estonia ................................................................................... 26 Box 2: Feasibility research in Israel ...................................................................................... 27 Box 3: Statistics Canada's Trust Centre ................................................................................ 38 Box 4: Metadata templates for assessing administrative sources ...................................... 39 Box 5: A Quality Assurance Toolkit: Communication with data supply partners ................ 40 Box 6: Statistics Netherlands System of Base Registers ...................................................... 41 Box 7: Methods for data linkage and the assessment of linkage quality: a UK cross- government review .............................................................................................................. 66 Box 8: Determining occupancy at an address (the United States Census field operation) .............................................................................................................................................. 69 Box 9: Direct Enumeration (the New Zealand 2018 Census) ............................................... 70 Box 10: Demographic analysis in Spain...............................................................................