Nuno Crato Paolo Paruolo Editors Data-Driven Policy Impact Evaluation How Access to Microdata is Transforming Policy Design Data-Driven Policy Impact Evaluation Nuno Crato • Paolo Paruolo Editors Data-Driven Policy Impact Evaluation How Access to Microdata is Transforming Policy Design Editors Nuno Crato Paolo Paruolo University of Lisbon Joint Research Centre Lisbon, Portugal Ispra, Italy ISBN 978-3-319-78460-1 ISBN 978-3-319-78461-8 (eBook) https://doi.org/10.1007/978-3-319-78461-8 Library of Congress Control Number: 2018954896 © The Editor(s) (if applicable) and The Author(s) 2019. This book is an open access publication. Open Access This book is licensed under the terms of the Creative Commons Attribution 4.0 Inter- national License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made. The images or other third party material in this book are included in the book’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the book’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Preface Policymaking is undergoing profound transformations, thanks to the availability of better data and the possibility to uncover causal effects by using improved statistical methods (namely, counterfactual impact evaluation techniques). This book aims to document these changes in a simple language, from a European perspective. The central ideas of the book can be summarised in four paragraphs. Firstly, statistical methods now exist to rigorously evaluate the impact of policy measures, even when data do not come from controlled experiments. Micro-econometricians and statisticians specialising in these counterfactual impact evaluation methods are well aware of both their potential and limitations. The scope for rigorous causal analysis that they offer is, however, not widely understood. Owing to the inherent complexity of society and the many concurrent factors that influence an outcome, decision-makers often doubt that it is possible to uncover clear causal relationships. Secondly, to evaluate policy impacts, it is crucial to have data on the basic units targeted by a policy, i.e. the so-called target group. Similar data need to be available for a comparable set of units, called the control group. This often translates into a need to access microdata, i.e. data at the level of individuals, households, businesses or communities. Thirdly, microdata are available from several sources; one such source that offers numerous advantages is administrative data (or registry data), i.e. data collected by public entities for their service activities. Data from different registries can be linked together, and also linked with external information, such as data from surveys or data collected by private businesses, to obtain comprehensive datasets that are suitable for policy research. If these datasets are properly organised, policy evaluations can be performed in real time. Fourthly, the use of microdata faces challenges: some real, others imaginary. There are issues with database quality, linkage and preservation of anonymity. Most obstacles can be overcome with appropriate organisation and modern computer science methods. There are also problems of a political nature; to overcome them, one needs to be aware of all the technical solutions that can be used to keep data v vi Preface safe, as well as to be willing to subject policies to reality checks and to learn from those checks. To properly analyse these topics from all necessary angles, this book collects contributions from a group of researchers, practitioners and statistical officers who work in the field, mostly based in Europe: 11 European countries are represented. The pool of contributors is complemented by experts from the Organisation for Economic Co-operation and Development and from Eurostat, who report cases drawn from their organisations’ particular experiences. The various articles in this book consider different policy areas. They include employment, health, professional training, social security and educational issues. Many of the contributions explain and apply various counterfactual econometric methods, serving as a useful reference for their study and use. The book provides a panorama of microdata issues relating to policy research, of administrative data availability, of various existing systems that can facilitate safe data use, of successful studies on policy impact and even of policy changes made on the basis of such studies. We hope it will be useful to a large readership in Europe and in the rest of the world. Policymakers and public policy scholars will find here various examples of successful policy evaluation. They will find arguments in favour of data-based policy studies and a clear case for improving the effectiveness of policy measures. Public administrators and technical staff at public administrations will find systematic examples showing that policy evaluation is a viable task, with dedicated methods and sophisticated techniques. Various chapters show in detail the reasons why a causal evaluation is possible and the conditions under which these analyses can provide conclusive evidence on the effects of policies. Statisticians and econometricians will find various discussions on the applicabil- ity of counterfactual impact evaluation methods and detailed case studies that show how various techniques can be applied with success. All readers will find practical examples of the most commonly used of these techniques, along with a discussion of their applicability. Statistical officers and database experts will find a state-of- the-art review of anonymisation issues and techniques and of database linkage and security. To sum up: Microdata exist and can be safely organised for a better knowledge of society and for policy research; data-based policy assessment, monitoring, evaluation and improvement are feasible and desirable, and they can be achieved in unprecedentedly fast ways by properly linking administrative data. The first chapter introduces these issues. The remaining contributions to this book are grouped into four parts, which are described below. Microdata for Policy Research The first part of this book deals with data issues. Paul Jackson provides an updated account of the recent history of the use of microdata. It is only recently, around the Preface vii beginning of the twenty-first century, that the value of microdata and administrative data started to become fully appreciated. In fact, using microdata from administra- tive sources for policy research is still innovative. Previously, microdata collected for general statistics were only used to compute summary statistics such as average income, or the number of unemployed people in a particular region. Original records were treated as completely confidential and then discarded. Now, microdata are understood to have enormous potential for enabling us to improve knowledge about policies, and to a large extent they are inexpensive, as they have been already collected. Official statistical authorities, both at national level and in Eurostat, make microdata available for research. Public administrations such as unemployment agencies and educational registers are increasingly urged by social researchers, political parties and independent organisations to make available the data that they collect regularly and, nowadays, keep in digital format. Ugo Trivellato explains in what sense microdata should be seen as a public good and makes the case for data availability. He highlights the importance of data release, both for research and for democratic participation. It is a matter of public interest that data should be made available for a better knowledge of society and for the evaluation of public policies. In addition, he reviews some recent advances in regulations and practices on microdata access—at the level of EU and several member states, and at transnational level—and concludes that remote data access is the most effective mode for safely sharing highly informative confidential data. The full use of administrative data requires the linkage of various databases. Natalie Shlomo explains how modern statistical techniques can be applied to data linkage and how probabilistic