DOCSLIB.ORG

Essnet Big Data II

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Essnet Big Data II ESSnet Big Data II Grant Agreement Number: 847375 -2018-NL-BIGDATA https://webgate.ec.europa.eu/fpfis/mwikis/essnetbigdata https://ec.europa.eu/eurostat/cros/content/essnetbigdata_en Work package K Methodology and quality Deliverable K5: First draft of the m ethodological report Final version, 17.06.2020 Prepared by: Piet Daas (CBS, NL) Jacek Maślankowski (GUS, PL) David Salgado (INE, ES) Sónia Quaresma (INE, PT) ESSnetTiziana co-ordinator: Tuoto, Loredana Di Consiglio, Giovanna Brancato, Paolo Righi (ISTAT, IT) Magdalena Six, Alexander Kowarik (STAT, AT) Work package leader: Alexander Kowarik (STAT, AT) [email protected] telephone : +43 1 71128 7513 Contents 0. Prologue/Starting point ...................................................................................................................... 4 1. Overview of Big Data based official statistics ................................................................................... 5 1.1 Using Big Data ............................................................................................................................. 6 1.2 Lessons learned from Google Flu Trends .................................................................................... 7 1.3 Large n, large p and large t .......................................................................................................... 7 1.4 Relation with Quality and Typology reports ............................................................................... 8 Input Phase ........................................................................................................................................... 10 2. Introductory overview of methods for text and image mining ........................................................ 10 2.1 Text mining and NLP................................................................................................................. 10 2.1.1 Text ..................................................................................................................................... 10 2.1.2 Text mining tasks ................................................................................................................ 11 2.1.3 Examples of text mining applications ................................................................................ 13 2.2 Image mining ............................................................................................................................. 14 2.2.1 Bag of visual words ............................................................................................................ 14 2.2.2 Deep Learning ..................................................................................................................... 15 2.2.3 Examples of image mining applications ............................................................................. 16 2.3 Discussion .................................................................................................................................. 16 Throughput Phase 1 ............................................................................................................................. 17 3. Big Data exploration ........................................................................................................................ 17 3.1 Visualization methods for large datasets ................................................................................... 18 3.1.1 Univariate data .................................................................................................................... 18 3.1.2 Bivariate data ...................................................................................................................... 20 2.1.3 Other relevant visualization methods .................................................................................. 21 2.2 Examples of visualization methods used ................................................................................... 23 3.3 Discussion .................................................................................................................................. 24 4. Combining sources........................................................................................................................... 25 4.1 Linking units .............................................................................................................................. 26 4.1.1 Reasons for linking ............................................................................................................. 26 4.1.2 Linking units and linking variables..................................................................................... 26 4.1.3 Linking methods ................................................................................................................. 27 4.1.4 Suggestions for improvements on methods ........................................................................ 27 4.2 Examples of combining sources ................................................................................................ 27 4.3 Discussion .................................................................................................................................. 28 5. Dealing with errors .......................................................................................................................... 29 5.1 Error dimension – Coverage ...................................................................................................... 29 5.2 Error dimension – Comparability over time .............................................................................. 29 5.3 Error dimension - Measurement errors ...................................................................................... 29 5.4 Model errors (raw data to statistical data).................................................................................. 30 5.5 Discussion (Link with Quality deliverable and relation with user needs) ................................. 30 Throughput Phase 2 ............................................................................................................................. 31 6. Inference .......................................................................................................................................... 31 6.1 General considerations ............................................................................................................... 31 6.2 Big Data as the main source of input ......................................................................................... 31 6.3 Conceptual differences ............................................................................................................... 33 6.4 Representativity (selectivity) ..................................................................................................... 33 6.5 Sampling .................................................................................................................................... 37 6.6 Accuracy assesment (Bias and variance) ................................................................................... 38 6.7 Additional considerations .......................................................................................................... 39 6.8 Discussion .................................................................................................................................. 39 7. Conclusions ...................................................................................................................................... 40 References ............................................................................................................................................ 41 Annexes................................................................................................................................................ 46 Annex A: BREAL (Big data Reference Architecture and Layers) .................................................. 46 0. Prologue/Starting point This report provides an overview of the current state of art of Big Data methodology. As we have found that the range of what one considers Big Data methodology differs tremendously on the experience and area of expertise of a particular statistical researcher, we decided to apply a very strict (narrow) focus on what is included in this report. Essential starting point of any method described in this report is that it: i) has been used by researchers involved in the ESSnet Big Data I and II projects, ii) is included in one of the Big Data based official statistics currently being published, iii) is used in one of the Big Data based statistics that is in the process of becoming officially published. Any other method, however interesting it may seem, is not included in this report. For the reader it is important to realize that the methods included in this report are divided in various (sub)sections that relate to the various phases of the statistical process in which they are used. The same phases, input, throughput (1 & 2) and output, were used in the quality guidelines report (WPK Del. K3) published earlier. 1. Overview of Big Data based official statistics At the moment of writing, 13 examples of statistics are known that make use of Big Data for statistics published by National Statistical Institutes (NSI’s). These are listed in table 1. Unfortunately, not all processes have already completely matured and only two are in production. These numbers will undoubtable increase
Load more

Recommended publications
  • 1988: Coverage Error in Establishment Surveys
    COVERAGE ERROR IN ESTABLISHMENT SURVEYS Carl A. Konschnik U.S. Bureau of the CensusI I. Definition of Coverage Error in the survey planning stage results in a sam- pled population which is too far removed from Coverage error which includes both under- the target population. Since estimates based coverage and overcoverage, is defined as "the on data drawn from the sampled population apply error in an estimate that results from (I) fail- properly only to the sampled population, inter- ure to include all units belonging to the est in the target population dictates that the defined population or failure to include speci- sampled population be as close as practicable fied units in the conduct of the survey to the target population. Nevertheless, in the (undercoverage), and (2) inclusion of some units following discussion of the sources, measure- erroneously either because of a defective frame ment, and control of coverage error, only or because of inclusion of unspecified units or deficiencies relative to the sampled population inclusion of specified units more than once in are included. Thus, when speaking of defective the actual survey (overcoverage)" (Office of frames, only those deficiencies are discussed Federal Statistical Policy and Standards, 1978). which arise when the population which is sampled Coverage errors are closely related to but differs from the population intended to be clearly distinct from content errors, which are sampled (the sampled population). defined as the "errors of observation or objec- tive measurement, of recording, of imputation, Coverage Error Source Categories or of other processing which results in associ- ating a wrong value of the characteristic with a We will now look briefly at the two cate- specified unit" (Office of Federal Statistical gories of coverage error--defective frames and Policy and Standards, 1978).
    [Show full text]
  • American Community Survey Design and Methodology (January 2014) Chapter 15: Improving Data Quality by Reducing Non-Sampling Erro
    American Community Survey Design and Methodology (January 2014) Chapter 15: Improving Data Quality by Reducing Non-Sampling Error Version 2.0 January 30, 2014 ACS Design and Methodology (January 2014) – Improving Data Quality Page ii [This page intentionally left blank] Version 2.0 January 30, 2014 ACS Design and Methodology (January 2014) – Improving Data Quality Page iii Table of Contents Chapter 15: Improving Data Quality by Reducing Non-Sampling Error ............................... 1 15.1 Overview .......................................................................................................................... 1 15.2 Coverage Error ................................................................................................................. 2 15.3 Nonresponse Error............................................................................................................ 3 15.4 Measurement Error ........................................................................................................... 6 15.5 Processing Error ............................................................................................................... 7 15.6 Census Bureau Statistical Quality Standards ................................................................... 8 15.7 References ........................................................................................................................ 9 Version 2.0 January 30, 2014 ACS Design and Methodology (January 2014) – Improving Data Quality Page iv [This page intentionally left
    [Show full text]
  • R(Y NONRESPONSE in SURVEY RESEARCH Proceedings of the Eighth International Workshop on Household Survey Nonresponse 24-26 September 1997
    ZUMA Zentrum für Umfragen, Melhoden und Analysen No. 4 r(y NONRESPONSE IN SURVEY RESEARCH Proceedings of the Eighth International Workshop on Household Survey Nonresponse 24-26 September 1997 Edited by Achim Koch and Rolf Porst Copyright O 1998 by ZUMA, Mannheini, Gerinany All rights reserved. No part of tliis book rnay be reproduced or utilized in any form or by aiiy means, electronic or mechanical, including photocopying, recording, or by any inforniation Storage and retrieval System, without permission in writing froni the publisher. Editors: Achim Koch and Rolf Porst Publisher: Zentrum für Umfragen, Methoden und Analysen (ZUMA) ZUMA is a member of the Gesellschaft Sozialwissenschaftlicher Infrastruktureinrichtungen e.V. (GESIS) ZUMA Board Chair: Prof. Dr. Max Kaase Dii-ector: Prof. Dr. Peter Ph. Mohlcr P.O. Box 12 21 55 D - 68072.-Mannheim Germany Phone: +49-62 1- 1246-0 Fax: +49-62 1- 1246- 100 Internet: http://www.social-science-gesis.de/ Printed by Druck & Kopie hanel, Mannheim ISBN 3-924220-15-8 Contents Preface and Acknowledgements Current Issues in Household Survey Nonresponse at Statistics Canada Larry Swin und David Dolson Assessment of Efforts to Reduce Nonresponse Bias: 1996 Survey of Income and Program Participation (SIPP) Preston. Jay Waite, Vicki J. Huggi~isund Stephen 1'. Mnck Tlie Impact of Nonresponse on the Unemployment Rate in the Current Population Survey (CPS) Ciyde Tucker arzd Brian A. Harris-Kojetin An Evaluation of Unit Nonresponse Bias in the Italian Households Budget Survey Claudio Ceccarelli, Giuliana Coccia and Fahio Crescetzzi Nonresponse in the 1996 Income Survey (Supplement to the Microcensus) Eva Huvasi anci Acfhnz Marron The Stability ol' Nonresponse Rates According to Socio-Dernographic Categories Metku Znletel anci Vasju Vehovar Understanding Household Survey Nonresponse Through Geo-demographic Coding Schemes Jolin King Response Distributions when TDE is lntroduced Hikan L.
    [Show full text]
  • Coverage Bias in European Telephone Surveys: Developments of Landline and Mobile Phone Coverage Across Countries and Over Time
    www.ssoar.info Coverage Bias in European Telephone Surveys: Developments of Landline and Mobile Phone Coverage across Countries and over Time Mohorko, Anja; Leeuw, Edith de; Hox, Joop Veröffentlichungsversion / Published Version Zeitschriftenartikel / journal article Empfohlene Zitierung / Suggested Citation: Mohorko, A., Leeuw, E. d., & Hox, J. (2013). Coverage Bias in European Telephone Surveys: Developments of Landline and Mobile Phone Coverage across Countries and over Time. Survey Methods: Insights from the Field, 1-13. https://doi.org/10.13094/SMIF-2013-00002 Nutzungsbedingungen: Terms of use: Dieser Text wird unter einer CC BY-NC-ND Lizenz This document is made available under a CC BY-NC-ND Licence (Namensnennung-Nicht-kommerziell-Keine Bearbeitung) zur (Attribution-Non Comercial-NoDerivatives). For more Information Verfügung gestellt. Nähere Auskünfte zu den CC-Lizenzen finden see: Sie hier: https://creativecommons.org/licenses/by-nc-nd/4.0 https://creativecommons.org/licenses/by-nc-nd/4.0/deed.de Diese Version ist zitierbar unter / This version is citable under: https://nbn-resolving.org/urn:nbn:de:0168-ssoar-333873 Survey Methods: Insights from the Field Anja Mohorko, Department of Methodology and Statistics, Utrecht University Edith de Leeuw, Department of Methodology and Statistics, Utrecht University Joop Hox, Department of Methodology and Statistics, Utrecht University 25.01.2013 How to cite this article: Mohorko, A., de Leeuw, E., & Hox, J. (2013). Coverage Bias in European Telephone Surveys: Developments of Landline and Mobile Phone Coverage across Countries and over Time. Survey Methods: Insights from the Field. Retrieved from http://surveyinsights.org/?p=828 Abstract With the decrease of landline phones in the last decade, telephone survey methodologists face a new challenge to overcome coverage bias.
    [Show full text]
  • A Total Survey Error Perspective on Comparative Surveys
    A Total Survey Error Perspective on Comparative Surveys October 2nd 2014 – ITSEW – Washington DC Beth-Ellen Pennell, University of Michigan Lars Lyberg, Stockholm University Peter Mohler, University of Mannheim Kristen Cibelli Hibben, University of Michigan (presenting author) Gelaye Worku, Stockholm University © 2014 by the Regents of the University of Michigan Presentation Outline 1. Background 2. Integrated Total Survey Error Model Contribution to Errors: Representation: Coverage, Sampling, Nonresponse errors Measurement: Validity, Measurement errors Measurement error – expanded view 3. Conclusion © 2014 by the Regents of the University of Michigan Background 1. The scope, scale, and number of multicultural, multiregional or multinational (3M) surveys continues to grow a. Examples: European Social Survey (20+ countries), the World Values Survey (75+ countries), the Gallup World Poll (160 countries) 2. The purpose is comparability – An estimate for one country should be reasonably comparable to another. 3. However, comparability in 3M surveys is affected by various error sources that we are used to considering plus some additional ones. © 2014 by the Regents of the University of Michigan Background, continued… 1. 3M surveys have an error typology that differs from monocultural surveys. a. Error sources in 3M surveys are typically amplified b. 3M surveys often require additional operations – i.e. adaptation, translation 2. Few attempts have been made to adapt the TSE framework to 3M surveys (except see: Smith’s (2011) refined TSE model). 3. The proposed TSE typology integrates error sources – and associated methodological and operational challenges that may limit comparability. © 2014 by the Regents of the University of Michigan Total Survey Error (Groves et al., 2004) © 2014 by the Regents of the University of Michigan An Integrated TSE Model Some general points: 1.
    [Show full text]
  • Ethical Considerations in the Total Survey Error Context
    Ethical Considerations in the Total Survey Error Context Julie de Jong 3MCII International Conference July 26-29, 2016 Chicago, Illinois © 2014 by the Regents of the University of Michigan Acknowledgements • Invited paper in Advances in Comparative Survey Methods: Multicultural, Multinational and Multiregional Contexts (3MC), edited by T.P. Johnson, B.E. Pennell, I.A.L. Stoop, & B. Dorer. New York, NY: John Wiley & Sons. • Special thanks to Kirsten Alcser, Christopher Antoun, Ashley Bowers, Judi Clemens, Christina Lien, Steve Pennell, and Kristen Cibelli Hibben who contributed substantially to earlier iterations of the chapter “Ethical Considerations” in the Cross-Cultural Survey Guidelines from which this chapter draws material. © 2014 by the Regents of the University of Michigan Overview – Legal and professional regulations and socio-political contexts can lead to differences in how ethical protocols are realized across countries. – Differences in the implementation of ethical standards contributes to the sources of total survey error (TSE) • TSE describes statistical properties of survey estimates by incorporating a variety of error sources and is used in the survey design stage, in the data collection process, and evaluation and post survey adjustments (Groves & Lyberg, 2010) • The TSE framework is generally used to design and evaluate a single-country survey – The framework has been extended to 3MC surveys and the impact of TSE on overall comparison error (Smith 2011) 3 © 2014 by the Regents of the University of Michigan Objectives
    [Show full text]
  • Standards and Guidelines for Reporting on Coverage Martin Provost Statistics Canada R
    Standards and Guidelines for Reporting on Coverage Martin Provost Statistics Canada R. H. Coats Building, 15th Floor, Tunney’s Pasture, Ottawa ONT, Canada, K1A 0T6, [email protected] Key Words: Quality indicators, coverage, ideal, target, survey and frame populations 1. Introduction The level of coverage provided by a survey or a statistical program has a major impact on the overall quality of the survey process. It is therefore important that users of the survey data be informed of the coverage quality provided by the survey and of any potential problem associated with identified defects in terms of coverage. Those measures of coverage add to the set of quality indicators that together provide a complete picture that facilitate an adequate use of the survey data and an optimization of the survey resources to allow for its constant improvement. But how should one define coverage (based on what units, etc.)? And how should survey managers report on it? This paper outlines the content and describes the procedures, issues and topics which will have to be covered in standards and guidelines for reporting on coverage at Statistics Canada to standardize the terminology, the definitions and the measures relating to these coverage issues. These standards and guidelines have been proposed for inclusion in Statistics Canada's Quality Assurance Framework as a supplement to this agency's Policy on Informing Users of Data Quality and Methodology. 2. Statistics Canada’s Quality Assurance Framework Statistics Canada (STC) has always invested considerable effort in achieving an adequate level of quality for all of its products and services.
    [Show full text]
  • Survey Coverage
    MEMBERS OF THE FEDERAL COMMITTEE ON STATISTICAL METHODOLOGY (April 1990) Maria E. Gonzalez (Chair) office of Management and Budget Yvonne M. Bishop Daniel Kasprzyk Energy Information Bureau of the Census Administration Daniel Melnick Warren L. Buckler National Science Foundation Social Security Administration Robert P. Parker Charles E. Caudill Bureau of Economic Analysis National Agricultural Statistical Service David A. Pierce Federal Reserve Board John E. Cremeans Office of Business Analysis Thomas J. Plewes Bureau of Labor Statistics Zahava D. Doering Smithsonian Institution Wesley L. Schaible Bureau of Labor Statistics Joseph K. Garrett Bureau of the Census Fritz J. Scheuren Internal Revenue service Robert M. Groves Bureau of the Census Monroe G. Sirken National Center for Health C. Terry Ireland Statistics National Computer Security Center Robert D. Tortora Bureau of the Census Charles D. Jones Bureau of the Census PREFACE The Federal Committee on Statistical Methodology was organized by the Office of Management and Budget (OMB) in 1975 to investigate methodological issues in Federal statistics. Members of the committee, selected by OMB on the basis of their individual expertise and interest in statistical methods, serve in their personal capacity rather than as agency representatives. The committee conducts its work through subcommittees that are organized to study particular issues and that are open to any Federal employee who wishes to participate in the studies. working papers are prepared by the subcommittee members and reflect only their individual and collective ideas. The Subcommittee on Survey Coverage studied the survey errors that can seriously bias sample survey data because of undercoverage of certain subpopulations or because of overcoverage of other subpopulations.
    [Show full text]
  • STANDARDS and GUIDELINES for STATISTICAL SURVEYS September 2006
    OFFICE OF MANAGEMENT AND BUDGET STANDARDS AND GUIDELINES FOR STATISTICAL SURVEYS September 2006 Table of Contents LIST OF STANDARDS FOR STATISTICAL SURVEYS ....................................................... i INTRODUCTION......................................................................................................................... 1 SECTION 1 DEVELOPMENT OF CONCEPTS, METHODS, AND DESIGN .................. 5 Section 1.1 Survey Planning..................................................................................................... 5 Section 1.2 Survey Design........................................................................................................ 7 Section 1.3 Survey Response Rates.......................................................................................... 8 Section 1.4 Pretesting Survey Systems..................................................................................... 9 SECTION 2 COLLECTION OF DATA................................................................................... 9 Section 2.1 Developing Sampling Frames................................................................................ 9 Section 2.2 Required Notifications to Potential Survey Respondents.................................... 10 Section 2.3 Data Collection Methodology.............................................................................. 11 SECTION 3 PROCESSING AND EDITING OF DATA...................................................... 13 Section 3.1 Data Editing ........................................................................................................
    [Show full text]
  • Chapter VIII Non-Observation Error in Household Surveys in Developing Countries
    Household Sample Surveys in Developing and Transition Countries Chapter VIII Non-observation error in household surveys in developing countries James Lepkowski University of Michigan Ann Arbor, Michigan, United States of America Abstract Non-observation in a survey occurs when measurements are not or cannot be made on some of the target population or the sample. The non-observation may be complete, in which case no measurement is made at all on a unit (such as a household or person), or partial, in which case some, but not all, of the desired measurements are made on a unit. The present chapter discusses two sources of non-observation, non-coverage and non-response. Non-coverage occurs when units in the population of interest have no chance of being selected for the survey. Non-response occurs when a household or person selected for the survey does not participate in the survey or does participate but does not provide complete information. The chapter examines causes, consequences and steps to remedy non-observation errors. Non-coverage and non- response can result in biased survey estimates when the part of the population or sample left out is different than the part that is observed. Since these biases can be severe, a number of remedies and adjustments for non-coverage and non-response are discussed. Key terms: non-response, non-coverage, bias, target population, sampling frame, response rates. 149 Household Sample Surveys in Developing and Transition Countries A. Introduction 1. Non-observation in survey research is the result of failing to make measurements on a part of the survey target population.
    [Show full text]
  • Methodological Background 1
    Methodological Background 1. Available Data 1.1. Eurobarometer In terms of coverage of the household population, face-to-face interviews are often viewed as the gold standard by which other modes are compared (e.g., Groves et al., 2009). Since 2000 the Eurobarometer, which is based on face-to-face interviews, contains a question about both landline and mobile phone ownership. This provides us with a unique data set to analyze landline and mobile phone coverage figures across European countries and over time. The Eurobarometer regularly collects data for the European Community across EU members and applicant countries through face-to-face interviews. The Eurobarometer has a separate data collection for East and West Germany, Republic of Cyprus and the Turkish Republic of Northern Cyprus, and Great Britain and Northern Ireland. In 2000 seventeen countries were part of the Eurobarometer, namely Austria, Belgium, Denmark, Finland, France, Germany (East and West), Great Britain, Greece, Ireland, Italy, Luxembourg, Netherlands, Portugal, Spain, and Sweden. Sixteen new countries joined in the year 2004, namely Bulgaria, Croatia, Cyprus (Republic of Cyprus and Turkish Republic of Northern Cyprus), Czech Republic, Estonia, Hungary, Latvia, Lithuania, Malta, Northern Ireland, Poland, Romania, Slovakia, Slovenia, and Turkey. We have analyzed all available data, see the analysis section for details. Each wave of the Eurobarometer consists of face-to-face interviews with adults aged 18+ and includes a core questionnaire plus an additional questionnaire with special topics. For each standard Eurobarometer survey new and independent samples are drawn; since October 1989 the basic sampling design is a multi-stage probability sample (for more details, see Gesis Eurobarometer Survey series, 2012).
    [Show full text]
  • A Recommendation for Net Undercount Estimation in Iran Population and Dwelling Censuses
    A Recommendation for Net Undercount Estimation in Iran Population and Dwelling Censuses Sepideh Mosaferi a Hamidreza Navvabpourb a Allameh Tabataba’i University, Tehran, Iran b Allameh Tabataba’i University and Statistical Research and Training Center, Tehran, Iran December 15, 2011 Abstract Census counts are subject to different types of nonsampling errors. One of these main errors is coverage error. Undercount and overcount are two types of coverage error. Undercount usually occurs more than the other, thus net undercount estimation is important. There are various methods for estimating the coverage error in censuses. One of these methods is dual system (DS) that usually uses data from the census and a post-enumeration survey (PES). In this paper, the coverage error and necessity of its evaluation, PES design and DS method are explained. Then PES associated approaches and their effects on DS estimation are illustrated and these approaches are compared. Finally, we explain the Statistical Center of Iran method of estimating net undercount in Iran 2006 population and dwelling census and a suggestion will be given for improving net undercount estimation in population and dwelling censuses of Iran. Keywords: Census; coverage error; dual system; net undercount; nonsampling error; post- enumeration survey. 1 Introduction arXiv:1910.05620v1 [stat.ME] 12 Oct 2019 The fundamental purposes of the censuses are providing information to policy makers, planners, constructing statistical frames, etc. Thus an accurate and complete count of the population is important in censuses. Despite great efforts during the censuses, occurrence of errors such as coverage error is inevitable. Coverage error affects policy-making and planning issues of a country (see Freedman and Navidi (1992)).
    [Show full text]