DOCSLIB.ORG

Quality Guidelines for Frames Insocialstatistics

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Quality Guidelines for Frames Insocialstatistics Quality Guidelines for Frames in Social Statistics Version 1.51 ESSnet KOMUSO Quality in Multisource Statistics http://ec.europa.eu/eurostat/cros/content/essnet-quality-multisource-statistics_en Framework Partnership Agreement No 07112.2015.003-2015.226 Specific Grant Agreement No 3 (SGA-3) No 07112.2018.007-2018.0444 Work Package 2 QUALITY GUIDELINES FOR FRAMES IN SOCIAL STATISTICS Version 1.51, 2019-09-30 Prepared by: Thomas Burg (Statistics Austria), Alexander Kowarik (Statistics Austria), Magdalena Six (Statistics Austria), Giovanna Brancato (Istat, Italy) and Danuté Krapavickaité (LS, Lithuania) Reviewers: Fabian Bach (ESTAT), Lionel Vigino (ESTAT), Li-Chun Zhang (SSB, Norway), Ton de Waal (CBS, The Netherlands), Angela McCourt (CSO, Ireland), Eva-Maria Asamer (Statistics Austria) and Christoph Waldner (Statistics Austria) ESSnet co-ordinator: Niels Ploug (DST, Denmark), email [email protected], telephone +45 3917 3951 1 Quality Guidelines for Frames in Social Statistics Version 1.51 Table of contents 1 Purpose and Objectives of the Document ................................................................................................................ 3 2 Frames in Official Statistics ....................................................................................................................................... 5 2.1 Definition of a Frame ........................................................................................................................................ 5 2.2 Frames in Social Statistics ................................................................................................................................. 8 2.3 Processes involving Frames .............................................................................................................................. 9 3 Constructing and Maintaining Frames in Social Statistics ...................................................................................... 12 3.1 Sources for Constructing Frames .................................................................................................................... 12 3.2 Organization, support, planning and coordination ......................................................................................... 15 3.3 Methods for Constructing Frames .................................................................................................................. 19 3.4 Possible outputs when constructing frames in social statistics ...................................................................... 23 3.5 Updating Frames in social statistics ................................................................................................................ 25 4 Use of frames in social statistics ............................................................................................................................. 29 4.1 Sampling .......................................................................................................................................................... 29 4.1.1 Sampling Frames ..................................................................................................................................... 29 4.1.2 Contact variables ..................................................................................................................................... 34 4.2 Frames as input for processing ....................................................................................................................... 36 4.2.1 Frames supporting editing and imputation ............................................................................................ 36 4.2.2 Frames supporting weighting and calibration ........................................................................................ 38 4.3 Frames as input for statistical outputs ........................................................................................................... 40 4.3.1 Relevance ................................................................................................................................................ 40 4.3.2 Accuracy .................................................................................................................................................. 42 4.3.3 Timeliness and Punctuality ..................................................................................................................... 44 4.3.4 Accessibility and Clarity ........................................................................................................................... 46 4.3.5 Comparability .......................................................................................................................................... 48 4.3.6 Coherence ............................................................................................................................................... 50 5 Assessing and evaluation the quality of frames in social statistics ......................................................................... 53 5.1 Methods to assess the quality of a frame ....................................................................................................... 53 5.1.1 Quality assessment ................................................................................................................................. 53 5.1.2 Quality indicators .................................................................................................................................... 57 5.2 Quality and metadata management, quality improvement and quality reporting ........................................ 62 Annex I: Frame quality assessment: items, approaches and methods ........................................................................... 65 Annex II: References ....................................................................................................................................................... 69 Annex III: Requirements for frame contents .................................................................................................................. 71 Annex IV: Standardized questionnaire for metadata on frames data which are used by the social and population statistics .......................................................................................................................................................................... 73 Annex V: Glossary ........................................................................................................................................................... 90 2 Quality Guidelines for Frames in Social Statistics Version 1.51 1 Purpose and objectives of the document This document is an ESSnet KOMUSO deliverable1 under ESS.VIP Vision 2020.ADMIN, which is a project portfolio for achieving the goals of the European Statistical System (ESS) Vision 20202. Since the provision of guidelines is one of the key objectives set out in the ESS Vision 2020.ADMIN business case3, the main goal of this document is to provide guidelines for all processes relevant for frames in social statistics. In official statistics, the description of social phenomena by statistical figures is one of the main functions of national statistical systems and frames are essential to this function. In addition to this general objective, in order to help implement ESS Vision 2020, this document pursues three more specific objectives. The first specific objective concerns compliance with the Code of Practice. In the last decade of the last century, the question of quality in the production of official statistics was approached in a systematic way. By defining quality on a multidimensional basis and laying down quality dimensions in the EU Statistics Regulation (No 223/09), the concept for a framework, which was later enhanced by formulating and adopting the European Code of Practice (CoP) for official statistics, was established. Principle 4 of the CoP 'Commitment to Quality' – and specifically indicator 4.1 – states that 'Quality policy is defined and made available to the public. An organisational structure and tools are in place to deal with quality'. The Quality Assurance Framework (QAF) calls for the development of quality guidelines for relevant steps in the statistical process as one of the key methods for providing evidence of compliance with principle 4. This specific element, together with the stipulation of indicator 7.3 (' The registers and frames used for European Statistics are regularly evaluated and adjusted if necessary in order to ensure high quality.'), shows that the CoP demands the continuous development and use of good (or, if possible, best) practices for frames for population statistics, which must be expressed in the form of concrete guidelines. Accordingly, one of the main objectives of this document is to deliver a building block for safeguarding compliance with the Code of Practice in terms of the construction, use and assessment of frames in social statistics. A second specific objective of the guidelines is to provide producers of social statistics with systematic guidance for all process steps relevant for frames in social statistics. The procedures of National Statistical Institutes (NSIs) working with frames in social statistics seem to be more heterogeneous than the procedures for economic statistics, where the development and maintenance of the frame (which in most cases is equivalent to the business registers) were in some way generic for NSIs all over the world.
Load more

Recommended publications
  • Introduction to Social Statistics
    SOCY 3400: INTRODUCTION TO SOCIAL STATISTICS MWF 11-12:00; Lab PGH 492 (sec. 13748) M 2-4 or (sec. 13749)W 2-4 Professor: Jarron M. Saint Onge, Ph.D. Office: PGH 489 Phone: (713) 743-3962 Email: [email protected] Office Hours: MW 10-11 (Please email) or by appointment Teaching Assistant: TA Email: Office Hours: TTh 10-12 or by appointment Required Text: McLendon, M. K. 2004. Statistical Analysis in the Social Sciences. Additional materials will be distributed through WebCT COURSE DESCRIPTION: Sociological research relies on experience with both qualitative (e.g. interviews, participant observation) and quantitative methods (e.g., statistical analyses) to investigate social phenomena. This class focuses on learning quantitative methods for furthering our knowledge about the world around us. This class will help students in the social sciences to gain a basic understanding of statistics, whether to understand, critique, or conduct social research. The course is divided into three main sections: (1) Descriptive Statistics; (2) Inferential Statistics; and (3) Applied Techniques. Descriptive statistics will allow you to summarize and describe data. Inferential Statistics will allow you to make estimates about a population (e.g., this entire class) based on a sample (e.g., 10 or 12 students in the class). The third section of the course will help you understand and interpret commonly used social science techniques that will help you to understand sociological research. In this class, you will learn concepts associated with social statistics. You will learn to understand and grasp the concepts, rather than only focusing on getting the correct answers.
    [Show full text]
  • Esomar/Grbn Guideline for Online Sample Quality
    ESOMAR/GRBN GUIDELINE FOR ONLINE SAMPLE QUALITY ESOMAR GRBN ONLINE SAMPLE QUALITY GUIDELINE ESOMAR, the World Association for Social, Opinion and Market Research, is the essential organisation for encouraging, advancing and elevating market research: www.esomar.org. GRBN, the Global Research Business Network, connects 38 research associations and over 3500 research businesses on five continents: www.grbn.org. © 2015 ESOMAR and GRBN. Issued February 2015. This Guideline is drafted in English and the English text is the definitive version. The text may be copied, distributed and transmitted under the condition that appropriate attribution is made and the following notice is included “© 2015 ESOMAR and GRBN”. 2 ESOMAR GRBN ONLINE SAMPLE QUALITY GUIDELINE CONTENTS 1 INTRODUCTION AND SCOPE ................................................................................................... 4 2 DEFINITIONS .............................................................................................................................. 4 3 KEY REQUIREMENTS ................................................................................................................ 6 3.1 The claimed identity of each research participant should be validated. .................................................. 6 3.2 Providers must ensure that no research participant completes the same survey more than once ......... 8 3.3 Research participant engagement should be measured and reported on ............................................... 9 3.4 The identity and personal
    [Show full text]
  • THE HISTORY and DEVELOPMENT of STATISTICS in BELGIUM by Dr
    THE HISTORY AND DEVELOPMENT OF STATISTICS IN BELGIUM By Dr. Armand Julin Director-General of the Belgian Labor Bureau, Member of the International Statistical Institute Chapter I. Historical Survey A vigorous interest in statistical researches has been both created and facilitated in Belgium by her restricted terri- tory, very dense population, prosperous agriculture, and the variety and vitality of her manufacturing interests. Nor need it surprise us that the successive governments of Bel- gium have given statistics a prominent place in their affairs. Baron de Reiffenberg, who published a bibliography of the ancient statistics of Belgium,* has given a long list of docu- ments relating to the population, agriculture, industry, commerce, transportation facilities, finance, army, etc. It was, however, chiefly the Austrian government which in- creased the number of such investigations and reports. The royal archives are filled to overflowing with documents from that period of our history and their very over-abun- dance forms even for the historian a most diflScult task.f With the French domination (1794-1814), the interest for statistics did not diminish. Lucien Bonaparte, Minister of the Interior from 1799-1800, organized in France the first Bureau of Statistics, while his successor, Chaptal, undertook to compile the statistics of the departments. As far as Belgium is concerned, there were published in Paris seven statistical memoirs prepared under the direction of the prefects. An eighth issue was not finished and a ninth one * Nouveaux mimoires de I'Acadimie royale des sciences et belles lettres de Bruxelles, t. VII. t The Archives of the kingdom and the catalogue of the van Hulthem library, preserved in the Biblioth^que Royale at Brussells, offer valuable information on this head.
    [Show full text]
  • SAMPLING DESIGN & WEIGHTING in the Original
    Appendix A 2096 APPENDIX A: SAMPLING DESIGN & WEIGHTING In the original National Science Foundation grant, support was given for a modified probability sample. Samples for the 1972 through 1974 surveys followed this design. This modified probability design, described below, introduces the quota element at the block level. The NSF renewal grant, awarded for the 1975-1977 surveys, provided funds for a full probability sample design, a design which is acknowledged to be superior. Thus, having the wherewithal to shift to a full probability sample with predesignated respondents, the 1975 and 1976 studies were conducted with a transitional sample design, viz., one-half full probability and one-half block quota. The sample was divided into two parts for several reasons: 1) to provide data for possibly interesting methodological comparisons; and 2) on the chance that there are some differences over time, that it would be possible to assign these differences to either shifts in sample designs, or changes in response patterns. For example, if the percentage of respondents who indicated that they were "very happy" increased by 10 percent between 1974 and 1976, it would be possible to determine whether it was due to changes in sample design, or an actual increase in happiness. There is considerable controversy and ambiguity about the merits of these two samples. Text book tests of significance assume full rather than modified probability samples, and simple random rather than clustered random samples. In general, the question of what to do with a mixture of samples is no easier solved than the question of what to do with the "pure" types.
    [Show full text]
  • Precept 8: Some Review, Heteroskedasticity, and Causal Inference Soc 500: Applied Social Statistics
    Precept 8: Some review, heteroskedasticity, and causal inference Soc 500: Applied Social Statistics Alex Kindel Princeton University November 15, 2018 Alex Kindel (Princeton) Precept 8 November 15, 2018 1 / 27 Learning Objectives 1 Review 1 Calculating error variance 2 Interaction terms (common support, main effects) 3 Model interpretation ("increase", "intuitively") 4 Heteroskedasticity 2 Causal inference with potential outcomes 0Thanks to Ian Lundberg and Xinyi Duan for material and ideas. Alex Kindel (Princeton) Precept 8 November 15, 2018 2 / 27 Calculating error variance We have some data: Y, X, Z. We think the correct model is Y = X + Z + u. We estimate this conditional expectation using OLS: Y = β0 + β1X + β2Z We want to know the standard error of β1. Standard error of β1 r ^2 ^ 1 σu 2 2 SE(βj ) = 2 Pn 2 , where Rj is the R of a regression of 1−Rj i=1(xij −x¯j ) variable j on all others. ^2 Question: What is σu? Alex Kindel (Princeton) Precept 8 November 15, 2018 3 / 27 Calculating error variance P 2 ^2 i u^i σu = DFresid You can adjust this in finite samples by u¯^ (why?) Alex Kindel (Princeton) Precept 8 November 15, 2018 4 / 27 Interaction terms Y = β0 + β1X + β2Z + β3XZ Assume X ∼ N (?; ?) and Z 2 f0; 1g Alex Kindel (Princeton) Precept 8 November 15, 2018 5 / 27 Interaction terms Y = β0 + β1X + β2Z + β3XZ Scenario 1 When Z = 0, X ∼ N (3; 4) When Z = 1, X ∼ N (−3; 2) Do you think an interaction term is justified here? Alex Kindel (Princeton) Precept 8 November 15, 2018 6 / 27 Interaction terms Y = β0 + β1X + β2Z + β3XZ Scenario
    [Show full text]
  • User Guide April 2004 EPA/600/R04/079 April 2004
    ProUCL Version 3.0 User Guide April 2004 EPA/600/R04/079 April 2004 ProUCL Version 3.0 User Guide by Anita Singh Lockheed Martin Environmental Services 1050 E. Flamingo Road, Suite E120, Las Vegas, NV 89119 Ashok K. Singh Department of Mathematical Sciences University of Nevada, Las Vegas, NV 89154 Robert W. Maichle Lockheed Martin Environmental Services 1050 E. Flamingo Road, Suite E120, Las Vegas, NV 89119 i Table of Contents Authors ..................................................................... i Table of Contents ............................................................. ii Disclaimer ................................................................. vii Executive Summary ........................................................ viii Introduction ................................................................ ix Installation Instructions ........................................................ 1 Minimum Hardware Requirements ............................................... 1 A. ProUCL Menu Structure .................................................... 2 1. File ............................................................... 3 2. View .............................................................. 4 3. Help .............................................................. 5 B. ProUCL Components ....................................................... 6 1. File ................................................................7 a. Input File Format ...............................................9 b. Result of Opening an Input Data File
    [Show full text]
  • A Meta-Analysis Examining the Impact of Computer-Assisted Instruction on Postsecondary Statistics Education: 40 Years of Research JRTE | Vol
    A Meta-Analysis Examining the Impact of Computer-Assisted Instruction on Postsecondary Statistics Education: 40 Years of Research JRTE | Vol. 43, No. 3, pp. 253–278 | ©2011 ISTE | iste.org A Meta-Analysis Examining the Impact of Computer-Assisted Instruction on Postsecondary Statistics Education: 40 Years of Research Karen Larwin Youngstown State University David Larwin Kent State University at Salem Abstract The present meta-analysis is a comprehensive investigation of the effectiveness of computer-assisted instruction (CAI) on student achievement in postsec- ondary statistics education across a forty year period of time. The researchers calculated an overall effect size of 0.566 from 70 studies, for a total of 219 effect-size measures from a sample of n = 40,125 participants. These results suggest that the typical student moved from the 50th percentile to the 73rd percentile when technology was used as part of the curriculum. This study demonstrates that subcategories can further the understanding of how the use of CAI in statistics education might be maximized. The study discusses im- plications and limitations. (Keywords: statistics education, computer-assisted instruction, meta-analysis) iscovering how students learn most effectively is one of the major goals of research in education. During the last 30 years, many re- Dsearchers and educators have called for reform in the area of statistics education in an effort to more successfully reach the growing population of students, across an expansive variety of disciplines, who are required to complete coursework in statistics (e.g., Cobb, 1993, 2007; Garfield, 1993, 1995, 2002; Giraud, 1997; Hogg, 1991; Lindsay, Kettering, & Siegmund, 2004; Moore, 1997; Roiter, & Petocz, 1996; Snee, 1993;Yilmaz, 1996).
    [Show full text]
  • 5.1 Survey Frame Methodology
    Regional Course on Statistical Business Registers: Data sources, maintenance and quality assurance Perak, Malaysia 21-25 May, 2018 4 .1 Survey fram e methodology REVIEW For sampling purposes, a snapshot of the live register at a particular point in t im e is needed. The collect ion of active statistical units in the snapshot is referred to as a frozen frame. REVIEW A sampling frame for a survey is a subset of t he frozen fram e t hat includes units and characteristics needed for t he survey. A single frozen fram e should be used for all surveys in a given reference period Creat ing sam pling fram es SPECIFICATIONS Three m ain t hings need t o be specified t o draw appropriate sampling frames: ▸ Target population (which units?) ▸ Variables of interest ▸ Reference period CHOICE OF STATISTICAL UNIT Financial data Production data Regional data Ent erpri ses are Establishments or Establishments or typically the most kind-of-activity local units should be appropriate units to units are typically used if regional use for financial data. the most appropriate disaggregation is for production data. necessary. Typically a single t ype of unit is used for each survey, but t here are except ions where t arget populat ions include m ult iple unit t ypes. CHOICE OF STATISTICAL UNIT Enterprise groups are useful for financial analyses and for studying company strategies, but they are not normally the target populations for surveys because t hey are t oo diverse and unstable. SURVEYS OF EMPLOYMENT The sam pling fram es for t hese include all active units that are em ployers.
    [Show full text]
  • Report on Exact and Statistical Matching Techniques
    Statistical Policy Working Papers are a series of technical documents prepared under the auspices of the Office of Federal Statistical Policy and Standards. These documents are the product of working groups or task forces, as noted in the Preface to each report. These Statistical Policy Working Papers are published for the purpose of encouraging further discussion of the technical issues and to stimulate policy actions which flow from the technical findings and recommendations. Readers of Statistical Policy Working Papers are encouraged to communicate directly with the Office of Federal Statistical Policy and Standards with additional views, suggestions, or technical concerns. Office of Joseph W. Duncan Federal Statistical Director Policy Standards For sale by the Superintendent of Documents, U.S. Government Printing Office Washington, D.C. 20402 Statistical Policy Working Paper 5 Report on Exact and Statistical Matching Techniques Prepared by Subcommittee on Matching Techniques Federal Committee on Statistical Methodology DEPARTMENT OF COMMERCE UNITED STATES OF AMERICA U.S. DEPARTMENT OF COMMERCE Philip M. Klutznick Courtenay M. Slater, Chief Economist Office of Federal Statistical Policy and Standards Joseph W. Duncan, Director Issued: June 1980 Office of Federal Statistical Policy and Standards Joseph W. Duncan, Director Katherine K. Wallman, Deputy Director, Social Statistics Gaylord E. Worden, Deputy Director, Economic Statistics Maria E. Gonzalez, Chairperson, Federal Committee on Statistical Methodology Preface This working paper was prepared by the Subcommittee on Matching Techniques, Federal Committee on Statistical Methodology. The Subcommittee was chaired by Daniel B. Radner, Office of Research and Statistics, Social Security Administration, Department of Health and Human Services. Members of the Subcommittee include Rich Allen, Economics, Statistics, and Cooperatives Service (USDA); Thomas B.
    [Show full text]
  • Committee on National Statistics
    January 20, 2010 News from the Committee on National Statistics PEOPLE NEWS >We note with great sadness the untimely death on December 17, 2010, from complications of cancer, of Dr. Phyllis Kaniss, executive director of the American Academy of Political and Social Science (AAPSS) and a longtime teaching faculty member at the Annenberg School for Communication at the University of Pennsylvania (see http://www.asc.upenn.edu/News/NewsDetail.aspx?nid=816&ntype=faculty). She was the author of Making Local News (University of Chicago Press, 1991) and The Media and the Mayor’s Race: The Failure of Urban Political Reporting (Indiana University Press, 1995), which won the 1995 Bart Richards Award for media criticism. In 1999, she created the Student Voices Project, a youth civic engagement initiative of the Annenberg Public Policy Center that worked with school systems in cities throughout the country. Dr. Kaniss received a B.A. degree from the University of Pennsylvania and a Ph.D. in regional science from Cornell University. Her connection with CNSTAT is that she worked tirelessly and enthusiastically with our staff and members to organize the very successful joint CNSTAT-AAPSS Symposium on the Federal Statistical System—Recognizing Its Contributions; Moving It Forward that was held at the National Academies on May 8, 2009, and resulted in a special volume of the Annals of the AAPSS, edited by Ken Prewitt, ―The Federal Statistical System: Its Vulnerability Matters More than You Think‖ (http://www.sagepub.com/books/Book235999). We will miss Phyllis’s good cheer and extraordinary skills in furthering the use of social science to address important social problems.
    [Show full text]
  • Basic Statistical Concepts Statistical Population
    Basic Statistical Concepts Statistical Population • The entire underlying set of observations from which samples are drawn. – Philosophical meaning: all observations that could ever be taken for range of inference • e.g. all barnacle populations that have ever existed, that exist or that will exist – Practical meaning: all observations within a reasonable range of inference • e.g. barnacle populations on that stretch of coast 1 Statistical Sample •A representative subset of a population. – What counts as being representative • Unbiased and hopefully precise Strategies • Define survey objectives: what is the goal of survey or experiment? What are your hypotheses? • Define population parameters to estimate (e.g. number of individuals, growth, color etc). • Implement sampling strategy – measure every individual (think of implications in terms of cost, time, practicality especially if destructive) – measure a representative portion of the population (a sample) 2 Sampling • Goal: – Every unit and combination of units in the population (of interest) has an equal chance of selection. • This is a fundamental assumption in all estimation procedures •How: – Many ways if underlying distribution is not uniform » In the absence of information about underlying distribution the only safe strategy is random sampling » Costs: sometimes difficult, and may lead to its own source of bias (if sample size is low). Much more about this later Sampling Objectives • To obtain an unbiased estimate of a population mean • To assess the precision of the estimate (i.e.
    [Show full text]
  • Methodology, Survey, Statistics, Demography, Economics, History, Political Science, Sociology, Information and Communication
    EQUIPEX Acronym CALL FOR PROPOSALS DIME-SHS 2010 SCIENTIFIC SUBMISSION FORM B Acronym of the project DIME-SHS Titre du projet en Données, Infrastructure, Méthodes d'Enquêtes en Sciences français humaines et sociales Data, Infrastructure, Methods of investigation in the social sciences Project title in English and humanities Name: Laurent Lesnard Institution: Sciences Po Coordinator of the Laboratory: CDSP – Centre de données socio-politiques / Centre for project socio-political data Unit number: UMS 828 Tranche 1/Phase 1 Tranche 2/Phase 2 Requested funding 16 811 197 € 552 407 € □ health, well-being, nutrition and biotechnologies □ environmental urgency, eco-technologies Disciplinary field □ information, communication and nanotechnologies social sciences and humanities □ other disciplinary area Methodology, survey, statistics, demography, economics, Scientific areas history, political science, sociology, information and communication Organization of the coordinating partner Laboratory/Institution(s) Unit number Research organization CDSP / Sciences Po UMS 828 Sciences Po / CNRS Affiliations des partenaires au projet/Organization of the partner(s) Laboratory/Institution(s) Unit number Research organization GENES SES / Ined Université Paris CERLIS / Université Paris Descartes UMR 8070 Descartes/CNRS/Université Paris 3 Telecom ParisTech UMR 5141 Telecom ParisTech/CNRS CNRS/EHESS/INED/ Université GIS Réseau Quetelet de Caen Company Staff size Economic sector EDF R&D Energy 2000 1/66 EQUIPEX Acronym CALL FOR PROPOSALS DIME-SHS 2010 SCIENTIFIC SUBMISSION FORM B 1. SCIENTIFIC ENVIRONMENT AND POSITIONING OF THE EQUIPMENT PROJECT .......... 5 2. TECHNICAL AND SCIENTIFIC DESCRIPTION OF THE ACTIVITIES ......................... 9 2.1. Originality and innovative features of the equipment project..............................9 2.2. Description of the project....................................................................................11 2.2.1 Scientific programme 11 2.2.2 Structure and building of the equipment 16 2.2.3 Technical environment 18 3.
    [Show full text]