National Statistical Programs: Directions and Challenges
Total Page:16
File Type:pdf, Size:1020Kb
National Statistical Programs: Directions and Challenges Brian Harris-Kojetin, Ph.D. Director, CNSTAT Washington, DC • August 9, 2018 The Committee on National Statistics • Established in 1972 as a standing unit of the National Academies on the recommendation of the President’s Commission on Federal Statistics to provide an independent, objective resource for evaluation and improvement of federal statistical methods and operations. • There are about 50 standing units like CNSTAT in the Academies. 2 What is the Committee on National Statistics? • CNSTAT’s mission is to improve the statistical methods and information on which public policy decisions are based. It also serves as a coordinating force in the highly decentralized U.S. federal statistical system. • Over its 46-year history, CNSTAT has produced over 270 consensus, interim, letter, and workshop reports. • Every October and May, CNSTAT holds a public seminar on a topic of broad interest to the federal statistical and research communities and has a substantive luncheon with heads of major statistical agencies. 3 Who Serves on CNSTAT? Robert Groves (chair), Math/Statistics Sarah Nusser, Survey Research Mary Ellen Bock, Math/Statistics Colm O’Muircheartaigh, Survey Research Anne C. Case, Health Economics Jerome Reiter, Applied Statistics Michael Chernew, Health Economics Roberto Rigobon, Economic Statistics, Janet Currie, Welfare Economics Big Data Don Dillman, Survey Research Judith Seltzer, Sociology Tom Mesenbourg, Sr. Federal Statistics Management 4 Note: all members serve pro bono The CNSTAT Portfolio • Coordinating and Sustaining Federal Statistics • Decennial Census Coverage and Quality and the American Community Survey • Economic Measurement • Federal Household and Business Surveys • Health and Social Welfare • Science, Technology, and Innovation Indicators • Statistical Methods and Estimates for Policy Use 5 Recent Projects with TRB 6 The CNSTAT Portfolio Principles and Practices for a Federal Statistical Agency – Now in its 6th edition (released July 2017); emphasizes statistical agency independence to make possible the production of relevant, high-quality, timely, accessible, and nonpartisan statistics. 7 Current Challenges and Opportunities in Federal Statistics The way that statistics are currently produced by Federal statistical agencies faces threats from declining participation rates and increasing costs. • Although generally higher than other surveys, federal statistical surveys face increasing nonresponse and increased costs of data collection to maintain response rates • Agency budgets have decreased or remained flat • Agencies face increasing demands for more timely and more geographically detailed information • Increasingly alternative data sources are available that offer the potential of faster and more detailed information 8 Response Rates for Three Surveys in which at least One Interview is Done in Person 9 Census Bureau Economic Directorate Survey Unit Response Rates 2009-2014 90.0% 85.0% 80.0% 75.0% 70.0% 65.0% 60.0% 55.0% 50.0% 45.0% 40.0% 35.0% 30.0% 2009 2010 2011 2012 2013 2014 Report of Organization Annual Wholesale Trade Survey Annual Survey of Manufactures4 Annual Retail Trade Survey Service Annual Survey Manufacturers' Shipments, Inventories and Orders1 10 Montly Wholesale Trade1 Quarterly Financial Report - Manufacturing, Mining, Wholesale Trade, and Selected Service Industries1 Quarterly Financial Report - Retail Trade1 Annual Capital Expenditure Survey Montly Retail Trade Survey1 Quarterly Services Survey1 Overview of U.S. Federal Statistical System • The U.S. decentralized system includes about 100 agencies with an FY 2018 budget of about $6.5 billion (excluding decennial census) – About $2 billion contracted to private sector or states • A substantial portion of official statistics are produced by 13 principal statistical agencies. – Account for approximately 40 percent of resources dedicated to federal statistics 11 Federal Statistical System Decentralized 12 Statistics Account for 0.15% to 0.45% of the Federal Budget (U.S. Government Expenditures on Statistical Programs, 1990-2017, including Decennial Census, as Percent of Total Federal Outlays) 0.5 0.45 0.4 0.35 0.3 0.25 0.2 Percentage 0.15 0.1 0.05 0 Nominal dollars13 Decreasing and Flatlining Budgets Change in Agency Budget as % of Total Budget 30.00% 25.00% 20.00% 15.00% 10.00% 5.00% Δ2016-2017 0.00% Δ2017-2018 % % Change in Budget -5.00% -10.00% -15.00% -20.00% BEA BJS BLS BTS Census ERS EIA NASS NCES NCHS NCSES SSA SOI Agency Note: Census Bureau figures reflect ongoing programs, not periodic programs. Decrease in Labor Statistics Budget Priority Issues for the Future of U.S. Federal Statistics • New data sources— – How can agencies use administrative records and private-sector data sources to enhance federal statistics? – What privacy issues need to be addressed when combining data sources? – What quality frameworks and metrics are appropriate for these new sources and for blended estimates? – How can agencies collaborate to access, evaluate, and use these new data sources? 16 Statement of Task An ad hoc panel of nationally renowned experts in social science research, computing technology, statistical methods, privacy, and use of alternative data sources in the United States and abroad will conduct a study with the goal of fostering a paradigm shift in federal statistical programs. In place of the current paradigm of providing users with the output from a single census, survey, or administrative records source, a new paradigm would use combinations of diverse data sources from government and private sector sources combined with state-of-the art methods to give users richer and more reliable statistics leading to new insights about policy and socioeconomic behavior. The motivation for the study stems from the increasing challenges to the current paradigm, such as declining response rates and increasing cost and burden for surveys. The panel will 17 prepare two reports as part of this study. Panel on Improving Federal Statistics for Policy and Social Science Research Using Multiple Data Sources and State-of-the-Art Estimation Methods • Robert M. Groves, (Chair), • Sharon Lohr, Westat, Inc. Georgetown University • James P. Lynch, University of • Michael E. Chernew, Harvard Maryland University • Colm O’Muircheartaigh, • Piet Daas, Statistics Netherlands University of Chicago • Cynthia Dwork, Harvard • Trivellore Raghunathan, University University of Michigan • Ophir Frieder, Georgetown • Roberto Rigobon, University Massachusetts Institute of • Hosagrahar V. Jagadish, Technology University of Michigan 18 • • Frauke Kreuter, University of Marc Rotenberg, Electronic Maryland Privacy Information Center Acknowledgements Funding for the panel was provided by The Laura and John Arnold Foundation, with additional support from the National Academy of Sciences Kellogg Fund. 19 The Panel’s First Report: 20 Contents • Chapter 1: Introduction • Chapter 2: Current Challenges and Opportunities in Federal Statistics • Chapter 3: Using Government Administrative and Other Data for Federal Statistics • Chapter 4: Using Private-Sector Data For Federal Statistics • Chapter 5: Protecting Privacy and Confidentiality While Providing Access to Data for Research Use • Chapter 6: Advancing the Paradigm of Combining Data 21 Sources Using Government Administrative and Other Data for Federal Statistics • Recommendation 3-1 Federal statistical agencies should systematically review their statistical portfolios and evaluate the potential benefits and risks of using administrative data. To this end, federal statistical agencies should create collaborative research programs to address the many challenges in using administrative data for federal statistics. 22 Structured Data: Structured Data: Other Structured Semistructured Data Unstructured Data Censuses and Administrative Records Data Probability Surveys Collecting data from Data collected by Data that are Data that have structure, but also Data, such as in text, the universe or a government entities highly organized permit flexibility in structure so images, and videos, that sample of that for program and can easily be that they cannot be placed in a do not have any pre- population and administration, placed in a relational database or defined structure. estimating their regulatory, or law database or spreadsheet. Transformation Information of value characteristics spreadsheet. into structured form requires must first be extracted enforcement purpose through the They may still decisions with regard to the way from such data, after systematic use of require in which to standardize the which the extracted statistical substantial observed variety in structure. information can be methodology scrubbing and The scrubbing and placed in a structured transformation transformation for modeling and table for further for modeling and analysis is usually more difficult processing and analysis analysis than for structured data Decennial Federal Records Weather Web-scraped quantitative data Traffic videos Census of o Income Tax sensors Web logs Population o Social Satellite images Traffic and Housing Security Blogs and o sensors Economic Unemployme comments Census nt Water o Agriculture Medical quality Input in free text Census Records sensors fields Federal State Records statistical o SNAP data surveys Police accident 23 o ACS reports o CPS County Records o NHIS Other jurisdictions’ o AHS data o NCVS Current Barriers to Use of Alternative Data Sources