2010 ACS Content Test Evaluation Report Covering Parental Place of Birth
Total Page:16
File Type:pdf, Size:1020Kb
January 26, 2012 2012 AMERICAN COMMUNITY SURVEY RESEARCH AND EVALUATION REPORT MEMORANDUM SERIES #ACS12-RER-04 MEMORANDUM FOR ACS Research and Evaluation Steering Committee From: Enrique Lamas /Signed/ Chief, Population Division Prepared by: Luke J. Larsen, Elizabeth M. Grieco, and Patricia de la Cruz Foreign-born Population Branch Population Division Subject: 2010 ACS Content Test Evaluation Report Covering Parental Place of Birth Attached is the final American Community Survey Research and Evaluation report for the 2010 ACS Content Test Evaluation Covering Parental Place of Birth. This report describes the results of testing the parental place of birth questions in two locations on the ACS questionnaire. If you have any questions about this report, please contact Elizabeth Grieco at (301)763-5275, Luke Larsen at (301)763-2540, or Patricia de la Cruz at (301) 763-2377. Attachment: (2010 ACS Content Test Evaluation Report Covering Parental Place of Birth) cc: ACS Research and Evaluation Steering Committee ACS Research and Evaluation Team Donna Daily (ACSO) Todd Hughes Debbie Klein David Raglin Yuling Pan (CSM) Jennifer Tancreto (DSSD) Tony Tersine Karen Humes (POP) Victoria Velkoff Kurt Bauman (SEHSD) American Community Survey Research and Evaluation Program January 31, 2012 2010 ACS Content Test Evaluation Report Covering Parental Place of Birth FINAL REPORT Luke J. Larsen Elizabeth M. Grieco Patricia de la Cruz Population Division Intentionally Blank TABLE OF CONTENTS EXECUTIVE SUMMARY ....................................................................................................................... iv 1. BACKGROUND 1.1 Motivation for the 2010 Content Test ............................................................................................................................. 1 1.2 Previous Testing or Analysis .......................................................................................................................................... 2 1.3 Recommendations from Cognitive Testing .................................................................................................................... 2 1.4 Recommendations from the Expert Review Panel ......................................................................................................... 4 2. SELECTION CRITERIA 2.1 Critical Research Questions ............................................................................................................................................ 5 2.2 Supplementary Research Questions ................................................................................................................................ 6 2.3 Additional Research Topics ............................................................................................................................................ 6 3. METHODOLOGY 3.1 Data Collection Methods ................................................................................................................................................ 7 3.2 Sample Design ................................................................................................................................................................ 8 3.3 Methodology Specific to Parental Place of Birth 3.3.1 Operational Definitions and Recodes ................................................................................................................... 9 3.3.2 Population Universes .......................................................................................................................................... 11 3.3.3 Analysis Framework and Statistical Testing Procedures .................................................................................... 12 4. LIMITATIONS ......................................................................................................................................13 5. CRITICAL RESEARCH QUESTIONS AND RESULTS 5.1 Response to the Content Test and Content Follow-up .................................................................................................. 14 5.2 Are the response distributions of parental place of birth and generational status comparable to existing data sources? ......................................................................................................................................................................... 14 5.3 Which placement results in a lower missing data rate? ................................................................................................ 15 5.4 Do the two placements have similar or different response distributions? ..................................................................... 16 5.5 Which placement results in more reliable estimates? ................................................................................................... 17 5.6 Does changing the placement of the parental place of birth questions from before to directly after the ancestry question affect the item missing data rates, response distributions, or reliability for the ancestry question? ................ 18 5.7 Does the placement of the parental place of birth questions directly before the school enrollment question affect the item missing data rate, response distribution, or reliability for the school enrollment question? ............................ 20 5.8 Does the placement of the parental place of birth questions directly before the language spoken questions affect the item missing data rates, response distributions, or reliability for the language spoken questions? ......................... 21 5.9 Summary of Critical Analysis ...................................................................................................................................... 22 6. SUPPLEMENTAL RESEARCH QUESTIONS AND RESULTS 6.1 For each mode of data collection, do the two placements have differential item missing data rates, response distributions, or reliability of the data? .......................................................................................................................... 23 6.2 For each mail response stratum, do the two placements have differential item missing data rates, response distributions, or reliability of the data? .......................................................................................................................... 25 6.3 Does either placement elicit respondent or interviewer behaviors that may contribute to interviewer or respondent error? ........................................................................................................................................................... 26 6.4 Summary of Supplemental Analysis ............................................................................................................................. 27 7. ADDITIONAL RESEARCH TOPICS AND RESULTS 7.1 Comparison of Content Test Data to ACS Data for Proximal Variables ...................................................................... 27 7.2 Additional Analysis of Proximal Variables by Mode in Content Test Data ................................................................. 29 7.3 Summary of Additional Research ................................................................................................................................. 30 8. SUMMARY ...........................................................................................................................................32 References ..................................................................................................................................................34 TABLE OF APPENDICES Appendix A: Tables .............................................................................................................................. A-1 Appendix B: Images of the Mail Versions of the Control and Test Questions .....................................B-1 Appendix C: CATI and CAPI Versions of the Control and Test Questions .........................................C-1 Appendix D: Spanish Language Versions of the Control and Test Questions (CATI/CAPI Modes) ...................................................................................................... D-1 Appendix E: Process Flowchart of the Content Follow-Up Survey Reinterview ................................. E-1 Appendix F: Parental Place of Birth Crosswalk for World Region of Birth Recode using the American Community Survey .......................................................................... F-1 Appendix G: Parental Place of Birth Crosswalk for World Region of Birth Recode using the Current Population Survey .............................................................................. G-1 Appendix H: Parental Place of Birth Crosswalk for Broad Place of Birth Recode using the American Community Survey ......................................................................... H-1 Appendix I: Parental Place of Birth Crosswalk for the Selected Place of Birth Recodes using the American Community Survey ........................................................................... I-1 Appendix J: Ancestry Crosswalk for Regional Ancestry Recode using the American Community Survey ........................................................................................................... J-1 Appendix K: Ancestry Crosswalk for Selected Ancestry Recode using the American Community Survey ......................................................................................................... K-1 Appendix L: Language Crosswalk for Selected