Observing Census Enumeration of Non-English Speaking Households in the 2010 Census: Evaluation Report
Total Page:16
File Type:pdf, Size:1020Kb
This document was prepared by and for Census Bureau staff to aid in future research and planning, but the Census Bureau is making the document publicly available in order to share the information with as wide an audience as possible. Questions about the document should be directed to Kevin Deardorff at (301) 763-6033 or [email protected] December 12, 2012 2010 CENSUS PLANNING MEMORANDA SERIES No. 249 MEMORANDUM FOR The Distribution List From: Burton Reist [signed] Acting Chief, Decennial Management Division Subject: Observing Census Enumeration of Non-English Speaking Households in the 2010 Census: Evaluation Report Attached is the Observing Census Enumeration of Non-English Speaking Households in the 2010 Census: Evaluation Report. The Quality Process for the 2010 Census Evaluations, Experiments, and Assessments was applied to the methodology development, specifications, software development, analysis, and documentation of the analysis and results, as necessary. If you have questions about this report, please contact Yuling Pan at (301) 763-4950 or Stephen Lubkemann at (301) 763-8601. Attachment 2010 Census Program for Evaluations and Experiments December 6, 2012 Observing Census Enumeration of Non- English Speaking Households in the 2010 Census: Evaluation Report U.S. Census Bureau standards and quality process procedures were applied throughout the creation of this report Yuling Pan and Stephen Lubkemann Center for Survey Measurement This page intentionally left blank. ii Table of Contents List of Tables ................................................................................................................................................ v EXECUTIVE SUMMARY ......................................................................................................................... vi 1 INTRODUCTION ................................................................................................................................ 2 2 BACKGROUND .................................................................................................................................. 2 2.1 Theoretical approach ..................................................................................................................... 4 2.2 Research questions ........................................................................................................................ 7 3 METHODOLOGY ............................................................................................................................... 8 3.1 Study design .................................................................................................................................. 8 3.2 Selection of languages .................................................................................................................. 9 3.3 Research teams ............................................................................................................................ 10 3.4 Research sites .............................................................................................................................. 12 3.5 Steps and focus of the observation .............................................................................................. 12 3.6 Description of fieldwork ............................................................................................................. 16 3.7 Analysis methods ........................................................................................................................ 18 4 LIMITATIONS ................................................................................................................................... 19 4.1 Navigating multiple layers of Census Bureau organization: Timing effects .............................. 20 4.2 Interaction effects ........................................................................................................................ 21 4.3 Locating limited English proficient respondents ........................................................................ 21 4.4 Limited number of cases observed: Non-representativeness ...................................................... 22 4.5 Comparison and evidence ........................................................................................................... 22 5 RESULTS ........................................................................................................................................... 23 5.1 Negotiating and maintaining access to respondents .................................................................... 23 5.2 Forms of non-equivalency: Conceptual and meta-communicative ............................................. 52 5.3 The use and effectiveness of in-language materials in NRFU interviews .................................. 70 5.4 On-the-fly translation and use of interpreters ............................................................................. 76 5.5 Non-English speaking respondents’ perception of and reactions to the 2010 Census ................ 82 6 RELATED EVALUATIONS, EXPERIMENTS, AND/OR ASSESSMENTS .................................. 90 7 KEY LESSONS LEARNED, CONCLUSIONS, AND RECOMMENDATIONS ............................ 91 8 ACKNOWLEDGEMENTS .............................................................................................................. 105 9 REFERENCES ................................................................................................................................. 105 10 APPENDICES .............................................................................................................................. 115 iii 10.1 Appendix A: Observation protocol guide ................................................................................. 115 10.2 Appendix B: Enumerator debriefing questions ......................................................................... 119 10.3 Appendix C: Respondent debriefing questions ......................................................................... 121 10.4 Appendix D: The 2010 Census NRFU Questionnaire ............................................................. 124 10.5 Appendix E: The 2010 Census NRFU Information Sheet ........................................................ 127 iv List of Tables Table 1. Language teams and research sites ............................................................................................... 12 Table 2. Number of interviews observed and enumerators accompanied in field work ............................. 17 Table 3. Enumerators’ Conversational Strategies to Obtain Interview (English Households) ................... 25 Table 4. Signaling shared social identity .................................................................................................... 36 Table 5. Communicative conventions often deployed by bilingual enumerators in each language group . 46 Table 6. Fears that tended to be raised in particular groups ........................................................................ 90 v EXECUTIVE SUMMARY This study was conducted during the 2010 Census to observe Nonresponse Followup interviews with households that speak a language other than English, in areas of the U.S. with heavy concentrations of residents with limited English proficiency. A multilingual research team consisting of seven sub-teams in the seven primary languages (Arabic, Chinese, Korean, Portuguese, Russian, Spanish, and Vietnamese) was commissioned to carry out the research in the 2010 Census. The objectives of this research were to collect extensive qualitative data that could provide an empirical basis for beginning to: (1) identify which linguistic and sociocultural factors appear to affect the enumeration of non-English speaking populations during the Nonresponse Followup interview process; (2) examine the measures that were taken by enumerators to negotiate access to non-English speaking households and to collect the required census data from these households; (3) observe if and how in-language census materials were used in the field; and (4) gain additional insight into how non-English speaking immigrant populations perceived and reacted to the census and its public messaging. Findings from this study will help develop recommendations for the 2020 Census planning process, by identifying issues that should be addressed in order to improve the enumeration process in that subset of the population that has limited, or entirely lacks, English proficiency. Most specifically, the findings in this report may be potentially relevant to further refining the procedures and approaches used to translate census questions; to the re-design of questionnaires and interview protocols for non-English speaking respondents; to developing policies for the use of interpreters in enumeration interviews; and to the development of interviewer training. This report describes and compares the findings from a set of field studies that were designed to collect comparable data in seven distinct communities of language through ethnographically- informed direct observation and qualitative interviews. The primary data in these field studies were collected through the direct observation of 586 Nonresponse Followup interviews conducted among Arabic, Chinese, Korean, Portuguese, Russian, Spanish, and Vietnamese speakers. Following these initial observations, two additional debriefing interviews were conducted: first, with the respondents, and subsequently, with their enumerators. In each