Methodology Report and User's Guide for the 2008-2009 ANES Panel Study
Total Page:16
File Type:pdf, Size:1020Kb
Methodology Report and User’s Guide for the 2008–2009 ANES Panel Study August 2010 (Revised September 22, 2010) (Revised December 13, 2010) Matthew DeBell Jon A. Krosnick Arthur Lupia American National Election Studies Stanford University and the University of Michigan This document may be freely copied or redistributed, provided it is not altered. Suggested Citation Matthew DeBell, Jon A. Krosnick, and Arthur Lupia. 2010. Methodology Report and User’s Guide for the 2008–2009 ANES Panel Study. Palo Alto, CA, and Ann Arbor, MI: Stanford University and the University of Michigan. Acknowledgments This material is based upon work supported by the National Science Foundation under Grant Nos. SES- 0535332 and SES-0535334. ANES is also supported by Stanford University and the University of Michigan. Sections of this report may reprint previous documentation of the American National Election Studies without explicit attribution. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation, Stanford University, or the University of Michigan. The ANES Principal Investigators for the Panel Study were Jon A. Krosnick at Stanford University and Arthur Lupia at the University of Michigan. Vincent Hutchings at the University of Michigan was Associate Principal Investigator. The Panel Study director was Matthew DeBell. ANES staff on the study were Caroline Roberts, Jaime Ventura, and Jonathan Cowden, with additional support from Darrell Donakowski, Catherine Wilson, Pat Luevano, Heather Branton, Laurie Pierson, and David Howell. Research assistance at Stanford was provided by Wendy Gross, Farah Giga, Gaurav Sood, Lucila Figueroa, Mackenzie Israel-Trummel, Yo-Yo Chen, Helen Chen, Troy Wu, Jade Wang, Logan Ensign, Peter Tu, Abiy Teshome, and Mario Suntanu, with some additional ad hoc support from Autumn Carter, Mai El-Sadany, Lucinda Gibbs, Natasha Kim, Andy Nguyen, Deborah Oh, Noelle Spring, and Elaine Albertson. Research assistance at the University of Michigan was provided by Mi Hwa Hong, Yanna Krupnikov, Adam Levine, Spencer Piston, Alex Von Hagen-Jamar, Nazia Kahn, Mary Nameth, and Willo Jean-Baptiste. Knowledge Networks, Inc., performed the data collection and managed panel relations under a contract with Stanford University. Larry Osborn was the Knowledge Networks study director. Mike Dennis was the Knowledge Networks Principal Investigator. They were supported by a team including Melanie Thomas, Karen Semans, Naomi Keller, Tetyana Shvets, Charles DiSogra, Elisa Chan, Leticia Maciel, Xiaolu Tan, Xiuli Tang, Erlina Hendarwan, Mark Shanosky, and Alex Nagaev. Questionnaire content was contributed by participants in the ANES Online Commons. The members of the ANES Board of Overseers contributing to the Panel Study and reviewing Online Commons proposals were John Mark Hansen (chair), John Aldrich, Stephen Ansolabehere, Henry Brady, Brandice Canes-Wrone, Karen Cook, Catherine Eckel, Kenneth Goldstein, Donald Green, Vincent Hutchings, Paula McClain, Kathleen McGraw, Walter Mebane, Randy Olsen, Richard Petty, Gary Segura, Daron Shaw, V. Kerry Smith, Lynn Smith-Lovin, and Paul Sniderman. We are grateful for additional advice, comments, and collaboration pertinent to questionnaire development from Clark McCauley, Lynn Smith-Lovin, Keith Payne, Charles Judd, Mark Chaves, Paul Gronke, Ken Wald, Stanley Feldman, Neil Malhotra, Adam Glynn, Gary Segura, Ken Goldstein, Tony Greenwald, N. Sriram, Brian Nosek, Chappell Lawson, and many others. Contact The ANES website address is http://www.electionstudies.org. Any questions not answered on the ANES website or by this report may be directed to ANES staff by email at [email protected] 2 Contents 1. Introduction..................................................................................................................... 5 Panel Study Overview..................................................................................................... 5 “Full Release” of the Panel Data .................................................................................... 6 ANES Content and “Off-Wave” Content ....................................................................... 6 Panel Study at a glance ................................................................................................... 7 2. Survey Questions ............................................................................................................ 8 Online Commons ............................................................................................................ 8 Surveys on non-political topics: the “off-waves” ........................................................... 9 ANES Question Design, Development, and Pilot Testing ........................................... 11 Questionnaires and Wave Maps.................................................................................... 12 3. Sample Design and Recruitment Procedures................................................................ 18 Population and Sample ................................................................................................. 18 Household/Telephone Sample Design.......................................................................... 18 Person-Sampling Procedures ........................................................................................ 19 4. Data Collection Procedures........................................................................................... 26 Profile Survey ............................................................................................................... 26 Monthly Survey Procedures.......................................................................................... 26 Timeline and Sequence of Interviews........................................................................... 28 5. Data Processing............................................................................................................. 39 Cleaning and Quality Control ....................................................................................... 39 Administrative Variables .............................................................................................. 40 Derived Variables ......................................................................................................... 42 Disclosure Edits ............................................................................................................ 42 File-Build Program ....................................................................................................... 43 6. Recruitment Dispositions and Outcome Rates ............................................................. 44 Outcome Rates Defined ................................................................................................ 44 Calculating Final Dispositions...................................................................................... 45 Final Dispositions ......................................................................................................... 47 Estimating Eligibility.................................................................................................... 49 Response Rates and other Outcome Rates.................................................................... 50 7. Panel Retention ............................................................................................................. 52 Interviews at Each Stage............................................................................................... 52 Dropout Recovery......................................................................................................... 53 Retention Summaries.................................................................................................... 56 Retention Matrix ........................................................................................................... 67 Attrition Effects ............................................................................................................ 71 8. Weights ......................................................................................................................... 73 9. Accuracy of Estimates ................................................................................................ 78 10. Validation of Respondents & Data ............................................................................. 83 11. Analyzing the Data with Weights and Correct Significance Testing ......................... 89 Selecting Appropriate Weights..................................................................................... 89 Running Weighted Analysis ......................................................................................... 93 Statistical Significance Testing and Standard Errors.................................................... 93 12. Orientation to the Data File......................................................................................... 95 3 Cases on the file............................................................................................................ 95 File formats ................................................................................................................... 95 Variables on the file...................................................................................................... 95 Combining data from branched questions ...................................................................