
* Invited Papers Tuesday, July 26th, 2:00 p.m. - 3:30 p.m. Session: Adapting Interviewer Training Across Cultures Chair: Esther Ullman, University of Michigan Location: Michigan Ballroom I Perspectives on international training Katherine Mason, RTI International Steven Litavecz, RTI International David Plotner, RTI International In this presentation, Ms. Mason will discuss experiences with international trainings which have been conducted in the following countries: Nigeria, Kenya, Mexico, Indonesia, Ghana, Zambia, Thailand, India and China. Topics will include cultural perspectives on time management, reading items verbatim and challenges with multiple translations, dealing with proxies, respondent privacy, challenges with subcontractors and data transmission issues. Challenges and lessons learned of conducting computer assisted personal interviewing (CAPI) training and providing capacity building supports for a national household panel survey in Ghana Yu-chieh (Jay) Lin, University of Michigan The knowledge sharing of moving from paper to computer assisted personal interviewing (CAPI) and launching the national household panel survey in Ghana among global survey research and operational team members provides unique insights on adapting interviewer training with local contexts and overcoming both infrastructure and cultural challenges. Lessons learned include using the onsite train-the-trainer experience to finalize interviewer training components, customizing training activities based on trainees' immediate feedback and learning progress, being flexible with different working styles and communication approaches, understanding of differences and reacting quickly, and developing innovative technical solutions for field data collection, data sharing and analyses, and quality control. This presentation provides management and technical examples for audiences who are particularly planning to conduct survey research and data collection in developing countries or areas where need capacity building supports. Interviewer training for a pre-school evaluation in Chad Nathan Jones, University of Wisconsin Survey Center Many researchers and staff providing technical support for data collection in developing countries face challenges when hiring and training local interviewers. In order to apply best practices for data collection, assure cultural adaptation of survey measures, and effectively work with multi-cultural data collection teams, interviewer training needs to be adapted to the different cultures where it occurs. The panel, Training in Developing Countries, will feature presentations by representatives from several survey organizations. This presentation will highlight lessons learned while training interviewers and conducting surveys for parents and children attending a preschool program for Darfuri refugees living in the Goz Amer refugee camp in Eastern Chad. I will focus on strategies for training inexperienced interviewers, maintaining data quality, and field work management in harsh conditions. Tuesday, July 26th, 2:00 p.m. - 3:30 p.m. Session: Approaches to Test for Measurement Invariance Chair: Eldad Davidov, University of Zurich Location: Michigan Ballroom II Measurement invariance of different dimensions of nationalism in the ISSP : Comparisons over two time points and four countries Peter Schmidt, University of Giessen Jan Ciecuch, University of Zurich Eldad Davidov, University of Zurich Recently there has been a controversy about the use of techniques for establishing measurement invariance in the Journal Comparative Politics 2015 and 2016 dealing with scales in the World Value Study. In our study we evaluate two new techniques for establishing measurement invariance: Bayesian approximate invariance proposed by Muthen/ Aspourov(2012) and van der Schoot(2013) and the alignment procedure proposed by Aspourov/Muthen (2014). We analyze data from two waves of the ISSP identity module over several countries and evaluate the results based on the classical invariance tests compared with the bayesian approximate invariance test and the solution given by the alignment procedure. The cross-country comparability of the immigration module in the European Social Survey 2014-15 Jan Cieciuch, University of Zurich Eldad Davidov, University of Zurich Peter Schmidt, University of Giessen Rene Algesheimer, University of Zurich A special module about attitudes toward immigration and threat due to immigration was implemented in the 7th Round of European Social Survey (ESS). In our project we set two goals. The first one was to establish in a theory- driven way latent variables based on items included in the module. These latent variables can be used by researchers in their substantive work on immigration using the ESS. The second goal was to test for measurement invariance of these scales across 15 ESS countries. We proposed the four following latent variables: allowing for immigrants belonging to different ethnic groups than the majority population into the country; qualification for entry; and two types of threat due to immigrants, realistic and symbolic. First, we tested each latent variable in each country separately in single Confirmatory Factor Analyses (CFAs). Next, we tested for measurement invariance of each latent variable using multigroup Confirmatory Factor Analysis (MGCFA). We differentiated between three levels of measurement invariance, configural, metric and scalar, and we applied two approaches: an exact and an approximate measurement invariance approach. If full or partial exact measurement invariance could not be established, we tested whether approximate invariance was given. Configural and metric invariance was supported for all constructs across most countries. Unfortunately scalar invariance was supported for the latent variables only across a subset of countries. The subset of countries where approximate scalar invariance was established was larger than the subset of countries for which exact measurement invariance could be established. Comparing groups that are only partially and approximately comparable: an adaptive Bayesian approach Daniel L. Oberski, Tilburg University Muthén & Asparouhov (2012) introduced the idea of Bayesian approximate measurement invariance : rather than assume that all measurement parameters are equal across groups, a fudge factor is introduced that allows for relatively small random differences. The fudge factor must be specified in advance and results can be rather sensitive to this choice (Rudnev 2015). Moreover, when subsets of items not approximately invariant, the approximate invariance procedure does not do well at detecting these violating (partially noninvariant) items and can lead to serious bias in the estimates of interest (Van de Schoot et al. 2014). Muthén & Asparouhov (2013) called this the alignment problem and suggested a solution based on factor rotation methods. This talk discusses a different possible solution to the alignment problem that follows naturally from the Bayesian approach and connects directly with the regularization literature (Tutz 2012; Hastie et al. 2015). Our approach is to adaptively learn from the data which items are approximately invariant, and which are not. We discuss simulation results that compare the performance of this procedure with that of the standard approximate MI model. We also apply our new approach to empirical data, demonstrating how our approach may be useful for comparing groups that are only partially, and approximately, comparable. * Measurement invariance in international large-scale assessments: Integrating theory and method Fons van de Vijver, Tilburg University, The Netherlands Ralph Carstens, The IEA Data Processing and Research Center, Germany Wolfram Schulz, The Australian Council for Educational Research, Australia One of the aims of international large-scale assessments (ILSAs) of educational achievement is to collect standardized data that allow cross-national comparisons of student achievement, behaviors or attitudes, and the influences of school and classroom factors or family background on those outcomes. Complex modeling of cross-national data requires that the items designed to measure a latent factor have invariant psychometric characteristics, which means that they measure the same trait across countries, the measured latent constructs have the same meaning in all participating countries, and survey respondents interpret the items in a similar way. Non-invariant measures might due to systematic biases in the measurement instrument or differences in the way specific items are responded. Lack of measurement invariance can introduce bias and limit comparison across countries. This present study contributes to invariance evaluation over and beyond the existing investigations through the modeling of differences in measurement. We start with an overview of the basic concept of measurement invariance and its challenges with regard to ILSAs. We evaluate measurement invariance with the exact invariance assumptions and integrate a relatively new prototype of the assumptions with substantive insights about measurement bias from different factors namely in the context, item and person levels. We present a stepwise strategy for evaluating measurement invariance with a productive way of dealing with the unattainable ideal of strict invariance assumptions. With an empirical example, we argue that a generic latent structural and measurement modeling or simple structure of a common-factor model is unlikely to yield fully comparable
Details
-
File Typepdf
-
Upload Time-
-
Content LanguagesEnglish
-
Upload UserAnonymous/Not logged-in
-
File Pages104 Page
-
File Size-