Workshop on Interdisciplinary Standards for Systematic Qualitative Research Cultural Anthropology, Law and Social Science, Political Science, and Sociology Programs National Science Foundation Supported Workshop Report prepared by: Michèle Lamont Harvard University Patricia White National Science Foundation Acknowledgements We wish to thank James Granato, former NSF Political Science Program Director, Joane Nagel, former NSF Sociology Program Director, Stuart Plattner, former NSF Cultural Anthropology Program Director, and Christopher Zorn, former NSF Law and Social Science Program Director, for their help in planning and participation in the workshop, Karen Duke and Michelle Jenkins, NSF Social and Political Sciences Cluster staff members, for their administrative and technical support, Lauren Rivera and Sabrina Pendergrass, graduate assistants for their work with Professor Lamont on workshop and report preparations at Harvard University, Joane Nagel and Kristin Luker, Professor of Sociology and Jurisprudence and Social Policy, University of California, Berkeley for their thoughtful comments on the draft report, and the 23 workshop participants who took on the enormous task of representing their respective disciplines at the workshop through the submission and presentations of short papers prior to the workshop and comment and recommendations during and after the meeting. We also thank the 2007 graduate student cohort of the Department of Sociology at Harvard University and Camonia Long, Howard University graduate student, for their reactions to the report. Workshop Participants & Attendees Michèle Lamont, Harvard University, Jody Miller, University of Missouri, St. Louis Workshop Organizer Joane Nagel, University of Kansas Andrew Bennett, Georgetown University Beth Rubin, University of North Ted Bestor, Harvard University Carolina-Charlotte Kathleen Blee, University of Pittsburgh Gery Ryan, Rand Corporation Don Brenneis, University of California, Susan Silbey, Massachusetts Institute of Santa Cruz Technology John Bowen, Washington University in St. Louis Mario Small, University of Chicago John Comaroff, University of Chicago Kathleen Thelen, Northwestern University David Collier, University of California, Berkeley Lisa Wedeen, University of Chicago Colin Elman, Arizona State University Susan Weller, University of Texas Medical Wendy Espeland, Northwestern University Branch-Galveston Linda Garro, University ofofof California,California,California, Patricia White, (Sociology) National Science Los Angeles Foundation John Gerring, Boston University Alford Young, Jr., University of Michigan James Granato, University of Houston Christopher Zorn, University of South Carolina Wendy Griswold, Northwestern University Jennifer Hochschild, Harvard University Richard Lempert, National Science Foundation Felice Levine, American Education Research Association Workshop on Interdisciplinary Standards for 2 Systematic Qualitative Research Executive Summary On May 19-20, 2005, a workshop on Interdisciplinary Standards for Systematic Qualitative Research was held at the National Science Foundation (NSF) in Arlington, Virginia. The workshop was co­ funded by a grant from four NSF Programs—Cultural Anthropology, Law and Social Science, Political Science, and Sociology—to Dr. Michèle Lamont, Harvard University. Professor Lamont was assisted in organizing the workshop by representatives from each discipline who coordinated group reports. The Cultural Anthropology group was co-chaired by Drs. Ted Bestor (Harvard) and Gery Ryan (RAND); Law and Social Science by Dr. John Bowen, (Washington University, in St. Louis); Political Science by Andrew Bennett (George Washington University); and Sociology by Dr. Kathleen Blee (University of Pittsburg). It is well recognized that each of the four disciplines have different research design and evaluation cultures as well as considerable variability in the emphasis on interpretation and explanation, commitment to constructivist and positivist epistemologies, and the degree of perceived consensus about the value and prominence of qualitative research methods. Within this multidisciplinary and multimethods context, twenty-four scholars from the four disciplines were charged to (1) articulate the standards used in their particular field to ensure rigor across the range of qualitative methodological approaches;1* (2) identify common criteria shared across the four disciplines for designing and evaluating research proposals and fostering multidisciplinary collaborations; and (3) develop an agenda for strengthening the tools, training, data, research design, and infrastructure for research using qualitative approaches. Prior to the workshop each participant prepared a short paper addressing the three topics (qualitative research standards, evaluation criteria, and agenda of future opportunities and needs). During the workshop, breakout sessions were held where each disciplinary group was asked to discuss and list the field’s major qualitative research standards. These lists were then shared with the full workshop. Two categories of qualitative research standards were identified—(1) “shared” or key standards relevant to all four disciplines; and (2) standards judged essential by two or three disciplines. Participants also provided recommendations for improving qualitative research, training and infrastructures and identified promising areas of research in the four disciplines that would benefit from study using qualitative research approaches. The workshop report is organized into four sections: Qualitative Research Design and Methods; Standards for Qualitative Research across Disciplines (in Anthropology, Law and Social Science, Political Science and Sociology); Recommendations for Producing Top Notch Qualitative Research; and Promising New Research Areas and Topics. Written commentary prepared by workshop participants is included in the Appendix. What follows is a brief summary of each. 1*Methodological approaches include ethnography, historical and comparative analysis, textual and discourse analysis, focus groups, archival and oral history, observational studies, interpretation of images and cultural materials, and unstructured and semi-structured interviews. 3 Executive Summary Qualitative Research Design and Methods Workshop participants discussed the strengths of qualitative methods, standards qualitative research shares with quantitative research, and standards that are unique only to qualitative research. The major strength of qualitative research was judged to be the rich range of methodological tools available to study meaning, social processes, and group variations. The sample sizes and strategies of qualitative and quantitative approaches to research were contrasted. Qualitative research stresses in-depth contextualization, usually with small sample size. Qualitative research sampling techniques, while nonrandom, are usually attentive to demographic and theoretical dimensions. The combination of small and in-depth samples chosen for theoretical relevancy allows qualitative research findings a degree of significance or generalizability beyond individuals or single cases and provide opportunities to demonstrate rigor in all phases of a qualitative research project. “Shared” Criteria for Designing and Evaluating Qualitative Research Across Disciplines Workshop participants agreed that the four disciplines shared several standards for designing and evaluating high quality qualitative research. All value projects that: • Situate the research in appropriate literature; that is, the study should build upon existing knowledge • Clearly articulate the connection between theory and data • Describe and explain case selection; why particular sites, participants, events, or cases are chosen • Pay attention to alternative explanations and negative cases • Operationalize constructs and describe expected findings • Provide clear and detailed descriptions of both data collection and anticipated data analysis techniques: specify what counts as data, how the researcher will go about obtaining data and analyzing it • Describe the intellectual, social, and political significance of the research • Discuss generalizability or significance beyond the specific cases selected • Specify the limitations of the research and anticipate potential reviewer objections • Discuss the preparation of the researcher for the proposed project in terms of: o Cultural fluency o Language skill o Appropriate methodological/technical training o Knowledge of particular research context Recommendations for Producing Top Notch Qualitative Research Participants established that qualitative research could be enhanced by increased investments in education, training, and infrastructure. For example, NSF could enhance qualitative research through increased funding for students and scholars as well as publicizing its commitment to supporting high quality qualitative proposals. Support for pre-dissertation support, small pilot study grants for faculty, and student training opportunities through professional associations would likely be most effective. Workshop on Interdisciplinary Standards for 4 Systematic Qualitative Research Promising New Research Areas and Topics Each disciplinary working group articulated new and exciting research areas that would benefit immensely from qualitative research or are taking advantage of qualitative research approaches, and in the process opening up new avenues of understanding. Social and cultural anthropological studies of responses