07_184_Ch07.qxd 3/29/07 6:43 AM Page 137 Data Reduction Techniques for 7 Large Qualitative Data Sets EMILY NAMEY, GREG GUEST, LUCY THAIRU, AND LAURA JOHNSON You have your way. I have my way.As for the right way, the correct way, and the only way, it does not exist. FRIEDRICH NIETZSCHE ORKING WITH DATA COLLECTED through a team effort or in multiple sites can be both challenging and rewarding. The sheer Wsize and complexity of the data set sometimes makes the analy- sis daunting, cbut a large data set may also yield richer and more useful in- formation. In this chapter, we explore strategies for combining qualitative and quantitative analysis techniques for the analysis of large, qualitative data sets. Large is a relative term, of course, and many of the techniques de- scribed here are applicable to smaller data sets as well. However, the bene- fits of the data reduction techniques we propose increase as the data sets themselves grow in size and complexity. In our selection of techniques, we have taken a broad view of large qualitative data sets, aiming to highlight trends, relationships, or associations for further analysis, without deempha- sizing the importance of the context and richness of the data themselves. This perspective also brings focus to the multiple interpretive lenses that a group of researchers brings to team-based analysis. Throughout the chapter, we use examples from some of our research to illustrate the use of the methods discussed. In doing so, we identify the strengths and weaknesses of each method and suggest ways in which an ap- propriate technique may be chosen for a given research question and its corresponding data set. 137 07_184_Ch07.qxd 3/29/07 6:43 AM Page 138 138 EMILY NAMEY ET AL. Approaches to Data Analysis During the past few decades, qualitative research has greatly benefited from theoretical and methodological developments in data analysis (see, e.g., Bernard and Ryan 1998, Dey 1993, LeCompte and Schensul 1999). Analyses typically fall into one of two categories: content and thematic. In content analysis, the researcher evaluates the frequency and saliency of par- ticular words or phrases in a body of original text data in order to identify keywords or repeated ideas. In addition to simple word counts, content analysis can be expanded to include associated attributes of keywords and other semantic elements, such as synonyms, location in the text, and sur- rounding words or phrases (Dey 1993:59). Content analysis techniques are valued for their efficiency and reliabil- ity. With appropriate software, large numbers of text files can be quickly scanned and keywords tallied. And since the original, “raw” data are used, there is minimal interpretation involved in the word counts, resulting in greater reliability. The primary drawback to content analysis is that context is usually not considered or is highly constrained, limiting the richness of the summary data produced. Thematic analysis, in contrast, is more involved and nuanced. Thematic analysis moves beyond counting explicit words or phrases and focuses on identifying and describing both implicit and explicit ideas. Codes devel- oped for ideas or themes are then applied or linked to raw data as summary markers for later analysis, which may include comparing the relative fre- quencies of themes or topics within a data set, looking for code co-occur- rence, or graphically displaying code relationships. Reliability is of greater concern with thematic analysis than content analy- sis because research analysts must interpret raw text data in order to apply codes, and because interpretations may vary across analysts. Strategies for monitoring and improving intercoder agreement, and therefore reliability, add to the time required for thematic analysis, but the investment is well worth the context-rich coded data produced (see chapters 6 and 11, this volume). Both content and thematic analysis can be data-driven, as in grounded theory (Glaser and Strauss 1967, Kearney et al. 1994, Wright 1997), or the- ory-driven (Krippendorf 1980, Weber 1990). In a data-driven approach, the researcher carefully reads and rereads the data, looking for keywords, trends, themes, or ideas in the data that will help outline the analysis, before any analysis takes place. By contrast, a theory-driven approach is guided by specific ideas or hypotheses the researcher wants to assess. The researcher may still closely read the data prior to analysis, but his or her analysis cate- gories have been determined a priori, without consideration of the data. 07_184_Ch07.qxd 3/29/07 6:43 AM Page 139 DATA REDUCTION TECHNIQUES FOR LARGE QUALITATIVE DATA SETS 139 Theory-driven approaches tend to be more structured, and for this rea- son may be considered more reliable, in the sense that the same results are likely, regardless of the coder. Conversely, data-driven approaches may be considered to have greater validity because they are more flexible and open to discovery of themes or ideas not previously considered, resulting in the- ory that is “grounded” in the data. Fortunately, neither approach is so rigid as to prevent borrowing from the other, to maximize the findings of an analysis and to balance reliability and validity. Theory-driven analysis does not preclude the analyst from uncovering emergent, data-driven themes, which may then be added to the analysis, and similarly data-driven analy- ses may generate theories to explain emergent structure. The techniques presented below span both content and thematic analysis and are applica- ble to both theory- and data-driven approaches. The examples presented, however, are primarily drawn from data-driven thematic analyses. Framing the Analysis Large qualitative data sets generally encompass multiple research questions. Hence, very few, if any, analyses of such data sets simultaneously involve all of the data that have been collected. From the outset, researchers need to delineate the boundaries of a given analysis with a comprehensive analysis plan. This plan can include guidelines for data set reduction, including whether all the data will first be coded in an exploratory analysis, whether they will be partitioned in a way appropriate for theoretical analysis and hy- pothesis testing, or whether some data will simply not be included in spe- cific analyses. Eliminating data not relevant to the analysis at hand—or extracting the data that are relevant—is usually the first, and arguably the simplest, form of data reduction. As Miles and Huberman (1994: 11) ex- plain, Data reduction is not something separate from analysis. It is part of analy- sis. The researcher’s decisions—which data chunks to code and which to pull out, which evolving story to tell—are all analytic choices. Data reduc- tion is a form of analysis that sharpens, sorts, focuses, discards, and organ- izes data in such a way that “final” conclusions can be drawn and verified. In cases where the larger data set was compiled from more than one type of data collection instrument (e.g., semistructured in-depth inter- views, structured focus groups, and pile-sorting activities), the researcher needs to make a decision about the type of data she or he will select from the larger data set. She or he may choose to analyze data from only one 07_184_Ch07.qxd 3/29/07 6:43 AM Page 140 140 EMILY NAMEY ET AL. type of instrument, or from several different instruments. As described by Patton (1990), this form of “triangulation” across different data collection strategies during data analysis can be particularly helpful when dealing with large data sets. The researcher may also need to frame the analysis in terms of the sources of data or the categories of participants from whom the data were collected (MacQueen and Milstein 1999). This may require limiting the analysis to one or two sites of a multisite project, or limiting the num- ber of subgroups included in the analysis (e.g., including only data relevant in terms of select participant characteristics regardless of site). The researcher’s primary guide in all of these data reduction decisions is a clearly defined analysis objective, accompanied by an analysis plan. The ob- jective may be based on the research questions addressed in the project or it may be determined on the basis of specific reporting or publishing goals. In the analysis plan, the researcher also defines the level of the analysis—whether exploratory, descriptive, hypothesis testing, or comparative in nature—and chooses data accordingly. These initial data reduction decisions are critical to choosing an appropriate strategy for the ensuing analyses. Analytical Techniques In this section, we summarize the basic steps, strengths, and weaknesses of several analytical techniques for large qualitative data sets. The techniques do not comprise an exhaustive list; rather, on the basis of our experience in the field, we highlight some of the techniques we have found useful when working as a team in multisite research studies. In addition to these techniques, the fundamental step of reading and rereading text data files is still essential for adding context to theory-driven analyses and for identify- ing themes for data-driven analyses. Structural Coding One effective method for making large qualitative data sets more manage- able, for either content or thematic analysis, is to develop and apply a se- ries of “structural” codes to the data. As described by MacQueen et al. in chapter 6 of this volume, the term structural code refers to question-based, rather than theme-based, codes. This approach works for data collected us- ing structured or semistructured interview or focus group guides that have discrete questions and probes that are repeated across multiple files in a data set. Each discrete question and its associated probes are assigned a code that is then applied or linked to the question and subsequent response text in each data file.