University of Kentucky From the SelectedWorks of Patric R. Spence 2007 State of the Method: An Examination of Level of Analysis, Methodology, Representation and Setting in Current Organizational Communication Research Patric R Spence, University of Kentukcy Colin R Baker Available at: https://works.bepress.com/patric_spence/79/ Journal of the Northwest Communication Association, 36 (Spring 2007), 111–124. State of the Method: An Examination of Level of Analysis, Methodology, Representation and Setting in Current Organizational Communication Research Patric R . Spence Calvin College Colin R . Baker Wayne State University This study examines the frequency and content of organizationally related studies in six communication journals (Communication Monographs, Communication Studies, Communication Research Reports, Human Communication Research, Journal of Applied Communication Research and The Western Journal of Communication) over a seven-year period, from 1998–2004, for the purposes of describing current organizational research. The authors report that 14% of the journal articles were classified as “organizational” and of those, half were quantitative. The most common method used was the field study (46%), followed by surveys (29%). Few studies used multiple methodologies, and the authors discuss these implications. This study describes the characteristics of studies concerning organizational communication that are published in select communication journals. We are particularly interested in the types of methods and settings explored by researchers, and we ground our examination in prior studies that have Patric R. Spence (PhD, Wayne State University) is an Assistant Professor in the Department of Communication Arts and Sciences at Calvin College. Spence’s research interests include organizational crisis, risk communication and the role of new media in emergency response. His recent research has appeared in Journal of Applied Communication Research, Sociological Spectrum and the Journal of Emergency Management. Direct inquiries to [email protected]. Colin R. Baker (Ph.D., Michigan State University) is an Assistant Professor in the Department of Communication at Wayne State University. His research interests include communication during the organizational entry process, the impact of leadership on group communication, and social influence in applied contexts. 111 Journal of the Northwest Communication Association examined the history of organizational communication. Several published studies have examined the history of organizational communication re- search (Greenbaum, Hellweg & Walker, 1985; Tompkins & Redding, 1988) and topics researched (Allen, Gotcher, & Seibert, 1993). Journals have pub- lished issues devoted specifically to organizational communication [Hu- man Communication Research, 32(4) and Communication Research, 18(2)]. Yet, many questions concerning organizational research remain unexplored. For example, although a special issue of Human Communication Research was devoted to multi-level research, the issue did not identify the representa- tion of multi-level research in the field. An article identifying research methodologies in organizational communication studies was published in Communication Studies which covered years 1979–1989 (Wert-Gray, Center, Brashers, & Meyers, 1991). However the study did not examine the setting of data collection or type of analysis employed. Moreover, the data presented in the Communication Studies article are more than two decades old and we might expect that much has changed. This study addresses select issues which previously have been unexamined in the literature. First, the representation of organizational communication articles within communication journals is identified. Next, the extent to which authors examine communication phenomena in organizations at multiple levels of analysis is reviewed, followed by investigations into the preferred methods and settings used by organizational communication researchers. Organizational Communication and Multi-level Research As noted by Smith, Schneider and Dickson (2005), in organizational research the belief that everything interacts with everything else has been a consistent theme. Several behavioral researchers have suggested that the most significant impediment to the advancement of organizational research knowledge is the failure to simultaneously incorporate individual and organizational influ- ences interacting across levels of analysis (Johns, 2001). The problem is called the micro-macro distinction to organizational scientists, contextualization to sociologists, and multi-level research to others. Cappelli and Sherer (1991) argue that it is impossible to develop a common paradigm for organizational behavior without incorporating multiple levels of analysis in research. 112 State of the Method Scholars generally conduct research at only one level of analysis (often at the individual level, but other times at the group, organizational or in- dustrial levels). Regardless, researchers often look to the next lowest level for explanatory purposes (Hackman, 2003). The dominant methodological approach to multi-level research is based on what Klein, Dansereau and Hall, (1994) call the mixed determinants model. In this model, both context and individual variables are hypothesized to affect an outcome variable at the individual level of analysis. However, other models do exist (Chan, 1998; Earley & Brittain, 1993). The motivation for conducting multi-level research is to produce results that are valid on multiple levels, avoiding the impulse to simply generalize from one level of analysis to another. Results from one level of analysis do not generalize well or exactly to other levels except under the most restric- tive circumstances (Firebaugh, 1979). Therefore, generalizing data that have been aggregated to the group level, back down to the individual level at which the data were collected, is a well known ecological fallacy (Kozlowski & Klein, 2000). For example, aggregate team problem-solving ability may be highly related to success at the team level but does not necessarily gen- eralize to the problem solving abilities of the individual. Ecological cor- relations based on aggregate data are generally inflated estimates of lower level relationships (Robinson, 1950). This is true for findings at the lower level of analysis when generalization to a higher level is attempted. Such an attempt is called an atomistic fallacy. Unlike many areas of communication inquiry, studies of organizational communication allow for investigation of constructs at multiple levels of analysis. A study’s measurement may be at the organizational, group, or individual level, or a combination of several levels simultaneously. Klein and Kozlowski (2000) argue that although organizations exist at multiple levels, organizational researchers [in general] do not treat them as such. Alternatively, organizational communication scholars may be more attuned to these arguments, as the study of organizational communication has come a long way from being interpersonal communication “in a box.” Through quantitative methods that enable us to observe information linkages at multiple levels simultaneously, as well as qualitative case studies that allow 113 Journal of the Northwest Communication Association us to acquire an understanding of multiple influences on organizational behaviors, we may see a fair amount of articles investigating communica- tive behaviors at multiple levels of analysis. Therefore, scholars attune to these issues suggest a need to engage in multi-level research in both the theoretical and methodological realms. Moreover, organizational commu- nication scholars may be uniquely advantaged over other scholars to look at context characteristics in research. Thus, the extent to which organizational communication research examines communication phenomena at multiple levels of analysis is examined. Organizational Communication Research and Methodological Choice Organizational researchers have an advantage when it comes to levels of analysis issues in that while quantitative multilevel research is often statisti- cally complex, qualitative researchers may find it comparatively less taxing to address the impact of certain variables at multiple levels of analysis. As compared to other disciplines (e.g., business, management and industrial/ organizational psychology), communication researchers perhaps engage in more qualitative studies allowing these researchers to more easily and more often examine important issues at multiple levels of analysis. In addition to levels of analysis issues, questions related to methodologies and settings utilized by active researchers are important to our understanding of the present direction of the field. Organizational scholars hold many varying assumptions regarding the nature of scientific inquiry into the study of communication research. Mea- surement can be distinguished with respect to whether meaningful numeric symbols are used (quantitative measurement) or other non-numeric sym- bols are used to determine meaning (qualitative measurement) (Babbie, 1995; Frey, Botan & Kreps, 2000). Although some scholars advocate for one perspective over the other, many perceive value in seeking to under- stand organizational communication from a multiplicity of perspectives. In all, organizational communication scholars have a variety of perspectives from which they understand organizations. Examined next are the broad methodological