Copyright © eContent Management Pty Ltd. Journal of Management & Organization (2009) 15: 122–131. Audience response systems as a data collection method in organizational research MATTHEW W MCCARTER College of Business, University of Illinois at Urbana-Champaign, Champaign IL, USA ARRAN CAZA College of Business, University of Illinois at Urbana-Champaign, Champaign IL, USA ABSTRACT Audience responses systems are electronic devices allowing audience interaction and they are increasingly being used in educational and business settings to enhance various pedagogical and practical processes. This paper discusses how ARS technology may be used as a method of collecting data for research purposes. Specifically, this paper demonstrates ARS technology’s potential utility by duplicating findings from two organisational studies, it discusses how ARS technology may be used to address three prevalent data collection problems, and it suggests how ARS technology may pro- vide scholars with increased access to certain organisational settings, as well as greater integration between research and service activities. Keywords: audience response system, personal response system, audience response technology, data collection techniques, data collection technology, organizational research methods INTRODUCTION discuss how they are traditionally used. Second, ata collection is an important element of we duplicate findings from two organisational Dany research methodology. As scholars strive studies to demonstrate how ARS technology may to understand behavioural phenomena in organi- be used to test theory. Third, we discuss how sations, the need for data collection methods that ARS technology may be used to address three are convenient to both scholars and respondents prevalent data collection problems. Fourth, we and which may be used to gather data from a wide discuss other issues related to using an ARS for variety of organisational settings becomes impor- research, including limitations and benefits, such tant. This is especially true given evidence suggest- as increased access to certain organisational set- ing that individuals are becoming less inclined to tings and a greater integration between research participate in organisational research (Bryman and service activities. 2000). The purpose of this paper is to discuss how audience response system (ARS) technology may AUDIENCE RESPONSE SYSTEMS be used to collect data for research purposes. An audience response system is an electronic This paper is in four parts. First, we review device designed to allow immediate interaction the literature on audience response systems and between an individual presenter and a large audi- 122 JOURNAL OF MANAGEMENT & ORGANIZATION Volume 15, Issue 1, March 2009 Audience response systems as a data collection method in organizational research ence. An ARS typically has two parts. The first attendance, and engagement appear to increase component is a remote control (or ‘clicker’) that when ARS technology is used as compared to audience members use to respond to questions. when it is not (See Fies & Marshall 2006, for a The second component is an electronic receiver review of empirical evidence). (or ‘hub’) that records, and optionally, displays individuals’ responses. An ARS allows for a large ARS technology in organisations number of individuals to respond simultaneously ARS technology is used in organisational settings, (e.g. 1,000 people). Each individual response is as well. There is a growing body of literature in recorded by the hub and can be displayed via pro- the trade and management press advocating ARS jector or exported as a data file for use in other use. A basic search of the EBSCOhost database software. For example, a presenter during a board using the phrases ‘audience response system’ and meeting could use an ARS to present a multiple- ‘personal response system’ found 105 pieces in the choice question to a group of partners, have each practitioner and academic press (search conduct- partner choose an answer, and then immediately ed February 24, 2009). Examination of these display the number selecting each answer. pieces showed ARS use in a variety of organisa- tional contexts. For example, Krantz (2004) ARS technology in education reported on marketing organisations using an ARS technology was originally designed as a ped- ARS to receive feedback from potential customers agogical tool to enhance student learning in ele- at tradeshows. Hatch (2003) described the use of mentary, middle- and high-school, university, and ARS technology in such activities as strategic post graduate settings (Fies & Marshall 2006). planning, brainstorming, monitoring training Caldwell (2006) documents the use of ARS tech- effectiveness, and ice-breaking. ARS use has also nology in fifteen different disciplines of higher been reported to increase organisational efficiency education (e.g. medicine, physics, psychology; and effectiveness in decision making and team also see Duncan 2006). The growing use of ARS planning meetings (Training and Development units reflects the many benefits they can provide 2006). Organisations that have adopted ARS for teachers and students. For example, Homme, technology include Boeing, Academy of the US Asay, and Morgenstern (2004) reported an Federal Bureau of Investigation, IBM, John increase in attendance and enthusiasm of resident Deere, McGraw-Hill, National Academy Founda- doctors using an ARS for board review sessions. tion, Prentice-Hall, Raytheon, Toys ‘R’ Us, Unit- Similarly, Miller, Asher and Getz (2003) found ed States Army and Navy, Walt Disney World, that participants using ARS units reported greater and YMCA (e.g. see http://www.einstruction.com presentation quality, speaker ability, and attention for a list of firms using their particular ARS unit). during professional education programs when The diversity of these organisations suggests the compared to participants not using an ARS. Oth- broad-based adoption and use of ARS systems by ers have reported that use of an ARS benefits a organisations. range of classroom activities, including collecting demographic information, allowing students to ARS technology in academic research share knowledge and experiences pertinent to Despite the growing use of ARS units in academ- course content, polling student opinions on vari- ic institutions, there has been little consideration ous academic and public policy issues, testing of how an ARS may be used to gather data for comprehension of course material, giving in-class research purposes. Some authors have men- quizzes, and facilitating group discussion (Byrd, tioned the possibility, but most have not subject- Coleman & Werneth 2004; D’Arcy, Eastburn & ed the idea to rigorous consideration (e.g. Mullally 2007). Overall, student involvement, Gamito, Burhansstipanov, Krebs, Bemis & Volume 15, Issue 1, March 2009 JOURNAL OF MANAGEMENT & ORGANIZATION 123 Matthew W McCarter and Arran Caza Bradley 2005; Draper, Cargill & Cutts 2002). A Two fundamental hypotheses they tested in a notable exception is Bunz (2005), who compared series of studies were: data collection using machine-readable forms Hypothesis 1. The proportion of volunteers and an ARS. The results showed no difference will decrease as the size of the group increases. between methods in time pressure effects on responses. In addition, students were more Hypothesis 2. The proportion of volunteers engaged in answering questions through the ARS will decrease as the payoff for volunteering and found the ARS no more or less difficult than decreases. traditional forms. Overall, Bunz (2005) supports the use of ARS technology as a data collection Hypothesis 1 reflects the logic of free riding method, highlighting the benefit of quick, elec- (Olson 1965), since many volunteer situations tronic data storage, and suggesting that because require only one party to sacrifice (i.e. volunteer) many university and businesses are adopting before the entire group benefits. The presence of ARS units, their availability to scholars and many potential volunteers makes each person less response participants is increasing. willing to volunteer. Hypothesis 2 is based on the assumption that volunteers incur costs that non- USING AN AUDIENCE RESPONSE volunteers do not (Diekmann 1985). As volun- SYSTEM TO TEST THEORY teering becomes increasingly costly, individuals One important limitation in Bunz’s (2005) study are less likely to volunteer. was it did not investigate any behavioural phe- Murnighan and colleagues (1993) ran four nomena or theoretical relationship among con- experiments in a mixed-hybrid repeated meas- structs. Because of the study’s focus on ures design. The first experiment included respondents’ competence and ability to use com- Hypotheses 1 and 2 and the next three experi- puter-mediated technology (Bunz 2005), it did ments replicated these findings and made various not consider whether substantive research find- extensions. The results supported both hypothe- ings could be influenced by ARS technology use. ses: as group size increased, the proportion of In this section, we demonstrate how ARS tech- volunteers decreased; and as the payoff for volun- nology may be used for theory testing. teering decreased, the proportion of volunteers Portions to two organisational studies were decreased. There were no interactions predicted duplicated to show that ARS technology is a or observed. viable and unbiased method for collecting data. The first replicated study concerned volunteer Study 2: The