Link to Dissertation
Total Page:16
File Type:pdf, Size:1020Kb
NORTHWESTERN UNIVERSITY Ripening or rotting? Examining the temporal dynamics of team discussion and decision making in long-term teams. A DISSERTATION SUBMITTED TO THE GRADUATE SCHOOL IN PARTIAL FULFILLMENT OF THE REQUIREMENTS for the degree DOCTOR OF PHILOSOPHY Field of Media, Technology, and Society By Ilya Gokhman EVANSTON, ILLINOIS September 2020 2 ABSTRACT More than a quarter century of research finds that teams often fail to make high-quality decisions. This literature is based on observing team decisions in one-off decision making episodes, when in reality, most teams work together for an extended period of time, making repeated decisions together. Do teams improve or decline on decision making effectiveness over time? To answer this question, this dissertation contributes three studies on team decision making, examining how the processes and outcomes of team decision making evolve over time. In order to study team decision making over time, one needs parallel and comparable tasks on which to observe process and performance. And so, Study 1 developed and validated five parallel hidden profile tasks that require teams to share unique information in order to identify the optimal solution from three options. Using this newly developed battery of tasks, Studies 2 and 3 used mixed-methods to understand team decision making over time in eight 4-person teams. Study 2 was quantitative, examining the discussion and decision quality of teams during multiple sequential decision making episodes. Study 3 was qualitative, exploring the conversational dynamics over time. Quantitative analysis found that teams show an initial increase and subsequent decrease in discussion and decision quality as they work together. Additionally, a qualitative approach identified 18 themes that explain how teams make decisions and 9 additional themes that shed light on why decision making performance fluctuates over time. This dissertation highlights that teams struggle to maintain quality processes when completing decision making tasks that require them to leverage individuals’ unique information, such as hidden profiles, and provides guidance on how teams can remedy this shortfall. 3 Acknowledgements This research is based upon work supported by NASA under awards NNX16AQ48G (Project US-Russia), NNX15AM32G (Project CREWS), and 80NSSC18K0221 (Project SCALE). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Aeronautics and Space Administration. 4 Table of Contents ABSTRACT 2 Acknowledgements 3 Table of Contents 4 List of Tables 5 List of Figures 7 Executive Summary 8 CHAPTER 1 12 Hidden Profiles 14 Team Performance over Time 30 Furthering Science while Acknowledging Criticism 32 Qualitative Analysis: A Deeper Dive into Decision Making 33 Program of Research Summary 34 CHAPTER 2 35 Method 35 Contribution 44 CHAPTER 3 46 Method 56 Results 72 Discussion 92 CHAPTER 4 104 Method 106 Results 111 Discussion 148 CHAPTER 5 164 References 252 Appendix A 268 5 List of Tables Table 1 Disambiguating Terms: Previous Terms 186 Table 2 Disambiguating Terms: Revised Terms 187 Table 3 Summary of Hidden Profile Scenarios 188 Table 4 Scenario Information Distribution 189 Table 5 Descriptive of Items Retained for Phase 3 190 Table 6 Scenario Information Distribution Example for Scenarios 1-3 191 Table 7 Scenario Information Distribution Example for Scenarios 4-5 192 Table 8 Final Round of Item Set Performance 193 Table 9 General Team Process Measure Administration for Campaign 4 194 Table 10 General Team Process Measure Administration for Campaign 5 195 Table 11 Aggregation Metrics for Team Processes at Each Time Point 196 Table 12 ICC(1) Scores of Team Process Metrics 197 Table 13 Hypotheses and Corresponding Analyses 198 Table 14 Common and Unique Information Coverage 199 Table 15 Common and Unique Information Focus 200 Table 16 Common and Unique Information Consideration Coverage 201 Table 17 Common and Unique Information Consideration Focus 202 Table 18 Unique Negative and Positive Information Coverage 203 Table 19 Unique Negative and Positive Information Focus 204 Table 20 Unique Negative and Positive Information Consideration Coverage 205 Table 21 Unique Negative and Positive Information Consideration Focus 206 Table 22 Common Negative and Positive Information Coverage 207 Table 23 Common Negative and Positive Information Focus 208 Table 24 Common Negative and Positive Information Consideration Cov. 209 Table 25 Common Negative and Positive Information Consideration Focus 210 Table 26 Net Coverage Valence Preferences 211 6 Table 27 Net Focus Valence Preferences 212 Table 28 Net Consideration Coverage Valence Preferences 213 Table 29 Net Consideration Focus Valence Preferences 214 Table 30 Comparison of Highest Net Valence Selection Preference 215 Table 31 Information Coverage Percentage over Time 216 Table 32 Information Coverage Rank Scores 217 Table 33 Total Information Focus over Time 218 Table 34 Unique Information Focus Percentage over Time 219 Table 35 Unique Information Focus Percentage Rank Scores 220 Table 36 Information Consideration Coverage Percentage over Time 221 Table 37 Information Consideration Coverage Percentage Rank Scores 222 Table 38 Unique Information Consideration Focus Percentage over Time 223 Table 39 Unique Information Consideration Focus Percentage Rank Scores 224 Table 40 Selection of Best Answer over Time 225 Table 41 Selection of Best Answer Ranks Scores 226 Table 42 Performance as Avoidance of Worst Option over Time 227 Table 43 Preference Quality over Time 228 Table 44 Selected Option over Time 229 Table 45 Team Identity over Time 230 Table 46 Team Action Process over Time 231 Table 47 Team Viability over Time 232 Table 48 Team Implicit Coordination over Time 233 Table 49 Correlation Matrix of Discussion, Decision, and Process Measures 234 Table 50 Crew Decision Making Episodes Transcription Data 235 Table 51 Summary of Decision Making Components and Themes 236 Table 52 Temporal Decision Making Themes Components and Themes 237 7 List of Figures Figure 1 IPO model of team decision making 238 Figure 2 Recurring Phase Model of decision making 239 Figure 3 Team Identity Scale 240 Figure 4 Campaign 4 crew information 241 Figure 5 Campaign 5 Crew Information 242 Figure 6 Thematic analysis component and code results 243 Figure 7 Team tracking preferences 244 Figure 8 Individual information lists 245 Figure 9 Individuals not taking notes 246 Figure 10 Individuals sleeping during task 247 Figure 11 Preparing food and coffee during task 248 Figure 12 Talking while others still working 249 Figure 13 Interrupting other’s work 250 Figure 14 Observed process of decision making 251 8 Executive Summary Making quality decisions is a vital aspect of team success, whether a team of surgeons deciding between competing emergency procedures, an executive board selecting a strategic initiative to ensure an organization’s viability, or astronauts deciding which of three failing life systems should be attended to first. Team scholars have developed a rich literature on many of the tendencies, and often shortcomings, that occur when teams make decisions. However, team decision making has been studied almost exclusively using a cross-sectional design. This dissertation investigates team decision making over time. Specifically, I examined how teams exchange important information, how this information shapes their decisions, and how these functions evolve as teams work together. Chapter 1 reviews existing research on team decision making, focusing on studies utilizing hidden profiles. Hidden profiles are team decision making tasks that require teams to share unique information available to individual team members in order to identify the optimal team decision. In this literature review, I argue that information sharing, a term frequently used in decision making literature, is problematically vague; the exchange of information should be conceptualized as a broader paradigm of discussion quality that includes three distinct information processing attributes: information coverage (how much of available unique information is introduced into discussion), information focus (how much of a discussion focuses on unique information rather than information already available to all team members), and information consideration (whether individuals take into account information presented by team members that run counter to their individual option preference). Additionally, I place discussion 9 quality within the Input-Process-Output (IPO) decision making model and review antecedents of team discussion and decision quality. Chapter 2 describes the development and validation of five parallel hidden profile tasks that can be administered across a team’s life span to evaluate the processes and outcomes of team decision making over time. Task development and validation involved a five-phase process. In the first phase, five compelling space themed decision making scenarios were developed leveraging popular media and scientific publications. Tasks were designed so that each task presented teams with a best, middle, and worst option to choose from. Additionally, informational items were developed for each scenario that would shape team decision making and either support, negate, or not influence (i.e., neutral informational items) each option. The second phase tested these information items to ascertain their perceived valence and importance. This phase resulted