i
THE EFFECTS OF MULTITASKING TRAINING IN STAR CRAFT II
______
A Dissertation
Submitted to
the Temple University Graduate Board
______
In Partial Fulfillment of
the requirements for the Degree
Doctor of Philosophy
______
By
Aaron E. Ross
May 2013
Examining Committee Members:
Michael Sachs, Ph.D., Advisory Chair, Dept. of Kinesiology Casey Breslin, Ph.D., Department of Kinesiology Joseph DuCette, Ph.D., Department of Psychological, Organizational, & Leadership Studies in Education Larry Krafft, Ph.D., Department of Psychological, Organizational, & Leadership Studies in Education Mark Blair, Ph.D., External Member, Department of Psychology, Simon Fraser University
ii
©
Copyright
2013
by
Aaron E. Ross
iii
ABSTRACT
The effects of multitasking training in Star Craft II
By Aaron E. Ross
Doctor of Philosophy
Temple University, 2013
Doctoral Chair: Michael L. Sachs, PhD
This study explored the relationship between general or real-world multitasking and task specific multitasking as related to the real time strategy game Star Craft II:
Wings of Liberty (Blizzard, 2010). In addition to exploring this relationship, the research also attempted to examine the effect task specific multitasking training had within Star Craft II.
Data for the current research were collected in two phases. Phase one consisted of a pre/post-test design, with random assignment to either the gamer-control or experimental group. Participants in the experimental group were asked to complete 10 to 11 hours on the Star Craft II based Multitasking Trainer (stet_tcl, 2010) in between two, five trial blocks of the SynWin (Acivity Research Services,
2000); a computer based general multitasking measure.
Participants in the gamer-control group were asked to iv complete 10 to 11 hours of one-versus-one Star Craft II ladder matches, and complete the same pre/post-test SynWin battery. Both groups were asked to send the researcher their three most recent one-versus-one ladder matches prior to starting the assigned protocol, and three more upon completion.
Phase two participants only completed the pre-test
SynWin battery, and were assigned to either the non-gamer control or gaming-control groups based on their weekly use of PC/console games and Star Craft II play. Inclusion in the non-gamer control group required less than one hour of
PC/console gaming per week. Participants in the gaming- control group were asked to submit their three most recent one-versus-one Star Craft II ladder replays.
For the purpose of this research, the operational definition for Star Craft II multitasking was effective actions per-minute (EAPM), a subset of actions per-minute
(APM). Analysis of the gathered data from both phases of recruitment indicated a moderately strong positive relationship between SynWin scores and EAPM values (r =
0.636, p = 0.014). An evaluation of the effectiveness of the multitasking trainer was not completed due to a lack of adequate participation.
v
ACKNOWLEDGEMENTS
A long time ago in a galaxy far far away... or maybe it was just 13 years ago; I started my first semester at
Penn State University with the ‘outcome’ goal of earning my
Ph.D. in Sport Psychology. Little did I know back then how long the journey would take, or all the obstacles and adventures I would face along the way. For the sake of brevity, the journey can best be summarized in the words of
Garth Brooks, “…some of God’s greatest gifts are unanswered prayers.”
As I close one chapter of my life and begin another, I have to thank my parents for the unwavering love and support you have showered me with in all of my endeavors.
I truly would not have achieved all that I have without help from you both.
I want to thank all of the members of my committee:
Dr. Michael Sachs, Dr. Casey Breslin, Dr. Joseph DuCette,
Dr. Mark Blair, and Dr. Larry Krafft. My dissertation topic was definitely outside the norm, and your knowledge, mentorship, and support definitely helped me through the process. As I look back on this project, it is rewarding to vi have so many of the professors who have had an impact on my graduate career on my committee. Thank you!
Finally I have to thank my fiancée; Angie Mitchell.
For nearly three years you have stood by me through this process; watching GSL finals and attending BarCrafts. You have watched enough Star Craft II to know the difference between MVP the player and MVP the team, which alone is enough to make you a keeper. By the time this makes it to print, we should be well on our own journey together … boldly going where no one has gone before.
vii
TABLE OF CONTENTS
ABSTRACT...... III ACKNOWLEDGEMENTS...... V LIST OF TABLES...... IX LIST OF FIGURES...... X
CHAPTER
1. THE PROBLEM Introduction...... 1 Problem Statement...... 7 Hypotheses...... 8 Delimitations...... 8 Limitations...... 9 Conceptual and Operational Definitions...... 10
2. REVIEW OF LITERATURE Human Multitasking...... 12 Cognitive Model/Architectures...... 21 Video Game Research...... 27 Star Craft and Star Craft II Specific Literature....32
3. METHODOLGY Research Design...... 39 Participants...... 39 Instrumentation...... 40 Procedures...... 43 Data Analysis...... 47
4. RESULTS AND DISCUSSION General Descriptive Statistics...... 50 Primary Analysis...... 50 Secondary Analysis...... 54 Conclusions...... 55 Discussion...... 59 Implications for Future Research...... 66 Implications for Practice...... 68
viii
5. SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS FOR FUTURE RESEARCH Summary...... 72 Conclusions...... 74 Recommendations for Future Study...... 76
REFERENCES...... 78
APPENDECES
A. Descriptive Analysis...... 101 Analyses...... 101 Conclusions...... 104 Discussion...... 106 B. Figures...... 108 C. Consent Form...... 110 D. Demographic Questionnaire...... 117 E. Recruitment Poster...... 119
ix
LIST OF TABLES
Table Page
1. Descriptive Statistics of Gamers' Star Craft II League and Months Playing Star Craft II...... 51
2. Descriptive Statistics of Non-Gamers' Computer Use and Gamers' Star Craft II Game Play Per Week...... 52
3. Descriptive Statistics of Gamers' Hours Per Week of Star Craft II Usage...... 52
4. Descriptive Statistics of Gamers' Previous RTS and Brood War Experience...... 53
5. Gamer Pre-Test SynWin Scores...... 56
6. Non-Gamer-Control Pre-test SynWin Scores...... 57
7. Pre-Test SynWin Mean Comparisons by Trial Number.....58
8. SynWin Scores by Time and Group...... 102
9. APM Values by Time and Group...... 103
x
LIST OF FIGURES
Figure Page
1. Comparison of SynWin and Star Craft II User Interfaces...... 109 1
Chapter 1
THE PROBLEM
Introduction
Human multitasking is a term used to describe behaviors in which a person appears to perform multiple tasks simultaneously. The root of multitasking was defined in the computer engineering industry to describe a microprocessor’s ability to perform multiple tasks at once
(Abate, 2008). The processing of the human brain can be described in a computerized or mechanical fashion; however, the predominance of research on multitasking has demonstrated the challenges that humans face when they attempt to perform multiple tasks at one time (Pashler,
1994; Welford, 1952).
Recently, multitasking has become associated with research in cell phone use and driving. Many of these studies have reported proficiency deficits in dual task performance during simulated driving tests (Levy & Pashler,
2007). Recent studies on concurrent cell phone use and driving highlight the negative consequences when humans push the processing capacity of their brains past their 2 capabilities (Beede & Kass, 2006; Drews, Yazdani, Godfrey,
Cooper, & Strayer, 2009; Strayer & Drews, 2004).
Multitasking is also associated with real-time strategy (RTS) games such as Star Craft, Homeworld, Command and Conquer, and Warcraft. Real-time strategy games differ from turn-based games (i.e., chess) in that competitors are simultaneously moving pieces and implementing strategies.
In essence, real-time strategy games are attempting to replicate “real-world” scenarios.
What is Star Craft II?
Star Craft II: Wings of Liberty is the first installment in a real time strategy game trilogy developed by Blizzard Entertainment for both Microsoft Windows and
Mac OS X platforms. Star Craft II: Wings of Liberty is the sequel to the original Star Craft that was launched in
June of 1998. For over a decade, South Korea has represented the pinnacle for professional e-sports, with the primary game being Star Craft: Brood War, an expansion released in November of 1998 for the original Star Craft.
The release of Star Craft II in 2010 coincided with a giant surge in e-sports on a world-wide basis. The first year of Star Craft II saw an explosion of high paying tournaments and leagues. For the past three years the pinnacle of individual professional leagues has been the 3
Gom TV Global Star Craft II League (GSL). The GSL is based in South Korea, and consists of arguably the best 32 professional players in the world. The recent transition of the Korean Electronic Sport Association (KESPA) players from the famed Brood War leagues of the past decade to Star
Craft II has brought with it the first Star Craft II
Ongamenet Star League (OSL) season, and cemented Star Craft
II as the premiere world-wide e-sport.
While Star Craft II does not share the user base as popular PC games such as League of Legends, the game is still remarkably popular as a casual and spectator based game. Currently there are 3,934,127 active Star Craft II user accounts. Active user accounts by region: 1,448,543 users in the Americas (Former North America and Latin
America regions), 1,117,084 in Europe, 693,448 in Korea/
Taiwan, 830,670 in China, and 157,377 in Southeast Asia
(www.sc2ranks.com). While not all of these users are professional, many of them are very active members in the
Star Craft community. Teamliquid.net is the premiere
English website for Star Craft: Brood War and Star Craft
II. Routinely teamliquid.net has over 10,000 viewers on the site at any given time (www.teamliquid.net). With regards to the spectator aspect of professional Star Craft
II viewership, Nielsen reported 1.35 million unique viewers 4 in the male, 18 to 24 year old demographic for the MLG 2012
Spring Championship. Only the BCS Championship Game received a higher rating within that demographic (NFL Super
Bowl was not in the comparison) (Wilhelm, 2012).
How is Star Craft II related to multitasking?
In-game multitasking is a skill that is believed to be one of many factors related to success at all levels of
Star Craft II. Conventional thinking within the Star Craft community is that actions per-minute (APM) is an indicator of in-game multitasking or an indication of the potential for multitasking within the game. Multitasking is believed to be such a factor for improvement that many custom maps
(training games) have been developed with the sole purpose of improving Star Craft II related multitasking.
The relationship between APM and general multitasking is not completely clear. Beginning players, or bronze league players who nominally represent the bottom 20% of active participants, routinely have APM levels that are between 5 to 10% of elite professionals. As players progress in skill level, they typically are able to perform more and more tasks over the course of smaller and smaller amounts of time. As a result, professionals have APM levels in excess of 200 APM. 5
While the relationship between Star Craft II specific multitasking and a general multitasking ability is not yet understood, the literature does support the hypothesis that playing video games can enhance and/or maintain mental function. Basak, Boot, Voss, and Kramer (2008) examined the effect that RTS training had on executive control functions of older adults. Compared to the control, those participants who trained on the RTS significantly improved in task switching, working memory, visual short-term memory, and reasoning. Green and Bavelier (2003) found that video game players had significantly greater attentional capacity and task switching than non-video game players, and were able to generalize the learned visual skills to non-specific tasks.
The analysis of multitasking was viewed through the lens of Adaptive Control of Thoughts – Rationale (ACT-R)
(Anderson, 1993). Specifically, the architecture derived by Lebiere, Anderson, and Bothell (2001), which was designed to analyze simplified air-traffic control tasks, was utilized. The ACT-R model proposed by Lebiere et al.
(2001) contains three memory components: procedural memory, declarative memory, and goal stack. Procedural memory contains production rules which are comprised of condition- action pairs that determine which basic cognitive actions 6 take place under certain circumstances. Declarative memory contains knowledge structures called chunks that contain small sets of elements in labeled slots. Access to these knowledge structures is mediated through the current goal at the top of the goal stack. The goal stack contains multiple task related goals. As the importance or relevancy of the goal changes, based on changes in the environment, different procedural and declarative memories can be accessed to produce responsive actions (Lebiere,
2001).
The compatibility between the multitasking demands of
Star Craft II and the considerations taken into developing the ACT-R model (Lebiere, 2001) becomes apparent in the description of in-game multitasking presented by Plott
(2011, August 17). Plott describes the process of reprioritizing players' to-do-lists, and cycling through various goals to which players must pay attention, and the corresponding information on the screen that the players should be constantly checking as a step in their to-do- lists.
Exploring multitasking through the Star Craft II can provide a window into other domains that rely heavily on multitasking. Research into driving and multitasking has demonstrated the proficiency detriment humans face when 7 attempting to multitask (Levy & Pashler, 2007). However,
Lightman (2010) points to the flaw with driving research due to the lack of practice factored into the dual task design models that have been used. The general ACT-R model
(Anderson, 1993) and the model proposed by Liebere et al.
(2001) both allow for humans to exhibit multitasking behavior through task specific practice. Playing Star
Craft II constitutes time spent practicing multitasking specific skills. The ability to save game replays, combined with significant skill differences defined by leagues, provides researchers the opportunity to examine multitasking proficiency in both a cross-sectional and longitudinal fashion. Analyzing multitasking through Star
Craft II has the potential to provide more insight regarding the interaction between practice, skill, and experience with the ability of humans to exhibit multitasking behaviors in a wide variety of domains. This study was designed to expand upon the existing research in multitasking and explore the trainability of task specific multitasking through Star Craft II.
Problem Statement
The purpose of the study was designed to address the trainability of task specific multitasking skill while 8 exploring the relationship between task and general multitasking behaviors.
Hypotheses
The following hypotheses were examined in this study:
1. Participants in the experimental group will have significantly higher mean scores as compared to those participants in the control group across the four post-test
SynWin trials.
2. Participants in the experimental group will have significantly higher mean APM values as compared to those participants in the control group across the three post- test one-versus-one game replays.
3. There will be positive significant relationships between participants’ SynWyn scores and EAPM values at pre- test and post-test.
Delimitations
The following delimitations were present in this study:
1. Participation was primarily limited to the
Americas server (United States, Canada, Mexico, and South
America)(one participant played on the European server).
The ability to generalize results to other regions of play is limited. 9
2. Participation was limited to Temple University students and faculty and patrons of Bar Crafts in the
Philadelphia area.
3. Participants in the experimental and gaming- control groups completed the Star Craft II portions of the study at home on their own time. Despite incentives for finishing the protocol in 13 days, it was difficult to control how long it took participants to complete the study protocol.
Limitations
The following limitations were present in this study:
1. Participants in the experimental gaming groups were exclusively male. No female Star Craft II players volunteered to participate.
2. Participants were self–selected. Participant recruitment was done through online Star Craft II social communities, on campus flyers, in-class recruitment, and local Bar Crafts.
3. Participation among gamers was not equally stratified based on Star Craft II league. The number of participants in each Star Craft II league can be found in
Table 1 in Chapter 3.
10
Conceptual and Operational Definitions
For this study, the following are conceptual and operational definitions. Further Star Craft II specific terms are defined in Appendix B.
Actions Per Minute (APM): Actions per minute are defined as discrete user inputs per Star Craft II minute.
One minute in real time is equal to 84 seconds in-game time at the faster game setting (the setting used for competitive play). In patch 1.40, Blizzard changed the calculation of APM to exclude repeated unit selections, or input and selections that were deemed to be ineffective.
In Sc2gears, APM comprises two components: micro APM and macro APM (Belicza, 2011a).
Adjusted APM: Adjusted APM is the conversion of in- game APM (APM) to real time APM. Adjusted APM is calculated by dividing APM by 1.38 (Belicza, 2011a).
Effective APM (EAPM): Effective APM is a measure of the effective/useful player inputs over the course of one real time minute. The algorithm for EAPM is different from the APM reported by Blizzard after patch 1.40. According to Sc2gears, Blizzard’s interpretation of EAPM (APM post patch 1.40) “fails to consider some key aspects as to what is considered effective” (Belicza, 2011a). 11
Multitasking (human): Human multitasking describes a behavior in which a person appears to be performing multiple tasks simultaneously. Current models differ on whether human multitasking is supported by the cognitive architecture of the brain (Meyer & Kieras, 1997a, 1997b;
Meyer et al., 1995), or is simply an illusion (Anderson,
1993).
12
CHAPTER 2
REVIEW OF LITERATURE
The purpose of this study was to explore in-game and real-world multitasking through the real-time strategy
(RTS) game Star Craft II. This review of literature will be presented in the following sections: human multitasking, cognitive models/architectures, video game research, and Star Craft and Star Craft II specific literature.
Human Multitasking
Many activities that humans partake in require some level of task switching or multitasking. Preparing a multiple course meal, watching television while knitting, and driving a car while using a cell phone, are all examples of multitasking behaviors that many humans attempt on a daily basis. According to Waller (1997),
“individual-level multitasking processes involve a person's allocation of his or her own scarce cognitive resources among several tasks and the moderating impact of task elements, task processes, and task resources on individual multiple-task performance” (p. 295). 13
The first real examination of multitasking can be linked to Welford (1952) and the theory of the
Psychological Refractory Period (PRP). The psychological refractory period refers to the delay in reaction time (RT) to a second stimulus when it is presented within 0.5 sec. of the first stimulus. According to Welford (1952), the interference in RT to the second stimulus was due to a bottleneck in the central information processing stage.
The concept of a “bottleneck” in the central information processing stage that was supported by the psychological refractory period has generated the PRP design which has become the primary tool for assessing theories of dual-task performance (Pashler & Johnston, 1998).
Human multitasking is concerned with “how people perform multiple tasks either simultaneously or in rapid succession when each task involves its own distinct goals and stimulus-response associations” (Meyer & Kieras, 1997a,
1). The behavior of multitasking is particularly interesting to researchers because of the demands placed on the information-processing system, and the insight that is gained into how that system is organized and operates by studying it under duress (Atkinson, Hernstein, Lindzey, &
Luce, 1998; Meyer, & Kornblum, 1993; Posner, 1989). From a more tangible perspective, researchers have been interested 14 in multiple-task performance due to its use in important real-world jobs such as air traffic control (Lebiere,
Anderson, & Bothell, 2001) and power-plant operation (Boff,
Kaufman, & Thomas, 1986; Carvalho, dos Santos, Gomes,
Borges, & Guerlain, 2008; Wickens, 1991).
Recently multitasking has been associated with cell phone usage while driving. The first decade of the 21st century has witnessed the explosion of cell phone usage by the overwhelming majority of citizens in technologically advanced countries around the world. In the United States alone, it is estimated that there are over 320 million wireless subscriber connections utilizing 256,920 reported cell sites (CTIA, 2011). The growth in cell phone use in the United States has been related to an increase in vehicular accidents, with approximately 25% being related to cell phone use (www.edgarsnyder.com). Vehicular related multitasking research has highlighted the negative consequences associated with over-taxing the capabilities of an individual’s information processing system (Beede &
Kass, 2006; Drews, Yazdani, Godfrey, Cooper, & Strayer,
2009; Strayer & Drews, 2004).
Early models of information processing, such as single-channel hypotheses (Welford, 1952, 1959, 1980), central bottleneck models (De Jong, 1993; McCann & 15
Johnston, 1992; Pashler, 1984, 1994; Welford, 1967), and unitary-resource theories (Kahneman, 1973; Moray, 1967) provide rationale for the multitasking effects depicted in cell phone/driving studies. The key factor linking these theories is the premise that humans have a relatively limited processing capacity. However, more recent cognitive architectures proposed through Executive-Process
Interactive Control (EPIC) (Meyer & Kieras, 1997a, 1997b) and (ACT-R) (Anderson, 1993) have depicted a view of information processing that provides greater resource capacity and potential for multitasking behaviors than believed possible before.
Lightman (2010) applies the concepts of EPIC and ACT-R towards cell phone/driving literature, and points to the neglect of not taking into account the effect that practice has on developing multitasking skill in specific task domains. From a motor development standpoint, Lightman
(2010) addressed the issue of whether multitasking is a learned skill or an innate ability and challenged the common premise that humans do not have the attention or processing capacity to utilize cell phones and operate vehicles at the same time.
While there is some serious disagreement between the popular architecture (ACT-R vs. EPIC) as to how 16 multitasking actually occurs, the research is in agreement that multitasking training does, under certain conditions, lead to decreased performance costs associated with dual- task scenarios (Ruthruff, Johnston, & Van Selst, 2001;
Ruthruff, Johnston, Van Selst, Whitsell, & Remington, 2003;
Schumacher, Seymour, Glass, Fencsik, Lauber, Kieras, &
Meyer, 2001; Tombu & Jolicoeur, 2004, Van Selst, Ruthruff,
Johnston, & 1999). Behavioral research points to decreases in the cost of multitasking occurring through increased proficiency in the individual component tasks, resulting in a reduction of their interference effect (Ruthruff,
Johnston, Van Selst,(2001); Ruthruff et al., 2003).
Schumacher et al. (2001) utilized a bimodal task design (auditory-vocal and visual manual) in three related experiments to assess changes in reaction time due to training. The eight participants in experiment one were tested over five sessions, with each participant completing
2,064 trials over the course of the study. Results for visual manual (VM) and auditory-vocal (AV) tasks from session two to five depicted a decline in reaction time differences between dual task (AV and VM mixed trials) and homogenous trials (only AV or VM). By the fifth session, the difference in mean RTs for AV trials between homogenous and dual task trials was 4 + 4 ms (t(7)=0.95, p > 0.10). 17
The difference in mean RTs for VM trials between homogenous and dual task trials by session five was 11 + 10 ms
(t(7)=1.11, p > 0.10).
Van Selst et al. (1999) studied six participants over
36 practice sessions on dual-task interference using a PRP design with Task 1 being a vocal response and Task 2 being a manual response. Results from training produced a residual PRP of 50 ms on novel tasks which usually produced
300 to 400 ms PRP effects.
Ruthruff et al., (2001) build upon Van Selst et al.
(1999) to further examine the relationship between practice and reductions in dual-task interference in PRP paradigms.
Utilizing the same participants fro Van Selst et al.
(1999), Ruthruff et al. (2001) performed three experiments:
(1) new Task 1 paired with old Task 2, (2) old Task 1 with new Task 2, (3) replication of Van Selst et al. (1999), with the exception that participants made a manual rather than vocal response to Task 1 stimuli. The change in experiment three resulted in both tasks requiring a manual response. Results from experiments one and two demonstrated that prior experience on Task 1 was more beneficial than having prior experience on Task 2.
Practice on Task 1 resulted in significant reductions in
PRP effect size (F(9,36) = 5.19, p < 0.001, decrease in PRP 18 effect from 194 to 121 ms). Practice on Task 2, combined with little practice on Task 1, yielded non-significant PRP effect decreases (F(21,84)<1). Experiment three displayed a significant increase in PRP effect from Van Selst et al.
(1999) to Ruthruff et al., (2001) (exact same design, with the same participants, with Task 1 requiring a manual response instead a verbal response) (t(4) = 8.6, p < 0.001, and increase from 50 ms to 359 ms). The PRP effect in experiment three did not decrease significantly with practice (t(4) = 1.2, p > 0.2). Ruthruff et al. (2001) supported four main conclusions regarding practice and the
PRP paradigm:
(a) A processing bottleneck exists even after extensive practice; (b) the principal cause of the reduction in PRP interference with practice is shortening of Task 1 bottleneck stages; (c) a secondary cause is that one or more, but not all, of the Task 2 sub-stages that are postponed before practice are not postponed after practice (i.e., become automatized; and (d) the extent of PRP reduction with practice depends on the modalities of the 2 responses. (p. 3)
The literature points to certain multitasking conditions that do not support decreased performance costs with practice in dual-task (Bertelson & Tisseyre, 1969;
Borger, 1963; Davis, 1956; Halliday et al., 1959; Karlin &
Kestenbaum, 1968; Van Selst & Jolicoeur, 1997). According to Ruthruff et al.(2001), these studies use a manual-manual 19 design (physical stimuli paired responses) for the responses to the two stimuli in the PRP design, with nearly all studies reporting PRP effects of 200 ms or more. De
Jong (1993), as well as Ruthruff et al. (2001), both provided evidence that manual-manual designs create a response initiation bottleneck in addition to the central bottleneck.
While the majority of research into multitasking training focused on conceptual models, a more recent line of research has been directed towards the anatomical and physiological changes that occur in the brain due to training. In general, this line of research can be divided into those neural accounts that propose that training improves the efficiency of pre-existing neural circuits and those that propose training reorganizes the brain circuits supporting task performance (Jones, 2004).
Dux, Tombu, Harrison, Rogers, Tong, and Marios (2009) continued their line of research from Dux, Ivanoff,
Asplund, and Marios (2006) and Henson, Price, Rugg, Turner, and Friston (2002), and focused on the anatomical changes that occur in the prefrontal cortex. Participants performed both practice sessions, and trial sessions that were monitored through fMRI analysis as they completed both single-task or dual task trials. Blood Oxygen Level 20
Dependence (BOLD) analysis of the fMRI images taken from the pre-training, middle, and post-training sessions yielded significant differences in frontal cortex activation during single and dual task trials at pre-test
(t(3) = 3.4m p < 0.04), but not during post-test (t < 1, p
> 0.7). According to Dux et al. (2009), the decrease in activation was related to the decrease in PRP effect through a shortening of the central phase of information processing and provides neurological rationale and linkage with the findings of Ruthruff et al. (2001) and Ruthruff et al. (2003).
Currently research into human multitasking training appears to be in agreement that during dual-task scenarios that require bimodal sensory and response schedules, training appears to reduce RT. For the most part, multitasking training has been used as a device to study the organization and mechanics of the information processing system through PRP paradigm designs.
Specifically, research has focused on if, and how, the central processing bottleneck operates from a theoretical and anatomical perspective. The predominance of multitasking training has not actually been geared towards studying how to improve multitasking; as such, typically error rates are used only as a control variable, and in 21 some cases are viewed as a hindrance to the design model
(Ruthruff et al., 2001).
Cognitive Models/Architectures
The predominant models of human attention and cognitive processing propose the human brain has limited processing resources available for handling multiple tasks simultaneously. Collectively, these hypotheses are referred to as Resource models (Pashler, 1998). Resource models are subdivided into two separate categories: Unitary
Resource theories (Kahneman, 1973; McLeod, 1977) and
Multiple Resource theories (Navon & Gopher, 1979; Wickens,
2002). Unitary Resource theories propose that attention and processing are comprised of an amodal and quantifiable resource capacity. Multiple Resource theories propose that attention and processing resources are not amodal, but are separated by modality. If multiple tasks require different input and output modalities then the tasks can be performed simultaneously (Lightman, 2010).
Multitasking and driving is an area of research that has been given much attention recently. Many of these studies have reported proficiency deficits during dual task performance while driving (Levy & Pashler, 2007). Recent studies on concurrent cell phone use and driving highlight 22 the negative consequences when humans push the processing capacity of their brains past their capabilities (Beede &
Kass, 2006; Drews, Yazdani, Godfrey, Cooper, & Strayer,
2009; Strayer & Drews, 2004).
The deficit in driving proficiency and reaction time that is presented in driving/cell-phone use studies tends to support the amodal resource capacity proposed by the
Unitary Resource theories (Lightman, 2010). Unitary
Resource theories propose the existence of an amodal processing resource capacity of one response in the response selection stage (Tombu & Jolicoeur, 2003).
The existence of this processing capacity limiter in the selection processing stage is referred to as a bottleneck. The response-selection bottleneck (RSB) hypothesis is an integral piece of the Resource theories.
The RSB hypothesis is based on discrete stage theories, which propose that information flows through three linear processing stages: stimulus encoding (identification), response selection, and response execution (programming)
(Sanders, 1980; Sternberg, 1969). A response to incoming stimuli cannot be processed until the previous response has been processed and output (Schumacher et al., 2001).
Support for this information processing bottleneck has been found in dual task studies which demonstrate decreased task 23 proficiency and reaction time (Pashler, 1994; Welford,
1952).
Paramount to the Resource theories and the RSB hypothesis of human processing is the concept that not all stages have unlimited processing capacity. Modern microprocessors are comprised of multiple cores which allow for multiple tasks to be performed simultaneously.
Essentially, the RSB hypothesis proposes that human brains function as a single core processor, only allowing one stimulus response pairing to be performed at a time.
The primary challenges to the Unitary Resource theory and the RSB hypothesis are those that have evolved from the
Executive-Process/Interactive Control (EPIC) architecture
(Meyer & Kieras, 1997a, 1997b; Meyer et al., 1995). EPIC architecture does not discount the previously established information processing stages (i.e., stimulus encoding, response selection, response programming), but proposes that distinct processing units exist to retrieve sensory information based on the form of sensory modality. The sensory information is then analyzed by a cognitive processor, selects appropriate responses, and sends them to the motor processors, which prepare and output movements through the physical effectors. Unlike the Resource theories, EPIC architecture allows for the information 24 processing stages to be performed in parallel (Lightman,
2010).
EPIC and the later Adaptive Executive Control
(AEC)(Meyer & Kieras, 1997a, 1997b; Meyer et al., 1995) models propose that through sufficient practice, a transformation of declarative knowledge into procedural knowledge will take place, and that tasks comprised of different sensory modalities (e.g., auditory and visual) can be performed simultaneously with sufficient practice
(Meyer & Kieras, 1997a, 1997b; Schumacher et al., 2001).
The ability to perform simultaneous tasks that comprise different modality pairings was also supported by work conducted by Hazeltine, Ruthruff, and colleagues (Hazeltine
& Ruthruff, 2006; Hazeltine, Ruthruff, & Remington, 2006;
Hazeltine, Teague, & Ivry, 2002; Ruthruff, Hazeltine, &
Remington, 2006; Ruthruff et al., 2003).
Anderson (1993) described a production system model of cognition entitled Adaptive Character of Thought (ACT-R).
ACT-R is the successor to ACT (Anderson, 1983) that builds upon the same principles of its predecessor (Lebiere, 1998;
1999) and the perceptual-motor modules inspired by EPIC
(Byrne & Anderson, 2001). ACT-R, as with ACT, is comprised of both declarative memory and procedural memory, a goal setting structure for coordinating structures, an 25 activation-based retrieval system, and a scheme for learning new productions (Lebiere, 1998; 1999).
Conceptually, ACT-R is a production system theory that models the steps of cognition by a sequence of production rules. These production rules are charged with the coordinated retrieval of environmental information and memory (Lebiere, 2001). Various ACT-R models have been used to model tasks such as simple memory retrieval
(Anderson, Bothell, Libiere, & Matessa, 1998), visual search (Anderson, Matessa, & Lebiere, 1997), complex tasks
(e.g., learning physics)(Salvucci & Anderson, 2001), driving tasks (Salvucci, 2002), and air traffic control
(Lebiere, Anderson, & Bothell, 2001).
Lebiere et al. (2001) proposed the use of an ACT-R model to depict multitasking and cognitive workload during simplified traffic control tasks. Lebiere et al. (2001) describe their proposed model: “The newest version of ACT-
R has been designed to be more relevant to tasks that are being performed under conditions of time pressure and high information-processing demand” (p. 2).
The ACT-R model proposed by Lebiere et al. (2001) contains three memory components: procedural memory, declarative memory, and goal stack. Procedural memory contains production rules which are comprised of condition- 26 action pairs that determine which basic cognitive actions take place under certain circumstances. Declarative memory contains knowledge structures called chunks that contain small sets of elements in labeled slots. Access to these knowledge structures is mediated through the current goal at the top of the goal stack. The goal stack contains multiple task related goals. As the importance or relevancy of the goal changes, based on changes in the environment, different procedural and declarative memories can be accessed to produce responsive actions (Lebiere,
2001).
Brumby, Howes, and Salvucci (2007) proposed the use of cognitive constraint modeling (CCM); to explore dual-task trade-offs in dynamic driving tasks. Cognitive constraint modeling, described as being related to ACT-R, proposes that behavior can be predicted by finding the optimal strategies and the applicable constraints associated with human performance in a given domain (Howes, Vera, Lewis, &
McCurdy, 2004). The key difference between CCM and ACT-R, is the application of modeling within real-world settings compared to simulated tasks and environments (Brumby et al., 2007).
EPIC/AEC models (Meyer & Kieras, 1997a, 1997b) and
ACT-R (Anderson, 1993) share many similar components, but 27 what separates these models is the nature of the cognitive layers. EPIC/AEC allows for the stages of information processing to be run in parallel, while ACT-R limits the stages to serial processing (Bryne & Anderson, 2001).
Through sufficient practice, both EPIC/AEC and ACT-R can provide what appears to the human eye as multitasking; however, only EPIC/AEC provides the architecture for true multitasking.
The hypothesis that EPIC architecture proposes is extremely controversial in the literature. Ruthruff et al.
(2001) takes specific exception with Meyer and Kieras
(1997a, 1997b) and pointedly refute their claims that task practice is what leads to changes in PRP, and not instructions or motivation on the part of the participant that are critical for reducing PRP interference. The predominance of literature in the area of multitasking architecture and training utilizes PRP designs and submits to a bottleneck in the central stage of information processing.
Video Game Research
The general trend in video game research portrays the negative and violent relationships between teen and adolescents’ gaming and real world behaviors. Negative relationships between the amount of time spent video gaming 28 and school performance has been widely reported for children, adolescents, and college age students (Anderson &
Dill, 2000; Creasey & Meyers, 1986; Harris & Williams,
1985; Lieberman, Chaffe, & Robers, 1988; Roberts, Foehr,
Rideout, & Brodie, 1999; van Schie & Wiegman, 1997; Walsh,
2000).
With regards to the relationship between violent video games and aggressive behavior, a meta-analysis of 54 independent studies consistently reported that playing violent video games increases aggressive behaviors, increases aggressive cognitions, increases aggressive emotions, increases physiological arousal, and decreases pro-social behaviors (Anderson & Bushman, 2001).
Anderson’s (2004) view on violent and aggressive video games can best be summarized by his choice in quotes from
Bushman and Huesmann (2001):
The best estimate of the effect size of exposure to violent video games on aggressive behavior is about 0.26. This is larger than the effect of condom use on decreased HIV risk, the effect of exposure to passive smoke at work, and lung cancer, and the effect of calcium intake on bone mass. (p. 220)
Not all research into video game use has followed in the same negative vein. While not specifically aimed at measuring RT between video game players (VGPs) and non- video game players (NVGPs), numerous studies have 29 invariably shown that VGPs have faster reaction times
(Bialystok, 2006; Castel, Pratt, & Drummond, 2005; Clark,
Lanphear, & Riddick, 1987; Greenfield, deWinstanley,
Kilpatrick, & Kaye, 1994). According to Dye, Green, and
Bavelier (2009), while the decrease in RT that was reported in the previous studies was not unexpected, the decrease was not related to a significant decrease in accuracy, a contradiction to the prediction of speed/accuracy trade-off in choice RT in Hick’s Law (Hick, 1952). Studies exploring
RT and video game play have not stopped with relational analysis, but in a few cases have established a causal relationship between action-video-game play and decreased
RTs (Clark, Lanphear, & Riddick, 1987; Green, 2008).
A relatively large body of research has emerged examining the effect that action/first person shooter games have on cognitive and visual skill/ability (Feng, Spence, &
Pratt, 2007; Green & Bavelier, 2003, 2006a, 2006b, 2007).
Through five different studies utilizing first person shooter games (FPS), Green and Bavelier (2003) found that
VGPs had significantly greater attentional capacity than
NVGPs, and were able to generalize the learned visual skills to non-specific tasks. Green and Bevalier (2006a) found evidence that VGPs had significant improvements in visual acuity. With regards to gender differences, Feng et 30 al. (2007) found that action video game play reduced gender differences in visuospatial cognition.
Video game training has also been used to examine training effects in older adult populations (Basak, Boot,
Voss, & Kramer, 2008; Goldstein, Cajiko, Oosterbroek,
Michielsen, Van Houten, & Salvedera, 1997). Goldstein et al. (1997) found that older adults who trained for 25 hours on Super Tetris improved more than non-gamers in the
Sternberg reaction time task. Basak, Boot, Voss, and
Kramer (2008) used a highly educated group of 40 older adults with no video game playing experience to examine the effect that real-time strategy game playing had on cognitive decline. Over the course of seven to eight weeks, the participants the in experimental group completed
23.5 hours of training/game play on Rise of Nations: Gold
Edition (Microsoft Game Studios, 2004) and three trial sessions (pre-test, middle, and post-test). The three trial sessions consisted of operation span (solving mathematical problems, e.g., (9/1) + 2 = 9?), task switching (primarily measuring RT), N-Back test, VSTM,
Raven’s Advanced Progressive Matrices, stopping task, functional field of view, attentional blink, enumeration, and mental rotation. Pertinent to this study are the results from the task switching trials. A main effect by 31 group did not yield a significant difference in RTs
(F(1,34) = 2.09, p = 0.16); however, a significant difference in RTs was found between Session 2 and Session 3 for the experimental group (F(1,34) = 6.78, p = 0.01).
Kearney (2005) examined the effect that FPS games had on multitasking ability. Participants in the experimental group played Counter Strike (Valve Corporation, 2003) for two hours in a pre and post-test design with the use of the
SynWyn (Activity Research Services, 2000) as a measure of multitasking ability. Analysis indicated that two hours of
FPS play was sufficient to significantly increase multitasking ability over those in the control group who did not play any video games for the study.
Barlett, Vowels, Shanteau, Crow, and Miller (2009) expanded on and critiqued Kearney (2005) to explore the effect that game type (violent or non-violent) had on multitasking ability through the use of the SynWyn, and determined how many trials on the SynWyn would be required to reach asymptotic results. While not explored by Kearney
(2005), Barlett et al. (2009) determined that four trials for the SynWyn were required to reach asymptotic results.
With respect to the main research question, a significant main effect was found to exist for video game play and 32 multitasking (F(2,109) = 3.33, p < 0.04). There was not a significant difference in performance based on game type.
In general, video game research related to cognitive performance has found increases in visualization, tracking scanning, concentration, and selective attention. These findings are consistent across a wide variety of game types and formats (i.e., FPS, RTS, violent, and non-violent games). The interesting aspect of video game research is that there are diverging veins that point in the opposite directions regarding the virtues or pitfalls that players experience. The increased cognitive performance associated with action/FPS shooter games needs to be weighed against the reported negative behavioral and cognitive pitfalls associated with prolonged exposure to violent video games.
Star Craft and Star Craft II Specific Literature
The predominance of academic literature related to
Star Craft and Star Craft II revolves around the general themes of artificial intelligence design (Hsiesh & Sun,
2008; Machado, Fantini, & Chaimowicz, 2011; Shantia, Begue,
& Wiering, 2011, Synnaeve, & Bessiere, 2011; Weber, Mateas,
& Jhala, 2010; Yi, 2011), graphics (Filion & McNaughton,
2008), data transfer and delivery (Dainotti, Pescape, &
Ventre, 2005), and game security (i.e., map 33 hacking)(Burszrtein, Hamburg, Lagarenne, & Boney, 2001).
Currently, research pertaining directly with Star Craft in an academic realm, outside the scope of computer and software engineering, is rather limited. The major bright spot in esport and Star Craft II related research, is the ongoing SkillCraft study being conducted at Simon Fraser
University in Vancouver, Canada under the direction of Dr.
Mark Blair. The SkillCraft study is geared towards conducting the largest examination of expertise to date
(www.skillcraft.com).
Yun (2011) attempted to ascertain the performance balance between the playable races (Terran, Zerg, and
Protoss) in Star Craft II by utilizing logistical regression to examine the Global Star Craft II League (GSL) tournament play from October 2010 to March 2011. During the period of analysis, 852 games were played by 136 players on 14 different maps. According to Yun, most analysis of racial balance simply examines win ratio, but does not take into account the relative skill levels of the players or the maps the games are played. Analysis from this period of GSL tournament play indicated that racial balance was dependent on the maps being played. In general, the maps were shown to favor Terrans (T) over both
Protoss (P) and Zerg (Z) with the general trend indicating 34 that Terran had an advantage over both Protoss and Zerg
(TvP, u = 1.064+0.821, TvZ, u = 0.749 + 0.566, and PvZ, u =
-0.369 + 0.596). According to Yun (2011), because logistic regression does not have closed-form solutions of parameter distribution, a bootstrap sample of 10,000 datasets was performed with an estimation of six for each race combination. The analysis revealed: P(βTerran,Protoss > 0)~
0.839, P(βTerran,Zerg > 0) ~ 0.948, and P(βProtoss,Zerg > 0) ~
0.290.
Two exceptional posts by Phillips (September 17, 2011) and Phillips and Flew (October 29, 2011) demonstrate the level of multitasking at the professional level of Star
Craft II. In these studies, an analysis of worker production and spending quotient (SQ) was conducted.
Analysis was based on level of league play (bronze, silver, gold, platinum, diamond, master, and grand master).
Significant differences in harvester production were found between grand masters and masters, diamond and platinum, platinum and gold, and silver and bronze. For games that lasted 28 minutes, grand master players averaged 3.7 workers per minute, while bronze players averaged 1.5 workers per minute. After 28 minutes, on average, a grand masters player produces 19 more workers than a master’s level player (Phillips, September 17, 2011). 35
Spending Quotient is the calculation of players’ efficiency at spending their harvested minerals. Star
Craft II is, at its heart, an economy based RTS, which means the better you are at gaining resources and spending resources, the more of an economical advantage, and subsequently army size (i.e., supply advantage)/technology advantage (stronger, more cost effective units), you will have over your opponent. According to Phillips (September
17, 2011):
SQ does show robust differences between players of different leagues, and provides a (sorely lacking) quantitative measure of spending efficiency - a key element of macro. Furthermore, it works equally well for all 3 races (unlike measure of worker production). (Your Spending Quotient, para. 5)
Lewis, Trinh, and Kirsh (2011) analyzed 2,302 replays of Star Craft: Broodwar games played outside of Korea from
August 2010 to January 2011. Data collected from the replays included: APM, spatial variance of action (SVA),
Macro actions, Micro action, and Win state (win or loss).
Action Per-Minute analysis showed that among all comparisons within faction/match up and across win state comparisons are significant (α = 0.0056) except for TvT,
Zerg in ZvT, and Protoss in ZvP. Despite these match-ups not being significant with regards to the relationship between APM and winning the match, they did follow a 36 similar trend which indicated that APM was significantly related to success. Spatial variance of action revealed similar results to APM analysis, with all comparisons being significant (α = 0.0056) except TvT, Terran in ZvT, and
Zerg in ZvP. With regards to macro/micro action relationship and match success, no significant relationships were reported. Of interesting note, Lewis et al. (2011) mentioned multitasking training through Star
Craft as a potential direction for future research.
Whidby (2011) analyzed scouting analysis of Star Craft
II Zerg players through the use of SacOvie-Zerg (short for
“Sacrificial Overlord”) Scouting Game software. On-line participants (n = 1,173) were asked to watch professional and Grand Master level Zerg replays, and answer questions based on potential strategies the Zerg player’s opponents could possibly use based on the scouting information.
Participants were broken into two categories based on their league ranking; bronze through platinum were considered lower league players and diamond through grand master were considered higher league players. Unfortunately, Whidby’s analysis was limited to Krippendorff alpha agreement scores within league and within lower level and upper level players. Based on the graphs provided within the article, 37 different analysis of the data may have provided significant results between leagues and groups.
Star Craft related research is not just limited to the user, but has also incorporated the spectators. Cheung and
Huang (2011) examined how the relationship between the game designers, players, commentators, and crowds affected the experience for spectators of Star Craft. Through the use of an on-line qualitative design, Cheung and Huang examined articles, article comments, blog posts, comments, forum posts, videos, and video comments and identified nine personas that make-up the “spectator”: the uninformed bystander, the un-invested bystander, the inspired, the pupil, the unsatisfied, the entertained, the assistant, the commentator, and the crowd. Cheung and Huang concluded that the interaction between the various members of the
“spectator,” combined with asymmetrical information available to the spectators and the players, leads to a level of suspense for the spectator that is released through the entertaining source of the competition.
Currently, the literature pertaining to Star Craft or
Star Craft II is rather sparse where it is related to human multitasking, or even more generally human performance.
Real-time strategy games like Star Craft II are in their relative infancy when compared to turn-base strategy games 38 such as chess or checkers. What Star Craft II lacks in history, when compared to chess, it makes up for with a mechanism that allows for the potential for every game a user ever plays to be recorded and analyzed to the individual mouse click, key board stroke, and with special equipment, visual attention can be accounted for as well.
As Dr. Blair has realized, the potential to have a recording of every interaction an individual has, from the time they first attempt the game, until they stop playing, provides the potential for extraordinary access to extremely accurate longitudinal human behavior data.
39
CHAPTER 3
METHODOLOGY
The purpose of this study was to explore in-game and real-world multitasking through the real-time strategy
(RTS) game Star Craft II: Wings of Liberty. The methodology will be presented in the following sections: research design, participants, instrumentation, procedures, and data analysis.
Research Design
A within subjects design was utilized to examine the effect that multitasking specific training had on in-game and real-world multitasking.
Participants
Participants for the study consisted of 31 students
(25 males, 6 females) who ranged in age from 18 to 30 years of age (M = 23.12, SD = 3.41). All participants were either current students/faculty at Temple University or patrons at local Bar Crafts in the Philadelphia.
Participants self-identified their Star Craft II one- versus-one league of play, and were randomly assigned in a counter-balanced fashion to generate two relatively homogenous groups of gamers: experimental and gamer- 40 control. Participants in the experimental and gamer- control groups represented the first wave of testing that consisted of 11 participants (7 of whom completed the protocol). The second wave of participants self-identified themselves as those who played Star Craft II or those who played PC/console games less than one hour per week. Those participants in the second wave who played Star Craft II were placed in the non-return gamers group; participants who reported that they did not play PC/console games were placed in the non-gamer control group. For some analyses, the experimental, gamer-control, and non-return-gamer groups are combined, and are represented as the gamer group. To achieve a power of 0.80 (α2 = 0.05), 46 participants would be required for an effect size of d =
0.80 (Portney & Watkins, 2000). Due to unexpected challenges with recruitment, only 31 participants volunteered for the study.
Instrumentation
Star Craft II: Wings of Liberty
Star Craft II: Wings of Liberty (Blizzard, 2010) is the first installment in a real time strategy game (RTS) trilogy developed by Blizzard Entertainment (Irvine, CA) for both Microsoft Windows and Mac OS X platforms. Star
Craft II is an economy based RTS that offers the user both 41 single player campaigns and online multiplayer experiences.
The online multiplayer mode enables the user to choose between three different race options (Terran, Zerg, and
Protoss) and allows users to compete in one-versus-one through four versus four match-ups as well as free-for-all and user created custom games and maps.
Sc2gears 12.4
Sc2gears is a general Star Craft II utility developed as a free downloadable program that allows the user to analyze replays individually or through multiple replay searches (Belicza, 2011a). Of the many features that
Sc2gears offers, the most integral to this dissertation is the ability of the software to provide APM, EAPM, and a record of all user inputs during the course of the participants’ one-versus-one ladder matches.
SynWyn Version 1.2.33
SynWin (Acivity Research Services, 2000) is a Windows based version of the Synwork, an MS-Dos program (Elsmore,
1994) that was designed to assess an individual’s multitasking performance. SynWyn, as with the earlier
Synwork1, was designed to not emulate any specific environment, but provide elements of various work related tasks (Kearney, 2005). SynWyn is comprised of four subtasks: memory search, visual monitoring, auditory 42 monitoring, and arithmetic computation. The memory search, visual monitoring, and auditory monitoring are performed at rates determined by the researcher. In contrast, the arithmetic computations are performed at the pace of the participant (Proctor, Wang, & Pick, 1998).
Scoring for the SynWyn is comprised of a composite score for all four tasks. There is no penalty for not performing the memory search, arithmetic computations, and auditory tasks. In contrast, there is a penalty for not responding on the visual monitoring task (Proctor et al.,
1998).
Multitasking Trainer v0.941
Multitasking Trainer (stet_tcl, 2010) is a custom Star
Craft II map that is intended to improve the user’s in-game multitasking performance. The trainer consists of a rectangular shaped map with one hour-glass shaped mainland on the left and two horizontally separated islands on the right. The upper left portion of the mainland contains the computer opponent (artificial intelligence or A.I.) while the lower left portion is the starting location for the user. The upper right island has a Protoss probe (basic
Protoss harvesting unit - i.e., Santa's elves) that is controlled by the user, and Zerg zergling (most basic Zerg attack unit - i.e., human size cockroach) controlled by the 43 computer. On the lower right island, a neutral character is randomly located and obscured by a fog of war (area of the map that is obscured from view because it has not been scouted by the player).
The objective of this trainer is to constantly keep the probe away from the zergling in the upper right while building your base and army in the lower left, rescuing the neutral unit on the lower right and returning him to your base, and destroying the Terran A.I.’s command center in less than 20 minutes. Depending on the difficulty level, the game speed and aggressiveness of the A.I. increases, while the number of probe lives decreases as well as its number of hit points. Additionally, depending on the difficulty, there is an excess mineral cap that you are not allowed to exceed as well as a limit on manufacturing power
(command center energy, nexus chrono boost power, and queen power). Victory is achieved by destroying the A.I.’s command center in the allotted time period.
Procedures
Participants were recruited through the use of pamphlets distributed on campus, via social media, and local BarCraft events. If potential participants were interested in participating in the study, they were instructed to contact the researcher through e-mail or 44 phone to schedule an appointment. Participants recruited at the BarCraft event completed the pre-test protocol at the event.
At the initial contact meeting, the potential participants were informed of their rights as participants in the study and given an outline of their commitments for inclusion in the study. If the potential participants gave their consent, they were asked to complete a demographic questionnaire (see Appendix D).
Upon completion of the demographic questionnaire, the participants were brought to a computer that had the SynWin installed and loaded for use. Participants were given a brief description of the test which included a power point based picture of what the SynWin interface encompasses, and given five minutes to practice and become familiar with the program and the tasks that were to be completed. Reis and
Janseen (n.d.) utilized a three minute trial period with the SynWin prior to administering the full test. To minimize downtime for the participants and improve the portability of the measure (i.e., use at BarCrafts), the trial period was extended to the full five minutes.
The SynWin was performed on a 15 inch Lenovo ThinkPad
Edge (Lenovo, Morrisville, NC). The participants had the choice of using the centrally located touchpad, or a 45 wireless Razer Naga Epic (Razer USA Ltd., San Diego, CA).
Challenges with adjusting dots per inch (DPI) on the fly with the Naga Epic lead to a switch to the Razer Mamba (no participants utilized the touchpad) (Razer USA Ltd., San
Diego, CA). Both the Mamba and the Naga Epic utilize identical laser track technology and provide the same DPI range. To reduce the chance of disconnection error, both mice were used in the wired mode. All participants wore a
Razer Megalodon surround sound gaming headset (Razer USA
Ltd., San Diego, CA). The Megalodon was configured for the standard 2.0 digital sound instead of the 7.1 surround sound due to the lack of surround sound support for the
SynWin.
Upon completion of four SynWyn trials, participants were counterbalanced and randomly assigned to either the control or the experimental group. Barlett et al. (2009) determined that four trials of SynWin were required to reach asymptotic results. All participants who self- identified themselves as current Star Craft II gamers were asked to e-mail the researcher their three most recent one- versus-one ladder match before starting the 10 to 11 hours of training or ladder games for the study. Gaming participants were asked if they know how to recover game replays; if they were not aware of how to retrieve replays, 46 the researcher demonstrated the process for them.
Participants in the non-gaming control group were not asked to complete any other part of the study, beyond completing the demographic questionnaire and the SynWin trials.
Participants in the control group were asked to complete between 10 to 11 hours of one-versus-one online ladder matches on the Americas server for Star Craft II.
Participants were asked to save all replays.
Participants in the experimental group were asked to complete between 10 to 11 hours of use with Multitasking
Trainer. Participants were given an information sheet that included a download link for Multitasking Trainer and instructions. An e-mail containing a copy of the instructions and the download link was also sent to the participants. Participants were asked to save all replays.
The literature points to a wide range of treatment/ training time periods, with a low of two hours (Kearney,
2005) and a high of 50 hours (Dye, Green, & Bavelier,
2009). Two studies reported 10 hour methodologies (Feng,
Spence, & Pratt, 2007; Green & Bavelier, 2003). Ten to 11 hours was chose for the trial period due to the potential of asymptotic results of the SynWyn in Kearney (2005) and the impracticality and lack of resources required for the
50 hour time period utilized by Dye et al. (2009). 47
After completing the 10 to 11 hours training
(experimental group) or 10 to 11 hours of ladder play
(control group), both groups were asked to participate in a single one-versus-one ladder match and to schedule a follow-up meeting with the researcher to take place after they played the ladder match. Participants were asked to save the replay of the match and e-mail it, as well as their practice (experimental group) or ladder (control group) replays to the researcher.
When participants came back for their post-test meeting, they repeated the SynWin protocol from the pre- test, and were debriefed by the researcher. Participants who completed less than 10 hours or more than 11 hours of training or ladder games were excluded from the between group analysis.
Data Analysis
Replay Analysis
Effective Actions Per Minute (EAPM) and Actions Per
Minute (APM) data were gathered by analyzing the participants’ pre-test and post-test ladder 1v1 matches replays through Sc2gears (Belicza, 2011a). All EAPM and
APM data were converted from game-time to real-time by
Sc2gears.
48
Statistical Software
The Statistical Package for the Social Sciences (SPSS)
(International Business Machines, 2011) for Window Release
20.0 was used to analyze the data.
Hypotheses
The first hypothesis stated participants in the experimental group will have significantly higher mean scores as compared to those participants in the control group across the four post-test SynWin trials. A 2x2
Repeated ANOVA design was utilized to examine the mean score differences. The omnibus F test was used to determine the presence of a significant mean difference.
The second hypothesis stated that participants in the experimental group will have significantly higher mean APM values as compared to those participants in the control group across the three post-test one-versus-one game replays. A 2x2 Repeated ANOVA design was utilized to examine the mean score differences. The omnibus F test was used to determine the presence of a significant mean difference.
The third hypothesis stated that there will be positive significant relationships between participants’
SynWyn scores and EAPM values at pre-test and post-test. A
Pearson product-moment correlation coefficient was 49 performed to determine the relationship between pre and post-test SynWyn mean and high scores and pre and post-test
EAPM mean and high scores.
The alpha level for all statistical analyses for this study was set at 0.05 to limit the potential for a Type I error. The potential for a Type I error is directly equal to the alpha level.
50
CHAPTER 4
RESULTS AND DISCUSSION
The purpose of this study was to explore in-game and real-world multitasking through the real-time strategy
(RTS) game Star Craft II: Wings of Liberty. The results will be presented in the following sections: general descriptive statistics, primary analysis, secondary analysis, and conclusions. The results are followed by the discussion, which is geared towards integrating the findings of the present study with the current literature.
General Descriptive Statistics
Participants for this study consisted of 31 individuals (n = 31, male = 25, females = 6) from Temple
University and the Philadelphia area. The age of participants ranged from 18 to 30 years of age (M = 23.12,
SD = 3.41). The remainder of the descriptive data are presented in Tables 1, 2, 3, and 4.
Primary Analysis
The first and second hypotheses were based on exploring the difference between the experimental and gamer-control groups’ pre and post-test SynWin scores and 51 pre and post-test EAPM values. Both the first and second null hypotheses were designed to be measured by a 2 x 2
Table 1. Descriptive Statistics of Gamers’ Star Craft II
League and Months Playing Star Craft II.
League Number of Mean SD
Participants
Bronze 1 16 -
Silver 4 16 8.49
Gold 3 21.67 6.03
Platinum 8 18.50 6.95
Diamond 3 15 8.72
Masters 1 27 -
Grand 0 - -
Masters
Repeated Measures ANOVA; however, an insufficient amount of data were gathered to run this analysis in a statistically sound fashion. Despite the questionable use of an ANOVA to analyze this portion of the data set, the analyses were completed, and the results and discussion can be found in
Appendix A. The primary analysis is thus limited to the third null hypothesis. 52
The third hypothesis stated that there would be a positive statistically significant relationship between participants’ SynWyn scores and EAPM. Due to the number of
Table 2. Descriptive Statistics of Non-Gamers’ Computer Use and Gamers’ Star Craft II Game Play Per Week.
Group Number of Mean SD
Participants
Gamers 20 8.60 6.48
Non-Gamers 11 19.55 9.97
Table 3. Descriptive Statistics of Gamers’ Hours Per Week of
Star Craft II Usage.
Demographic Question Mean SD
Hours a Week 8.60 6.48
Hours a Week 1v1 Ladder 6.24 6.54
Hours a Week 1v1 Custom 1.29 2.68
Hours a Week Team Ladder 1.40 1.70
Hours a Week Watching 3.15 3.41
Professional Star Craft II
Competition
Hours a Week Watching Star Craft 2.43 2.78
II Related Programming
53 participants being close to the border between parametric and non-parametric measurements, both a Pearson product- moment correlation coefficient and a Spearman rank correlation coefficient were performed to determine the
Table 4. Descriptive Statistics of Gamers’ Previous RTS and
Brood War Experience.
Demographic Question Yes No
Played Other RTS Games 21 (100%)* 0
Played Star Craft Brood 19 (90.5%) 2 (9.5%)*
War
Played Star Craft Brood 7 (35%) 13 (65%)
War > Star Craft II
*One participant from the non-gamer-control had played Star Craft Brood War when he was younger, but no longer played PC/console games.
relationship between pre-test SynWin mean scores and pre- test EAPM mean values. Results from the Pearson correlation indicated the presence of a significant relationship between pre-test EAPM mean and pre-test SynWin mean scores (r = 0.636, p = 0.014). Contrary to the results from the Pearson correlation, the Spearman correlation yielded a non-significant result (rs = 0.516, p = 0.059).
54
Secondary Analyses
The addition of the non-gaming control group provided the opportunity to compare pre-test mean SynWin scores between gamers (experimental, gaming control, and non- return-gamer groups) and those participants who played
PC/console games less than 1 hour per week (non-gaming control). As with the primary analysis, the number of participants approached the border between parametric and non-parametric measurement. To account for both domains, a t-test and Mann-Whitney U test were utilized to assess group mean differences. Results from the t-test indicated that there was not a significant difference in pre-test mean SynWin scores between participants in the gaming and non-gaming control groups [t(29) = 0.34, p = 0.74).
Results from the Mann-Whitney U were similar, indicating that a significant difference did not exist between the two groups with regard to pre-test SynWin mean scores (U = 104, p = 0.81). Participants’ individual trial and pre-test mean SynWin scores can be found in Table 5 and Table 6.
The repeated measures aspect of the study design afforded the researcher the opportunity to examine the effect that multiple trials of the SynWin had on individual trial performance. A Repeated Measures One-Way ANOVA was conducted to analyze SynWin scores across the four pre-test 55 trials for all 31 participants. The omnibus F-test indicated the presence of a significant difference between mean SynWin scores based on trial number [F (1, 28) =
2 26.84, p< 0.01, ηp = 0.49]. Pairwise comparison of means indicated that participants across both the gamer and non- gamer-control groups scored significantly lower on the first pre-test SynWin trial as compared to the final three pre-test trials. Mean differences across the four trials based on group (gamer and non-gaming-control) were not
2 found to be significant [F (1, 28) = 0.103, p = 0.75, ηp =
0.004]. Trial means and significant pairwise comparisons can be found in Table 7.
Conclusions
The third hypothesis stated that there would positive statistically significant relationships between participants’ SynWyn scores and EAPM values at pre and post-test. The results from the Pearson (r = 0.636, p =
0.014) and Spearman (rs = 0.516, p = 0.059) correlations were contradictory, with both the Pearson and Spearman indicating moderate correlations; however, only the Pearson produced a significant relationship. These data may be negatively impacted by the third limitation presented in
Chapter 1, which addressed the lack of stratification in 56
Table 5. Gamer Pre-test SynWin Scores
Participant Trial 1 Trial 2 Trial 3 Trial 4 Pre-test
I.D Number Mean
1 601 833* 474 382 572.50
2 562 759 643 891* 713.75
3 263 607* 345 382 507.00
4 294 282 348 587* 377.75
5 290 678* 609 599 544.00
6 556* 525 404 556* 510.25
7 854 864* 618 852 797.00
8 460 558 508 609* 533.75
9 1034* 833 965 941 1030.50
10 524 787 749 793* 713.25
11 738 967* 847 959 877.75
12 518 655 676* 515 910.00
13 -** -** 705* 620 662.50
14 170 328 395* 257 287.50
15 947 1083 1114 1170* 1078.50
16 417 622 804* 637 620.00
18 155 349 613* 473 397.50
19 671 623 831* 712 709.25
22 369 867 897* 674 701.75
29 645 606 942* 826 754.75
*indicates participant’s highest pre-test SynWin score **participant deleted SynWin output from the first two trials
57
Table 6. Non-Gamer-Control Pre-test SynWin Scores
Participant Trial 1 Trial 2 Trial 3 Trial 4 Pre-test
I.D Number Mean
17 294 886 839 906* 731.25
20 403 716 646 776* 635.25
21 771 901 932 970* 709.25
23 503 770 820* 460 893.50
24 654 265 832* 436 701.75
25 125 373 326 376* 300.00
26 362 696* 668 644 592.50
27 857 909 917 966* 912.25
28 321 686 850* 655 628.00
30 324 284 565 762* 483.75
31 415 606* 479 526 506.50
*indicates participant’s highest pre-test SynWin score
Star Craft II league participants. Eight of the 20 participants in the gamer group were ranked as Platinum
League players. Considering the significance of the
Pearson correlation, the trending of the Spearman significance (p = 0.059), and the limitation and impact imposed by the over-sampling of platinum players, the researcher chose to reject the third null hypothesis.
58
Table 7. Pre-Test SynWin Mean Comparisons By Trial Number
Trial Number Trial Mean Mean Difference P Value
1 493.54 - -
2 669.07 -175.54 p < 0.01
3 694.28 -200.74 p < 0.01
4 688.44 -194.90 p < 0.01
The inclusion of the non-gamer-control participants allowed the researcher to examine the difference in general multitasking ability as measured by the SynWin. Both the independent sample t-test [t(29) = 0.34, p = 0.74)] and
Mann-Whitney U (U = 104, p = 0.81) indicated that a significant difference in pre-test SynWin mean scores did not exist between the gamer and non-gamer-control groups.
The analysis of this data set indicates that a significant difference in general multitasking ability does not exist between those participants who play Star Craft II and those participants who play PC/console games less than one hour a week.
The lack of a significant difference between the gamer and non-gamer-control allowed for the examination of trial effect on SynWin performance within the pre-test data set.
A One-Way Repeated Measures ANOVA indicated that a 59 significant difference between mean pre-test SynWin scores did occur based on trial number [F (1, 28) = 26.84, p <
2 0.01, ηp = 0.49], with trial one yielding significantly lower mean scores as compared to the three other pre-test trials. Regardless of group, participants’ SynWin scores on average did not significantly differ past the first trial within the pre-test protocol.
Discussion
This study was designed to determine the extent to which general multitasking ability and task specific multitasking are related through the use of Star Craft II.
In addition to examining the relationship between general and Star Craft II related multitasking, the research also attempted to explore the trainability of in game specific multitasking. By using Star Craft II as a vehicle, this study’s aim was to determine what aspects of multitasking can be trained in an effort to explore potential avenues for increasing skill acquisition in tasks that require high levels of multitasking (e.g., fighter pilots and air traffic controllers).
As with the best laid plans, sometimes they go awry.
In the case of this research, participant recruitment within the Star Craft community was a particular challenge.
Replication of this protocol, or the design and 60 implementation of a similar protocol, should be conducted with specific consideration to match both the intrinsic and extrinsic rewards with the participatory demands for high volumes of training time. Potential suggestions are presented in the section entitled: Recommendations for
Future Study.
Due to the lack of participation in the initial phase of recruitment, analysis was limited to the third hypothesis which addressed the relationship between real- world or general multitasking ability and in-game (Star
Craft II) or task specific multitasking. While this limitation did not allow for the analysis of training impact on Star Craft II multitasking, the analysis of the third hypothesis did generate interesting data regarding the skill versus ability aspects of multitasking. Current work by Lightman (2010) addresses this question through the analysis of cell phone use and driving literature, challenging the notion that practice and training is at the heart of cell phone/driving accidents, not attention capacity.
The skill versus ability question regarding multitasking is related to the general research on ability as it pertains to motor performance across multiple tasks.
Ability research can be placed into three general buckets: 61 general motor ability, specific motor abilities (Henry,
1961, 1968), and motor abilities (Fleishman, 1964, 1965,
1967). Of note, and related to the results, is the motor abilities research from Fleishman (1964, 1965, 1967), which utilized factor analysis to progress the concept of Henry’s many specific abilities (over 1,000) to groups of abilities that are related to many different tasks. Analysis of the relationship between the pre-test mean EAPM and SynWin scores revealed a moderate positive correlation (r = 0.636, p = 0.014), indicating that a general multitasking ability may underlie Star Craft II related multitasking.
The concept of a general multitasking ability, as supported by Elsmore through the development of the
Synworks1 (1994) and the SynWin (2000), and supported through the extensive use of both instruments in the literature (e.g., Barlett et al., 2009; Hambrick et al.,
2011; Kearney, 2005; Proctor, Wang, & Pick, 1998;), generates an interesting conflict within the study of multitasking; partially due to the express limitations placed on attention capacity in the predominance of cognitive architectures since Welford (1952, 1958, 1980).
Besides EPIC architecture (Meyer & Kieras, 1997a, 1997b), nearly every other cognitive theory on human multitasking proposes that general multitasking does not exist. Despite 62 what our brains may perceive, the general notion in cognitive psychology is that multitasking is simply rapid task and attention switching that is related to specific tasks and improved with skill development.
The effect that practice and training has on task and specific multitasking is highlighted by Lightman (2010), and is represented by much of the ACT-R literature; particularly those architectures modeling specific tasks
(Anderson, Bothell, & Lebiere, 1998; Anderson, Matess, &
Lebiere, 1997; Lebiere, Anderson, & Bothell, 2001;
Sallvucci, 2002; Salvucci & Anderson, 2001). Despite being hampered by the lack of participants, the analysis of the first hypothesis indicated the presence of a "significant" difference between pre and post-test SynWin trials, describing a trend indicating the presence of a designed training effect, or some other confounding factor (analyses included in Appendix A). Potentially, after initial trials, the SynWin is transferred from a general multitasking measure to a specific task that can be mastered. In essence, the test/re-test reliability of the general multitasking measure seemed to support the concept that multitasking is task specific!
The inclusion of the non-gamer control participants afforded the opportunity for the examination of 63 multitasking differences between Star Craft II gamers and non-gamers. Analysis of the data indicated there was not a significant difference between gamers and non-gamers SynWin pre-test scores [t(29) = 0.34, p = 0.74); U = 104, p =
0.81]. On the surface, the data are not supported by the previous literature examining the effect of video game play on cognitive function (i.e., Castel, Pratt, and Drummond,
2005; Green, 2008; Green & Bavelier, 2003, 2006a, 2007).
Particularly, the presented data do not seem to support the superior performance of expert action-gamers to non-gamers in attention control, executive control, and memory in
Boot, Kramer, Simons, Fabiani, and Gratton (2008). It should be noted that the expertise of the gamer participants in the current study could be questioned.
Only one of the gamer participants was ranked as 'Masters'
(top two percent of active accounts), with 16 of the 20 participants being ranked platinum or lower (bottom 80% of active accounts). While the mean hours per week use of
Star Craft II was 8.6 hours (SD = 6.48 hrs.), the level of play would typically be considered average for the Star
Craft community.
As noted earlier, the literature supports the use of the SynWin in multitasking research (Barlett et. al., 2009,
Chen, 2010; Kearney, 2005; Proctor, Wang, & Pick, 1998; 64
Rabinowitz, Breitbach, & Warner, 2009). However, two issues should be addressed before its continued use with a gaming population, particularly gamers who play Star Craft
II. First, a pre/post-test protocol, if possible, should be developed, and the resulting retest reliability should be determined.
The second issue, which may be related to the first, and could possibly present a confounding variable in this research, is the similarity in layout of the SynWin to the
Star Craft II user interface. The SynWin display is broken into four quadrants representing four different tasks. The upper right quadrant is a randomly generated arithmetic problem that requires the addition of two separate three digit numbers. The lower left quadrant is comprised of a visual recognition task that requires the user to monitor a fuel gauge as it runs down from 100 to zero. The user must click on the gauge to reset to 100 before it hits zero (See
Appendix C).
The user interface for Star Craft II displays numerical data relating the user's supply and economy in the upper right portion of the screen. Utilizing simple math, users are able to calculate how many, and what types, of units and buildings they can produce. The bottom left portion of the user interface provides the user with a mini 65 map, which provides an overview of the entire gaming map.
As with the gas gauge in the SynWin, the mini map requires frequent visual attention (See Appendix C). With these similarities between the user interfaces for the SynWin and
Star Craft II, it is not surprising that the participant with the highest mean SynWin score was also the highest one-versus-one ranked player that participated. It should also be noted that as this individual was setting the highest score on the SynWin, he was also attempting to have a conversation with the researcher!
While the training aspect of the study was not fully realized as intended, some good verbal feedback was gathered from the participants that may be useful for future application and research. The primary challenge reported with the multitasking training was that 10 hours was too much, and the trainer was boring. While 10 hours of dedicated training may not be an issue for an elite performer, athlete, or gamer, it appears to be an issue with those who play Star Craft II in a casual fashion.
Multitasking in Star Craft II can generally be broken down into two categories: those tasks that are required to successfully execute a build order ("macro" - managing your economy and unit and building production) and those tasks that are required to respond to and execute offensive and 66 defensive maneuvers ("micro" - unit control). While many casual gamers and amateurs have difficulty handling all the tasks required in one of those categories, highly skilled professionals are able to execute both simultaneously.
Referencing back to the participant who had the highest
SynWin score, he mentioned that his ability to execute his
"macro" and respond to the predictable actions of the multitasking trainer improved; however, he felt that his actual Star Craft II skill declined. While this opinion is only anecdotal from one participant, it does raise the issue of appropriateness of multitasking training based on the individual's stage of learning (i.e., verbal cognitive, associative, and autonomous).
Implications for Research
The evolution of this project generated both design and research advisories for future studies involving multitasking and gamer participants. Primary among the advisory points is the consideration of participant recruitment in training studies involving gamers. Training studies by their nature are time consuming, and generally involve smaller sample sizes (i.e., Schumacher et al.,
2001); however, to analyze the data in a statistically sound fashion, efforts should be made to recruit and sustain sample sizes more in-line with those dictated by 67 the power analysis. To accomplish higher levels of recruitment and sustainment, future researchers should consider both intrinsic and extrinsic motivators.
With regards to extrinsic motivation, an issue with this study was the mistake in offering entrance into a raffle for gaming mice as a form of compensation.
Considering the 10 hour training period, a more effective compensation would have been providing all participants who completed the study with a gaming mouse, or some other appropriate form of material or financial compensation.
Recruiting gamers, specifically Star Craft II gamers, fortunately provides a population that is very supportive of the game and the greater gaming community.
Unfortunately, while many Star Craft II gamers enjoy playing and improving, many participants in this study reported the 10 hours of training was too long and very boring. Pairing the 10 hours of training with entrance into a tournament among participants, may be a way to leverage the gamers’ intrinsic motivation for competition as a means to recruit and sustain participation.
During the design phase of this study, it was decided to use EAPM as measure of in-game multitasking instead of
APM in an effort to remove actions that are not directly related to initiating new actions or responding to new 68 stimuli. While there is no definitive measure of in-game multitasking in Star Craft, it was hypothesized that analyzing the actions performed by participants during engagements in their 1v1 replays matches would provide periods that would require and demonstrate the relative highest levels of multitasking. At the time, analysis of user actions during engagements required manual reading of the user command prompts, which was deemed to not be practical. Any future endeavors into examining user actions during engagements should consider developing third party software to analyze command prompts in a more time economical fashion.
Future research into Star Craft II multitasking behavior should be cognizant of the difference between reactionary multitasking and pre-planned multitasking. In
Star Craft II terminology, exploring these two forms of multitasking behaviors can be found in actions of a player responding to a multipronged attack (reactionary multitasking), or a player carrying out a coordinated, multi-pronged attack (or also moving through a production cycle) (pre-planned). The difference between the two forms of multitasking behavior should be considered, because the gamer who is initiating the multi-pronged attack generally does not have to deal with the cognitive challenges 69 associated with responding to multiple stimuli presented in close proximity with respect to time.
Implications for Practice
While the training portion of the study did not meet the participant requirements, some general recommendations can still be made regarding training multitasking in Star
Craft II. For beginner to intermediate players who are lacking in game mechanics, a good place to start their training would be to watch Day[9] Daily #340 (Plott, 2011,
August 17), where Plott analyzes and explains the process behind fundamental multitasking in Star Craft II. While
Plott (2011, August 17) does not recommend the use of a multitasking trainer, the concepts discussed in the episode are transferrable across multitasking trainers, other training schedules, and during actual game play.
Consideration for a player’s skill level should be made before engaging in multitasking training. Two participants from the study that had obtained a Masters ranking in 1v1, had found the training to be somewhat detrimental to their game play because they found themselves focusing on basic tasks they had already somewhat mastered instead of the higher order aspects of game play such as overall strategy and unit control. From a perspective of implementation, it appears that multitasking 70 training should be utilized with beginning players as a means to teach them the fundamental production cycles that require manual dexterity for striking appropriate keys and rapid, process oriented switches in attentional focus.
After players have demonstrated a level of proficiency with the basic fundamentals of the game (e.g., production cycles, efficient build orders, resource management), players can then start taking advantage of the psychological refractory period that results from the cognitive bottleneck in the central stage of information processing (Welford, 1952). Developing strategies that utilize pre-planned multitasking (multi-pronged coordinate attacks), players can over-tax the attention and processing capacity of their opponents. While it is important for players to develop an understanding for the advantage of pre-planned multitasking strategies, gamers should consider mastering fundamentals before relying on creative strategies. Routinely players have hit plateaus in their game play because they have not mastered the basic macro oriented multitasking foundational skills.
The afore mentioned plateau is most certainly related to the concept of practice regimens; more specifically deliberate practice. While there is presently no literature reviewing the detailed practice regimens of Star 71
Craft II players, the lack of participation in the present study would tend to support the notion that a large percentage of players do not engage in deliberate practice.
Deliberate practice is defined as a “highly structured activity, the explicit goal of which is to improve performance. Specific tasks are invented to overcome weaknesses, and performance is carefully monitored to provide cues for ways to improve it further” (Ericsson,
Krampe, & Tesch-Romer, 1993, p. 368). Deliberate practice is one of the hallmarks of expertise (Ericsson et al.,
1993), and a staple found in many of the Day[9] dailies devoted to improving Star Craft II game play. Multitasking training, whether it occurs in a fashion presented by
Day[9]in daily #340 (Plott, 2011, August 17) or through
Multitasking Trainer v0.941 (stet_tcl, 2010), are both examples of deliberate practice which have a place in the training regimens of Star Craft II gamers in the lower ranks of 1v1 league play.
72
CHAPTER 5
SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS
FOR FUTURE STUDY
The purpose of this study was to explore in-game and real-world multitasking through the real-time strategy
(RTS) game Star Craft II: Wings of Liberty. This chapter will be presented in the following sections: summary, conclusions, and recommendations for future study.
Summary
Participation for this study consisted of 31 individuals (n = 31, male = 25, females = 7) from Temple
University and the Philadelphia area. The age of the participants ranged from 18 to 30 years of age (M = 23.12,
SD = 3.41)
Multitasking was measured in two distinct fashions for this study: general multitasking ability and in-game multitasking. General multitasking ability was measured through the use of the SynWin (Activity Research Services,
2000); a computer based generic work environment. In-game multitasking was tracked for the gamer participants from their submitted Star Craft II one-versus-one game replays.
Game replay data were processed through Sc2gears (Belicza, 73
2011a) to obtain the participants' effective actions per minute (EAPM).
Phase 1 of data collection consisted of a pre and post-test design where participants completed a pre-test
SynWin battery (consisted of one practice and four trials), a demographic questionnaire, and submitted their three most recent one-versus-one Star Craft II ladder match game replays. Upon completing the pre-test, participants were assigned to either the experimental or gamer-control groups. Participants in the experimental group were instructed to complete between 10 and 11 hours of game play on the Multitasking Trainer v0.95 custom map (stet_tcl,
2010). Participants in the gamer-control group were instructed to complete between 10 and 11 of one-versus-one ladder matches. Upon completion of the training/ladder matches, participants were instructed to play three more one-versus-one ladder matches and forward the replays of the games to the researcher prior to their post-test appointment. For the post-test appointment, participants followed the same SynWin battery protocol as the pre-test.
Recruitment for phase 1 lasted from March 2012 to August
2012. Twelve participants completed the pre-test portion of the study, with only seven completing the whole protocol. 74
Due to the lack of participation, a second phase of data collection was introduced with the goal of collecting data utilizing only the pre-test protocol. Participants included those who played Star Craft II (non-return gamers) and those who played PC and console games less than one hour a week (non-gamer control). Data collection for the second phase lasted from August 2012 to October 2012 and included 11 participants in the non-gamer control and nine participants in the non-return gamer group.
The inability of the researcher to meet the numerical participant demands of the original study design limited the analysis of the three original hypotheses to only the third: There will be positive significant relationships between participants’ SynWyn scores and EAPM values at pre- test and post-test. To examine the third hypothesis, both a Pearson product-moment correlation coefficient and a
Spearman rank correlation coefficient were completed to determine the relationship between pre-test SynWin mean scores and pre-test EAPM mean values.
Conclusions
The third hypothesis stated that there would be a positive significant relationship between participants’
SynWyn scores and EAPM. The results from the Pearson (r =
0.636, p = 0.014) and Spearman (rs = 0.516, p = 0.059) 75 correlations were contradictory, with both the Pearson and
Spearman indicating moderate correlations; however, only the Pearson produced a significant relationship. These data may be negatively impacted by the third limitation, which addressed the lack of stratification in Star Craft II league participants. Eight of the 20 participants in the gamer group were ranked as Platinum League players.
Considering the significance of the Pearson correlation, the trending of the Spearman significance (p = 0.059), and the limitation and impact imposed by the over-sampling of platinum players, the researcher chose to reject the third null hypothesis.
The inclusion of the non-gamer-control participants allowed the researcher to examine the difference in general multitasking ability as measured by the SynWin. Both the independent samples t-test [t(29) = 0.34, p = 0.74)] and
Mann-Whitney U (U = 104, p = 0.81)indicated that a significant difference in pre-test SynWin mean scores did not exist between the gamer and non-gamer-control groups.
The analysis of this data set indicates that a significant difference in general multitasking ability does not exist between those participants who play Star Craft II and those participants who play PC/console games less than one hour a week. The interpretation of this result should be 76 tempered, as the impact of the homogenous nature of skill level with in the gamer participants is not known, and warrants future research.
The lack of a significant difference between the gamer and non-gamer-control allowed for the examination of trial effect on SynWin performance within the pre-test data set.
A One-Way Repeated Measures ANOVA indicated that a significant difference between mean pre-test SynWin scores did occur based on trial number [F (1, 28) = 26.84, p <
2 0.01, ηp = 0.49], with trial one yielding significantly lower mean scores as compared to the three other pre-test trials. Regardless of group, participants’ SynWin scores on average did not significantly differ past the first trial within the pre-test protocol.
Recommendations for Future Study
The following recommendations are made for future study:
1. Before a replication of Phase 1 of this study is attempted, an examination of the test/re-test validity for the SynWin should be determined.
2. To increase Pre/post-test retention for training protocols with this population, appropriate compensation must be considered. For example, instead of using a raffle 77 system for gaming mice, it may be necessary to provide all participants who complete the study with a gaming mouse.
3. To provide better control over the participants, and the data, a one-time in-lab data collection covering two days for all participants should be considered. A possible design could include SynWin pre-testing, three one-versus-one ladder games followed by five hours of laddering (control group) or multitasking training
(experimental group)on day one. Day two would include five more hours of laddering or multitasking training, followed by three one-versus-one ladder games, and the SynWin post- testing.
4. If future research follows this study’s model of allowing the participants to complete the training and laddering in their own home, then efforts should be made to generate an internet based version of the SynWin, or a similar online test should be developed.
5. Efforts should be made to recruit a balanced Star
Craft II gaming population that reflects the various skill levels.
78
REFERENCES
Abate, C.J. (2008). You say multitasking like it’s a good
thing. The NEA Education Journal, 7-13.
Acivity Research Services (2000) SynWin [computer
software]. Chula Vista, CA.
Anderson, C.A. (2004). An update on the effects of playing
violent video games. Journal of Adolescence, 27(1),
113-122.
Anderson, C.A., & Bushman, B.J. (2001). Effects of violent
games on aggressive behavior, aggressive cognition,
aggressive affect, physiological arousal, and pro-
social behavior: A meta analytic review of the
scientific literature. Psychological Science, 12, 353-
359.
Anderson, C.A., & Dill, K.E., (2000). Video games and
aggressive thoughts, feelings, and behavior in the
laboratory and life. Journal of Personality and Social
Psychology, 78, 772-790.
Anderson, J.R. (1993). Rules of the mind. Hillsdale, NJ:
Erlbaum.
Anderson, J.R., Bothell, D., Lebiere, C., & Matessa, M.
(1998). An integrated theory of list memory. Journal
of Memory and Language, 38, 341-380. 79
Anderson, J.R., Matessa, M., & Lebiere, C. (1997). ACT-R:
A theory of higher level cognition and its relation to
visual attention. Human Computer Interaction, 12(4),
439-462.
Atkinson, R.C., Hernstein, R.J., Lindzey, G., & Luce, R.D.
(1998). Steven’s handbook of experimental psychology
(2nd Ed.). New York: John Wiley.
Barlett, C.P., Vowels, C.L., Shanteau, J., Crow, J., &
Miller, T. (2009). The effect of violent and non-
violent computer games on cognitive performance.
Computer in Human Behavior, 96-102.
Basak, C., Boot, W.R., Voss, M.W., & Kramer, A.F. (2008).
Can training in a real-time strategy game attenuate
cognitive decline in older adults? Psychology of
Aging, 23(4), 765-777.
Beede, K.E., & Kass, S.J. (2006). Engrossed in
conversation: The impact of cell phones on simulated
driving performance. Accident Analysis and Prevention,
38(2), 415-421.
Belicza, A. (2011a). Sc2gears. (Version 8.1) [Software].
Available from http://www.sc2gears.com
Bertelson, P., & Tisseyre, F. (1969). Refractory period of
c-reactions. Journal of Experimental Psychology, 79,
122-128. 80
Bialystok, E. (2006). Effect of bilingualism and computer
video game experience on the Simon task. Canadian
Journal of Experimental Psychology, 60, 68-79.
Blizzard. (2010). Star craft II: Wings of liberty
[computer software]. Irvine, CA.
Boff, W.A.K., Kaufman, L., & Thomas J. (1986). Temporal
sensitivity. Handbook of perception and human
performance, 1, 6-1.
Boot, W.R., Kramer, A.F., Simnons, D.J., Fabiani, M., &
Gratton, G. (2008). The effects of video game playing
on attention, memory, and executive control. Acta
Psychologica, 129, 387-398.
Borger, R. (1963). The refractory period and serial choice
reactions. Quarterly Journal of Experimental
Psychology, 15, 1-12.
Brumby, D.P., Howes, A., & Salvucci, D.D. (2007). A
cognitive constraint model of dual-task trade-offs in
a highly dynmaic driving task. In the Proceedings of
the SIGCHI Conference on Human Factors in Computing
Systems (CHI 2007) (pp. 233-242). New York, NY: ACM
Press.
81
Bursztein, E., Hamburg, M., Lagarenne, J., & Boneh, D.
(2011). OpenConflict: Preventing real time map hacks
in online games. Proceedings from The 2011 IEEE
Symposium on Security and Privacy (SP), Stanford, CA.
Byrne, M.D., & Anderson, J.R. (2001). Serial modules in
parallel: The psychological refractory period and
perfect time-sharing. Psychological Review, 108(4),
847-869.
Bushman B.J, & Huesmann L.R. 2001. Effects of televised
violence on aggression. In D. Singer & Singer (Eds.),
Handbook of children and the media, (pp. 223–54).
Thousand Oaks, CA: Sage.
Carvalho, P.V.R., dos Santos, I.L., Gomes, J.O., Borges,
M.R.S., & Guerlain, S. (2008). Human factors approach
for evaluation and redesign of human-systems
interfaces of a nuclear power plant simulator.
Displays, 29(3), 273-284.
Castel, A.D., Pratt, J., & Drummond, E. (2005). The effects
of action video game experience on the time course
inhibition of return and the efficiency of visual
search. Acta Psychologica, 119, 217-230.
82
Chen, H.T.M (2010). Personality characteristics and
multitasking-related assessments as supplemental
predictors of a-school training performance and
success among naval hospital corpsmen. [Unpublished
doctoral dissertation]. The University of Memphis,
Memphis, TN.
Cheung, G., & Huang, J. (2011). Star Craft from the stands:
Understanding the game spectator. Proceedings of the
2011 Annual Conference on Human Factors in Computing
Systems, Vancouver, BC, Canada.
Clark, J.E, Lanphear, A.K., & Riddick, C.C. (1987). The
effects of videogame playing on the response selection
processing of elderly adults. Journal of Gerontology,
42, 82-85.
Creasey, G.L., & Meyers, B.J. (1986). Video games and
children: Effects on leisure activities, schoolwork,
and peer involvement. Merrill-Palmer Quarterly, 32,
251-262.
CTIA. (2011). CTIA semi-annual wireless industry survey.
Retrieved from
http://files.ctia.org/pdf/CTIA_Survey_MY_2011_Graphics
83
Dainotti, A., Pescape, A., & Ventre, G. (2005). A packet-
level traffic model of Star Craft. Proceedings from
the Second International Workshop on Hot Topics in
Peer-toPeer Systems, 2005. Naples, Italy.
Davis, R. (1956). The limits of the “Psychological
Refractory Period.” Quarterly Journal of Experimental
Psychology, 8, 24-38.
De Jong, R. (1993). Multiple bottlenecks in overlapping
task performance. Journal of Experimental Psychology:
Human Perception and Performance, 19, 965-980.
Drews, F.A., Yazdani, H., Godfrey, C.N., Cooper, J.M., &
Strayer, D.L. (2009). Text messaging during simulated
driving. Human Factors, 51(5), 762-770.
Dux, P.E., Tombu, M.N., Harrison, S., Rogers, B.P., Tong,
F., & Marios, R. (2009). Training improves
multitasking performance by increasing the speed of
information processing in human prefrontal cortex.
Neuron, 63(1), 127-138.
Dux, P.E., Ivanoff, J., Asplund, C.L., & Marios, R. (2006).
Isolation of a central bottleneck of information
processing with time-resolved fMRI. Neuron, 52(6),
1109-1120.
84
Dye, M.W.G., Green, C.S., & Bavelier, D. (2009). Increasing
speed of processing with action video games. Current
Direction in Psychological Science, 18(6), 321-326.
Elsmore, T. (1994). Synwork1: A pc-based tool for
assessment of performance in a simulated work
environment. Behavior Research Methods, Instruments &
Computers, 26(4), 421-426.
Ericsson, K.E., Krampe, R.T., & Tesch-Romer, C. (1993). The
role of deliberate practice in the acquisition of
expert performance. Psychological Review, 100(3), 363-
406.
Feng, J., Spence, I., & Pratt, J. (2007). Playing action
video game reduces gender difference in spatial
cognition. Psychological Science, 18(10), 852-855.
Filion, D., & McNaughton, R. (2008). Effects & techniques.
In N. Tatarchuk (Eds.), Advances in real-time
rendering in 3D graphics and game course – SGGRAPH
2008 (133-164). New York, NY: ACM.
Fleishman, E.A. (1967). Individual differences and motor
learning. In R.M. Gagne (Ed.), Learning and individual
differences (pp. 165-191). Columbus, OH: Merrill.
85
Fleishman, E.A. (1965). The description and prediction of
perceptual-motor skill learning. In R. Glaser (Ed.),
Training researcher and education (pp. 137-175). New
York: Wiley.
Fleishman, E.A. (1964). The structure and measurement of
physical fitness. Englewood Cliffs, NJ: Prentice Hall.
Goldstein, J., Cajiko, L., Oosterbroek, M., Michielsen, M.,
Van Houten, O., & Salvedera, F. (1997). Videogames and
the elderly. Social Behavior and Personality, 25, 345-
352.
Green, C.S. (2008). The effects of action video game
experience on perceptual decision making. Unpublished
doctoral dissertation, University of Rochester,
Rochester, NY.
Green, C.S., & Bavelier, D. (2003). Action video game
modifies visual selective attention. Nature, 423, 534–
537.
Green, C.S., & Bavelier, D. (2006a). Effect of action
video games on the spatial distribution of
visuospatial attention. Journal of Experimental
Psychology: Human Perception and Performance, 1465–
1468.
86
Green, C.S., & Bavelier, D. (2006b). Enumeration versus
multiple object tracking: The case of action video
game players. Cognition, 101, 217–245.
Green, C.S., & Bavelier, D. (2007). Action video game
experience alters the spatial resolution of attention.
Psychological Science, 18, 88–94.
Greenfield, P.M., de Winstanley, P., Kilpatrick, H., &
Kaye, D. (1994). Action video games and informal
education: Effects on strategies dividing visual
attention. Journal of Applied Developmental
Psychology, 15, 105-123.
Hambrick, D.Z., Rench, T.A., Poposki, E.M., Darowski, E.
S., Roland, D., Bearden, R.M., Oswald, F.L., & Brou,
R. (2011). The relationship between the ASVAB and
multitasking in Navy Sailors: A process-specific
approach. Military Psychology, 23, 365-380
Halliday, A.M., Kerr, M., & Elithorn, A. (1959). Grouping
of stimuli and apparent exceptions to the
psychological refractory period. Quarterly Journal of
Experimental Psychology, 11, 72-89.
Harris, M.B., & Williams, R. (1985). Video games and school
performance. Education, 105(3), 306-309. 87
Hazeltine, E., & Ruthruff, E. (2006). Modality pairing
effects and the response selection bottleneck.
Psychological Research, 70, 504-513.
Hazeltine, E., Ruthruff, E. Remington, R.W. (2006). The
role of input and output modality pairings in dual-
task performance: Evidence for content-dependent
central interference. Cognitive Psychology, 52, 291-
395.
Hazeltine, E., Teague, D., & Ivry, R. B. (2002).
Simultaneous dual-task performance reveals parallel
response selection after practice. Journal of
Experimental Psychology: Human Perception and
Performance, 28,527–545.
Henry, F.M. (1968). Specificity vs. generality in learning
motor skill. In R.C. Brown & G.S. Kenyon (Eds.)
Classical studies on physical activity (pp. 341-340).
Englewood Cliffs, NJ: Prentice Hall. (Original work
published in 1958)
Henry, F.M., (1961). Reaction time – movement time
correlations. Perceptual and Motor Skills, 12, 63-66.
88
Henson, R.N.A., Price, C.J., Rugg, M.D., Turner, R., &
Friston, K.J. (2002). Detecting latency differences in
event related BOLD responses: Application to words
versus nonwords and initial versus repeated face
presentations, Neuroimagine, 15, 83-97.
Hick, W.E. (1952). On the rate of gain of information.
Quarterly Journal of Experimental Psychology, 4, 11-
26.
Howes, A., Vera, A.H., Lewis, R.L., & McCurdy, M.P. (2004).
Cognitive constraint modeling: A formal approach to
supporting reasoning about behavior. In Proceedings of
the 26th Annual Meeting of the Cognitive Science
Society (pp. 595-600). Mahwah, NJ: Lawrence Earlbaum
Associates, Inc.
Hsieh, J.L., & Sun, C.T. (2008). Building a player strategy
model by analyzing replays of real-time strategy
games. Proceedings from IJCNN 2008: IEEE
International Joint Conference on Neural Networks,
2008. (IEEE World Congress of Computational
Intelligence). Hong Kong, China.
International Business Machines Corporation. (2011). IBM
SPSS Statistics for Windows, Version 20.0. Armonk, NY.
Jones, J. (2004). How does practice make perfect. Nature
Neuroscience, 7, 10-11. 89
Kahneman, D. (1973). Attention and effort. Englewood
Cliffs, NJ: Prentice-Hall.
Karlin, L., & Kestenbaum, R. (1968). Effect of number of
alternative on the psychological refractory period.
Quarterly Journal of Experimental Psychology, 20, 167-
178.
Kearney, P.R. (2005). Cognitive calisthenics: Do FPS
Computer games enhance the player’s cognitive
abilities? Proceedings of DIGRA 2005 Conference:
Changing Views-Worlds in Play. Retrieved from
http://www.digra.org/dl/db/06276.14516.pdf
Lebiere, C., Anderson, J.R., & Bothell, D. (2001). Multi-
tasking and cognitive workload in an ACT-R model of
simplified air traffic control task. Proceeding from
The tenth Conference on Computer Generated Forces and
Behavioral Representation. Retrieved from http://act-
r.psy.cmu.edu/papers/233/10TH-CGF-LAB.pdf
Lebiere, C., & West, R.L. (1999). A dynamic ACT-R model of
simple games. In Proceedings of the Twenty-first
Conference of Cognitive Science Society, pp. 296-301.
Mahwah, NJ: Erlbaum.
90
Lebiere, C. (1998). The dynamics of cognition: An ACT-R
model of cognitive arithmetic. Ph.D. Dissertation. CMU
Computer Science Department Technical Report CMU-CS-
98-186. Pittsburgh, PA.
Lewis, J.M., Trinh, P., & Kirsh, D. (2011). A corpus
analysis of strategy video game play in Star Craft:
Broodwar. In Proceedings of the 33rd Annual Conference
of the Cognitive Science Society, 687-692.
Levy, J., & Pahsler, H. (2007). Task Prioritization in
multitasking during driving: Opportunity to abort
concurrent task does no insulate braking responses
from dual-task slowing. Applied Cognitive Psychology,
22, 507-525.
Levy, J., Pashler, H., & Boer, E. (2006). Central
interference in driving: Is there any stopping the
psychological refractory period? Psychological
Science, 17, 228-235.
Lieberman, D.A., Chafee, S.H., & Robers, D.F. (1988).
Computers, mass media, and schooling: Functional
equivalence in uses of new media. Social Science
Computer Review, 6, 224-241.
91
Lightman, E. (2010). Practice effects, emotion, and
mechanisms of dual-task interference in driving and
cell phone research. Unpublished master’s thesis,
Georgia Institute of Technology, Atlanta, GA.
Machado, M.C., Fantini, E.P.C., & Chaimowicz, L. (2011).
Player modeling: Toward a common taxonomy.
Proceedings from 2011 16th International Conference on
Computer Games (CGAMES). Louisville, KY.
McCann, R.S., & Johnston, J.C. (1992). Locus of the single-
channel bottleneck in dual-task performance. Journal
of Experimental Psychology: Human Perception and
Performance, 18, 471-484.
Mcleaod, P. (1977). A dual-task response modality effect:
Support for multi-processor models of attention,
Quarterly Journal of Experimental Psychology, 29, 651-
667.
Meyer, D.E., & Kieras, D.E. (1997a). A computational theory
of executive cognitive processes and multiple-task
performance: Part 1. Basic mechanisms. Psychological
Review, 104, 3-65.
Meyer, D.E., & Kieras, D.E. (1997b). A computational theory
of executive processing and multiple-task performance:
Part 2. Accounts of psychological refractory period
phenomena. Psychological Review, 104, 749-791. 92
Meyer, D.E., Kieras, D.E., Lauber, E., Schumacher, E.H.,
Glass, J., Zurbriggen, E. Gmedindl, L., & Apfelblat,
D. (1995). Adaptive executive control: Flexible
multiple-task performance without pervasive immutable
response-selection bottlenecks. Acta Psychologica, 90,
163-190.
Meyer, D.E., & Kornblum, S. (Eds.). (1993). Attention and
performance XIV. Synergies in experimental psychology,
artificial intelligence, and cognitive neuroscience.
Cambridge, MA: M.I.T. Press.
Microsoft Game Studios (2004). Rise of Nations: Gold
Edition [Software].
Moray, N. (1967). Where is capacity limited? A survey of a
model. Acta Psychologica, 27, 84-92.
Navon, D., & Gopher, D. (1979). On the economy of the human
processing system. Psychological Review, 86(3), 214-
255.
93
Neiper, S., Morgan, O., & Culver, G. (2001). Affective
doctrinal education as cased study in Air Force core
competency education. Proceedings from The
Interservice/Industry Training, Simulation & Education
Conference (I/ITSEC) 2001 (Conference Theme:
Warfighter Readiness Through Innovative Training
Technology). Retrieved from
http://ntsa.metapress.com/app/home/contribution.asp?re
ferrer=parent&backto=issue,17,151;journal,11,22;linkin
gpublicationresults,1:113340,1
Pashler, H.E. (1984). Processing stages in overlapping
tasks: Evidence for a central bottleneck. Journal of
Experimental Psychology: Human Perception and
Performance, 10, 358-377.
Pashler, H.E. (1994). Dual-task interference in simple
tasks: Data and theory. Psychological Bulletin, 116,
220-244.
Pashler, H.E. (1998). The psychology of attention.
Cambridge, MA: MIT Press.
Pashler, H.E., & Johnston, J.C. (1998). Attentional
limitations in dual-task performance. In H.E. Pashler
(Ed.), Attention (pp. 155-189). Hove, UK: Psychology
Press. 94
Phillips. (September 17, 2011). Do you macro like a pro?
Analysis of macro across all tiers of the SC2 ladder.
Retrieved from:
http://www.teamliquid.net/forum/viewmessage.php?topic_
id=266019
Phillips & Flew. (October 29, 2011). SQ leaderboard by
region. Retrieved from:
http://www.teamliquid.net/forum/viewmessage.php?topic_
id=280421
Plott, S. (2011, August 17). Day[9] daily #340 - Newbie
Tuesday: Gold level analysis [Video file]. Retrieved
from http://day9.tv/d/Day9/day9-daily-340-newbie-
tuesday-gold-level-analysis/
Portney, L.G., & Watkins, M.P. (2000). Foundations of
Clinical research: Applications to practice. (2nd Ed.)
Upper Saddle River, NJ: Prentice Hall Health.
Posner, M.I. (Ed.). (1989). Foundations of cognitive
science (Chapter 16, pp.631-682). Cambridge, MA:
M.I.T. Press.
Proctor, R.W., Wang, D.Y., & Pick, D.F. (1998) An empirical
evaluation of the SYNWORK1 multiple-task work
environment. Behavior Research Methods, Instruments, &
Computers, 30(2), 287-305. 95
Rabinowitz, Y.G., Breitbach, J.E., & Warner, C.H. (2009).
Managing aviator fatigue and neurocognitive
functioning. Military Medicine, 174(4), 358-362.
Reis, A., & Janssen, D. (n.d.). The effects of feedback on
student performance while performing multiple tasks
simultaneously. Effects of Feedback, 57-64.
Roberts, D.F., Foehr, U.G., Rideout, V.J., & Brodie, M.
(1999). Kids & media @ the new millennium. Menlo Park,
CA: Kaiser Family Foundation.
Ruthruff, E., Hazeltine, E., & Remington, R. W. (2006).
What causes residual dual-task cost after practice?
Psychological Research, 70, 494-503.
Ruthruff, E., Johnston, J.C., & Van Selst, M. (2001). Why
practice reduces dual-task interference. Journal of
Experimental Psychology: Human Perception and
Performance, 27, 3-21.
Ruthruff, E., Johnston, J.C., Van Selst, M., Whitsell, S.,
& Remington, R. (2003). Vanishing dual-task
interference after practice: Has the bottleneck been
eliminated or is it merely latent? Journal of
Experimental Psychology: Human Perception and
Performance, 29, 280-289.
96
Salvucci, D.D. (2002). Modeling driver distraction from
cognitive tasks. In Proceeding of the 24th Annual
Conference of the Cognitive Science Society (pp. 792-
797). Mahwah, NJ: Lawrence Erlbaum Associates.
Salvucci, D.D., & Anderson, J.R. (2001). Integrating
analogical mapping and general problem solving: The
path-mapping theory. Cognitive Science, 25, 67-110.
Sanders, A.F. (1980). Stage analysis of reaction processes.
In G.E. Stelmach & J. Requin (Eds.), Tutorials in
motor behavior (pp. 331-354). Amsterdam: North-
Holland.
Schumacher, E.H., Seymour, T.L., Glass, J.M., Fencsik,
D.E., Lauber, E.J., Kieras, D.E., & Meyer, D.E.
(2001). Virtually perfect sharing in dual-task
performance: Uncorking the central cognitive
bottleneck. Psychological Science, 12(2), 101-108.
Shantia, A., Begue, E., & Wiering, M. (2011). Connectionist
reinforcement learning for intelligent unit mirco
management in Star Craft. Proceedings from The 2011
International Joint Conference on Neural Networks
(IJCNN). San Jose, CA.
Sternberg, S. (1969). Discovery of processing stages –
Extension of Donders’ method. Acta Psychologica, 30,
276-315. 97 stet_tcl. (2010). Multitasking Trainer. (Version 0.941)
[Software]. Available from:
http://www.sc2mapster.com/maps/multitasking-
trainer/files/9-multitasking-trainer-v0-941/
Strayer, D.L., & Drews, F.A. (2004). Profiles in driving
distraction: Effects of cell phone conversations on
younger and older drivers. Human Factors: The Journal
of the Human Factors and Ergonomics Society, 46(4),
640-649.
Synnaeve, G., & Bessiere, P. (2011). A Bayesian model for
Plan recognition in RTs games applied to Star Craft.
Proceedings of the Seventh AAAI Conference on
Artificial Intelligence and Interactive Digital
Entertainment, 79-84.
Tombu, M., & Jolicoeur, P. (2004). Virtually no evidence
for virtually perfect time-sharing. Journal of
Experimental Psychology: Human Perception and
Performance, 30, 795-810.
Valve Corporation. (2003). Counter Strike. [computer
software]. Kirkland, WA.
Van Schie, E.G.M., & Wiegman, O. (1997). Children and
videogames: Leisure activities, aggression, social
integration, and school performance. Journal of
Applied Social Psychology, 27, 1175-1194. 98
Van Selst, M., Ruthruff, E., & Johnston, J.C. (1999). Can
practice eliminate the psychological refractory period
effect? Journal of Experimental Psychology: Human
Perception and Performance, 25, 1268-1283.
Waller, M.J. (1997). Keeping the pins in the air: How work
groups juggle multiple tasks. In M.M. Beyerlein & D.A.
Johnson (Eds.), Advances in interdisciplinary studies
of work teams (Vol. 4, pp. 217-247). Stanford, CT:
JAI Press.
Walsh, D. (2000). Testimony submitted to the United States
Senate Committee on Commerce, Science, and
Transportation. Hearing on the impact of interactive
violence on children. Retrieved from:
http://www.senate.gov/~commerce/hearings/0321wall.pdf.
Weber, B.G., Mateas, M., & Jhala, A. (2010). Applying goal-
driven autonomy to Star Craft. In Proceedings
from The Sixth AAAI Conference on Artificial
Intelligence and Interactive Digital Entertainment,
101-106. Santa Cruz, CA. Retrieved from
http://users.soe.ucsc.edu/~%20bweber/pubs/gda_aiide201
0.pdf
Welford, A.T. (1952). The “psychological refractory period”
and the timing of high speed performance – A review
and a theory. British Journal of Psychology, 42, 2-19. 99
Welford, A.T. (1959). Evidence of single-channel decision
mechanism limiting performance in a serial reaction
task. Quarterly Journal of Experimental Psychology, 2,
193-210.
Welford, A.T. (1967). Single channel operation in the
brain, Acta Psychologica, 27, 5-21.
Welford, A.T. (1980). The singe channel hypothesis. In A.T.
Welford (Ed.), Reaction time (pp. 215-252). San Diego,
CA: Academic Press.
Whidby, M. (2011). SacOvie-Zerg scouting for Star Craft II.
(Unpublished Graduate Paper, Maryland University,
Baltimore, MD). Retrieved from:
http://www.umiacs.umd.edu/~mawhidby/docs/ sacovie.pdf
Wickens, C.D. (2002). Multiple resource and performance
prediction. Theoretical Issues in Ergonomics Science,
3(2), 159-177.
Wickens, C.D. (1991). Processing resources and attention,
In D. Damos (Ed.), Multiple-task performance (pp. 3-
34). London: Taylor & Francis.
Wilelm, A. (2012, July 18). MLG’s esports events now
attract more young male viewers than the Rose Bowl.
Retrieved from: http://thenextweb.com/media/2012/07/18/
mlgs-esports-events-now-attract-more-young-male-
viewers-than-the-rose-bowl/ 100
Yi, S. (2011). Adaptive strategy decision mechanism for
Star Craft AI. Springer Proceeddings in Physics, 138,
47-57.
Yun, H. (2011). Using logistic regression to analyze the
balance of the case of Star Craft II. Technical
Report. Retrieved from
http://arxiv.org/pdf/1105.0755v1.pdf
101
APPENDIX A
DESCRIPTIVE ANALYSIS
This appendix is intended to address the primary analyses (hypotheses one and two) and other secondary analyses that stretched the viable use of statistical analysis. Both the first and second hypotheses suffered from low participation in the experimental and gamer- control groups. While the analyses for the first two hypotheses are presented, they should not be viewed as statistically sound, and should be held to a descriptive context. The contents of this appendix will be presented in the following sections: analyses, conclusions, discussion.
Analyses
The first hypothesis stated that participants in the experimental group will have significantly higher mean score as compared to those participants in the control group across the four post-test SynWin trials. Both a 2x2
Repeated Measures ANOVA and Friedman test were utilized to examine the differences in means. The omnibus F-test indicated the presence of a significant difference between 102 pre-test (M = 722.75)/post-test (M = 944.13) SynWin mean
2 scores [F (1, 6) = 25.48, Δ = 221.34, p = 0.002, ηp =
0.809]. A comparison of scores based on group
(experimental/gamer-control) did not yield a significant
2 difference [F (1, 6) = 0.00, p = 0.992, ηp = 0.00]. Results from the Friedman test replicated those of the Repeated
Measures ANOVA, indicating the presence of a significant difference in SynWin values across the eight trials [X²(7)
= 38.750, p = 0.000]. SynWin scores by group and time are presented in Table 8.
Table 8. SynWin Scores by Time and Group.
Group Pre-Test Post-Test Mean Difference
Experimental 714.75 951.00 36.25
Gamer-Control 730.75 937.25 6.50
Mean Differences 16.00 14.25 -
*= p < 0.05
The second hypothesis stated that participants in the experimental group will have significantly higher mean APM values as compared to those participants in the control group across the three post-test one-versus-one game replays. Both a 2x2 Repeated Measures ANOVA and a Friedman test were utilized to complete this analysis. The omnibus 103
F-test revealed that the interaction between time and group
APM values did not yield a significant difference [F (1, 5)
2 = 0.241, p = 0.644, ηp = 0.046]. Both time [F (1, 5) =
2 0.19, p = 0.896, ηp = 0.004] and group [F (1, 5) = 0.56, p
2 = 0.487, ηp = 0.101] comparisons followed the same trend and yielded non-significant mean differences. Results from the Friedman test replicated those of the Repeated Measures
ANOVA, indicating a significant difference in APM values did not exist across the six sets of game replays [X²(5) =
2.336, p = 0.801] Action Per-Minute values by time and group are presented in Table 9.
Table 9. APM Values by Time and Group
Group Pre-Test Post-Test Mean Difference
Experimental 111.33 110.44 36.25
Gamer-Control 125.75 127.33 6.50
Mean Difference 14.42 17.89 -
*= p< 0.05
While not included in the second hypothesis, an examination of pre/post-test EAPM values was warranted due to the significant relationship between EAPM and SynWin with the pre-test group. A One-Way Repeated Measures ANOVA was conducted to analyze the difference between pre-test 104
(m= 83.71 EAPM) and post-test (M = 79.86 EAPM). Results of the analysis indicated the mean EAPM scores did not significantly differ from pre to post-test [F (1, 6) =
2 1.15, p = 0.324, ηp = 0.161].
The third hypothesis stated that there would be a positive statistically significant relationship between participants’ SynWyn scores and EAPM values at pre and post-test. Analysis of the relationship between SynWin scores and EAPM values was limited to pre-test data in the examination of the third null hypothesis; here the analysis expands to the post-test data. A Spearman rank correlation coefficient was performed to determine the relationship between post-test SynWin mean scores and pre-test EAPM mean values. Results from the Spearman correlation yielded a non-significant result (rs = 0.036, p = 0.94).
Conclusions
The first hypothesis stated that participants in the experimental group will have significantly higher mean scores as compared to those participants in the control group across the four post-test SynWin trials. Results from the Repeated Measure ANOVA [F (1, 6) = 25.48, p =
0.002, ηp2 = 0.809] and the Friedman test [X²(7) = 38.750, p = 0.000] indicated the that only mean differences in
SynWin scores based on time (pre-test/post-test) yielded 105 significance. While the interaction between time and group was not found to be significant, the first null hypothesis does not appear to be supported by the data.
The second hypothesis stated that participants in the experimental group will have significantly higher mean APM values as compared to those participants in the control group across the three post-test one-versus-one game replays. Analysis from the Repeated Measure ANOVA [F (1,
5) = 0.241, p = 0.644, ηp2 = 0.046] and the Friedman test
[X²(5) = 2.336, p = 0.801] indicated the lack of significant mean differences in APM values; thus supporting the second hypothesis.
The third hypothesis stated that there would be a positive statistically significant relationship between participants’ SynWyn scores and EAPM. Analysis of post- test mean SynWin scores and mean EAPM values revealed that both variables were not significantly related (rs = 0.036, p
= 0.94). As reported in the analysis of the second null hypothesis, mean EAMP values did not vary between the pre and post-test games; however, mean SynWin scores were found to be on average significantly higher in the post-test. The significant difference in SynWin scores appears to be related to a practice effect, possibly altering the 106 relationship; therefore the third null hypothesis is still rejected
Discussion
The first and second null hypotheses both suffered from a lack of participants, as the study did not yield the
46 participants called for in the design (issues and implications on future study design are detailed in Chapter
5). In the context of description rather than statistical analysis, the collected data on mean pre/post-test SynWin scores depicts a lack of “significant” difference in multitasking scores between the experimental group and the gamer-control. The only “significant” difference is the mean difference in SynWin scores over time. From a descriptive perspective, the data point to a lack of training effect on general multitasking ability for Star
Craft II multitasking trainer. With regards to Star Craft
II specific multitasking, the data appear to indicate the
Star Craft II multitasking trainer did not significantly impact in-game multitasking as defined by EAPM values.
The relationship between SynWin scores and EAPM values was found to be significant in the pre-test data set; indicating a moderate relationship between general and Star
Craft II based multitasking. This relationship did not extend to the post-test analysis. The average time between 107 pre-test and post-test appointments was 17.13 days (one participant was 29 days). A review of relevant literature did not provide any other studies which had as wide of a time difference between pre and post-test measurement with the SynWin. The data appear to raise questions regarding the reliability of the use of the SynWin in pre/post-test designs when the study affords the participant an extended treatment period.
108
APPENDIX B
FIGURES
109
Figure 1. Comparison of SynWin and Star Craft II User
Interfaces.
110
APPENDIX C
CONSENT FORMS 111 112
113
114
115
116
117
APPENDIX D
DEMOGRAPHIC QUESTIONNAIRE
118
119
APPENEDIX E
RECRUITMENT POSTER
120