Do we know what really works? A systematic review about using video games for cognitive training Brito F1,2, Virgolino A1,3, Fialho M1, Miranda AC1,3, Peixoto J3, Neves I1,3, Camolas J1,4, Reis MF1,3, Carriço L5, Santos O1,3

1 Instituto de Saúde Ambiental, Faculdade de Medicina, Universidade de Lisboa 2 Fundação para a Ciência e a Tecnologia fellowship 3 Instituto de Medicina Preventiva e Saúde Pública, Faculdade de Medicina, Universidade de Lisboa 4 Serviço de Endocrinologia, Hospital de Santa Maria, CHLN 5 Faculdade de Ciências, Universidade de Lisboa th International Conference for 13 Cognitive Neuroscience

INTRODUCTION METHODS

Applied games (i.e., serious games with gamified environments) aim to promote the learning of A systematic review was conducted following PRISMA guidelines4. Consulted electronic databases were PubMed, SciELO and PsycARTICLES. The search in SciELO and PubMed was performed on January 24th, 2017 and new contents and skills, as well as to facilitate behaviour change1. Use of games is more and was restricted to studies published between 2007-2017 and within last 10 years, respectively. PsycARTICLES search was conducted on January 26th-29th, 2017, targeting articles published from January 1st, 2006 to more supported by health professionals, game-designers and patients, who tend to consider December 31th, 2016. Additional inclusion criteria (common to all databases) were: studies with humans and English-language written papers. 2 conventional cognitive training strategies as rather monotonous . The autonomous motivation Search keywords (both free-text and MeSH terms) formed two blocks and were combined pairwise. Block 1: cognitive outcomes (cognitive function, cognitive performance, processing speed, attention, orientation, to learn and repeat non-habitual actions is maintained when positive emotions (e.g., having executive function, executive system, working memory, inhibitory capacity, cognitive flexibility, planning, decision making, visual memory, verbal memory); Block 2: /game-elements (game, , fun, feeling curious) are associated with learning and repetitive tasks. This makes video games a videogame, online videogame, , applied game, , edutainment, rehabitainment, exergames, gamification, virtual reality exposure therapy, virtual reality, virtual environment, 3D-environments, 3 powerful resource for cognitive self-control training . However, the evidence about what makes virtual gardening, virtual kitchen, virtual supermarket, real-life simulation game, sports game, role-play game, puzzle game, dance-, , endless running games, strategy games, game video games effective in promoting general or specific cognitive functions is still scarce. elements, videogame level, game level, level of play, game controller, controller, multitask, interaction mode, multiplayer, leaderboard, scoreboard, first-person shooter, self-representation, narration, narrative context, This project aimed to summarize and integrate existing knowledge about which video games’ teams, time pressure, parallel communication systems). Additional articles were manually recruited through an exhaustive search in relevant-field journals. elements are being used and their effectiveness in promoting attention and working memory. An initial screening for eligibility, based on title and abstract analysis, was done independently by two or, in case of disagreement, three team members. Considering the high number of identified/eligible studies, the analysis for this poster was limited to those having attention and working memory as cognitive outcomes and having detailed information about game elements. RESULTS

From the 2777 initially identified papers, 155 were considered eligible for narrative synthesis Figure 2 summarizes some data regarding games purposes and elements and regarding study design. As can be seen, cognitive training is the most prevalent purpose of the selected video games. Most part of papers do (methodological appreciation and evidence summary) (Figure 1). not mention if the video game includes a narrative nor if it includes an Avatar or a Character (for enhancing players’ identification/immersion with the game). Randomized controlled trial is the most common design for studying video game effects (in terms of its health promotion effect or in terms of cognitive assessment or training effect).

Table 1 presents the narrative synthesis of 10 selected papers concerning the association between video games and cognitive outcomes (attention and working memory). For this table, only papers with a detailed description of game elements were included (n=10). In these papers, video games were studied either in terms of their effectiveness for cognitive training or in terms of their adequacy for cognitive assessment/evaluation.

Table 1. Narrative synthesis of the eligible papers

Game Elements Cognitive Outcomes Population & Sample Group Exposure to the Game Transfer User Experience Title Game Title Game Type Game purpose Score system Reward system Study design (Sessions, weeks, Follow-up Narrative Avatar/ Tutorials/ Win / Lose (Baseline [B], End of intervention [EI], (points, level, life (Badges, digital Time pressure Competition Multiplayer Control duration/session) Effect Assessment Attention Working Memory Other (storyline) Character scaffolding conditions dropout% [D]) Figure 1. PRISMA fluxogram bar, ...) award,…) [5] Maclin et al., 2011 Space Fortress Computer game Cognitive training (a) Yes Yes Yes (a) (a) (a) (a) (a) (a) Healthy subjects (B:39|EI:37| D:5,13%) Yes 20h Yes No Yes ↔ (a) (a) [6] Janssen et al., 2015 Space Fortress Computer game Cognitive training (a) Yes Yes Yes (a) (a) (a) (a) (a) RCT Multiple slerois (B:34|EI:28| D:17,7%) Yes 20 sessions | 8weeks | 1h Yes No Yes ↔ ↔ (a) [7] Anguera et al., 2013 NeuroRacer Computer game Cognitive training (a) No Yes Yes (a) (a) Yes (a) No RCT Older adults (B:60|EI:46|D:23,3%) Yes 12 sessions | 4weeks | 1h Yes Yes No ↑ ↑ (a) [8] Dovis et al., 2015 Braingame Brian Computer game Cognitive training Yes Yes Yes Yes Yes (a) Yes No No RCT ADHD children (B:89| EI:78| D: 12,4%) Yes 25 sessions | 5weeks |35-50 min Yes Yes No (a) ↑ ↑ (visual STM) ↑ (task switching) [9] Basak et al., 2008 Rise of Nations Computer game Cognitive training (a) (a) (a) (a) Yes Yes (a) Yes (a) RCT Healthy subjects (B:40|EI: 39|D:2,5%) Yes 15 sessions | 4-5 weeks | 1.5h Yes (a) (a) (a) ↑ ↑ (visual STM) ↑ (reasoning) N=155 N=155 ↑ Divided attention [10] Martel et al., 2016 Motion Rehab Exergame Cognitive training Yes Yes (a) Yes (a) (a) Yes (a) No Non-RCT Healthy elderly (B:10| EI: -| D: -%) No 15 sessions | 15 weeks | 15 min Yes No No ↔ Sustained (a) (a) attention [11] Van de Weijer-Bergsma Lion Game Computer game Cognitive assessment (a) (a) Yes Yes (a) (a) (a) (a) No Validation study Children (B:442|EI: 374|D: 15,4%) No (a) Yes No (a) (a) ↔ (a) et al., 2015 [12] Parsons et al., 2007 Virtual Classroom Virtual reality Cognitive assessment Yes (a) Yes (a) (a) (a) (a) (a) (a) Validation study ADHD children (B:20|EI:n/a|D:n/a) Yes (a) No No Yes ↔ (a) (a) [13] Renison et al., 2012 Virtual Library Task Virtual reality Cognitive assessment (a) (a) (a) No (a) (a) Yes (a) (a) Validation study TBI (B:60|EI:n/a|D:n/a) Yes 1 session | 9-20 min No No Yes (a) ↔ ↔ (interference) Virtual Reality Stroop [14] Armstrong et al., 2013 Virtual reality Cognitive assessment Yes (a) (a) (a) (a) (a) (a) (a) (a) Validation study Soldiers (B:56 | EI: 49|D:12,5%) No 1 session No No (a) ↔ (a) (a) Task (a) Information not available ↑ Positive effect ↔ No effect

N=155 N=155 Collected data suggest poorly-detailed description of video games and lack of consensual definitions of game-elements (e.g., avatar vs. character) or technological borders (virtual environments vs. reality). Key-aspects Figure 2. Game purpose, game of user-experience such as fun, enjoyment and usability only rarely are evaluated. Cognitive endpoints are usually considered as dependent variables of the whole gaming experience, but without adequate elements and study design understanding of which specific game-elements have effect. Far transference of cognitive competences is seldom assessed and effectiveness evaluation tends to be done for short-term follow-up periods.

DISCUSSION REFERENCES

1. Fleming T, et al. Serious Games and Gamification for Mental Health: Current Status and Promising Directions. Front Psychiatry. 2017;7(January). Both methodological limitations and lack of information about video games’ 2. Collerton J, et al. A comparison of computerized and pencil-and-paper tasks in assessing cognitive function in community-dwelling older people in the Newcastle 85+ pilot study. J Am Geriatr Soc. 2007;55(10):1630–5. 3. Anguera JA, et al. Video game training enhances cognitive control in older adults. Nature. 2013;501(7465):97–101. properties/characteristics hinder a proper assessment of the effectiveness of video games in 4. Moher D, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009;6:e1000097. 5. Maclin E, et al. Learning to multitask: Effects of video game practice on electrophysiological indices of attention and resource allocation. Psychophysiology. 2011;48:1173–1183. promoting cognitive functions. The trend on using videogames to promote cognitive function 6. Janssen A, et al. The effects of video-game training on broad cognitive transfer in multiple sclerosis: A pilot randomized controlled trial. J. Clin. Exp. Neuropsychol. 2015;3395:1–18 7. Anguera J, et al. Video game training enhances cognitive control in older adults. Nature. 2013;501:97–101. justifies and urges for a thorough evaluation of the impact of different game-elements, enabling 8. Dovis S, et al. Improving executive functioning in children with ADHD: training multiple executive functions within the context of a computer game. a randomized double-blind placebo controlled trial. PLoS One. 2015 Apr 6;10(4):e0121651 game’s personalization to individuals’ clinical condition. 9. Basak C, et al. Can training in a real-time strategy videogame attenuate cognitive decline in older adults? Psychol. Aging. 2008;23:765–777. 10. Martel M, et al. Efeitos da intervenção com game na atenção e na independência funcional em idosos após acidente vascular encefálico. Fisioter. e Pesqui. 2016;23:52–58 11. Van de Weijer-Bergsma E, et al. Validity and reliability of an online visual–spatial working memory task for self-reliant administration in school-aged children. Behav. Res. Methods. 2015;47:708–719 12. Parsons T, et al. A controlled clinical comparison of attention performance in children with ADHD in a virtual reality classroom compared to standard neuropsychological methods. Child Neuropsychol. 2007;13:363–381 13. Renison B, et al. The Ecological and Construct Validity of a Newly Developed Measure of Executive Function: The Virtual Library Task. J. Int. Neuropsychol. Soc. 2012;18:440–450. 14. Armstrong C, Reger G, Edwards J, Rizzo A, Courtney C, Parsons T. Validity of the Virtual Reality Stroop Task (VRST) in active duty military. J. Clin. Exp. Neuropsychol. 2013;35:113–23 View publication stats