Challenges in Evaluating Exergames with Older Adults

Did They Really Like the Game? -- Challenges in Evaluating Exergames with Older Adults

Jan Smeddinck Kathrin M. Gerling Abstract University of Bremen University of Saskatchewan Exergames offer exciting perspectives not only for rec- Bibliothekstraße 1 110 Science Place reational, but also for therapeutic use. Increasing num- Bremen 28359, Germany Saskatoon S7N 5C9, Canada bers of older adults in many industrialized countries [email protected] [email protected] raise the need for affordable and reliable solutions to help people to stay healthy and fit at advanced ages. Marc Herrlich Rainer Malaka While games user research is vital to increasing the University of Bremen University of Bremen quality of game designs and improving game design Bibliothekstraße 1 Bibliothekstraße 1 processes, many research tools are difficult to use with Bremen 28359, Germany Bremen 28359, Germany senior gamers, and the target group is especially vul- [email protected] [email protected] nerable to mistakes both in game design and games user research. Markus Krause We report and classify some of these challenges along University of Bremen with possible approaches, aiming to fuel intensified ex- Bibliothekstraße 1 change about methodological experiences among re- Bremen 28359, Germany searchers in the field. [email protected] Author Keywords Evaluation, methodology, games, entertainment, game design, older adults, physical therapy.

ACM Classification Keywords K.4.2 [Computers and Society]: Social Issues œ Assis- Copyright is held by the author/owner(s). tive technologies for people with disabilities, Handi- CHI‘12, May 5œ10, 2012, Austin, Texas, USA. capped persons/special needs; K.8.0 [Personal Compu- ACM 978-1-4503-1016-1/12/05. ting]: General - Games.

Introduction Challenges: Evaluating Games with Elderly Industrialized societies are faced with a growing num- The challenges that were noted in a range of studies ber of older adults, and many industries have started to involving exergames developed at the University of embrace senior citizens as a new target audience. Even Bremen, Germany (Figure 1) and the University of Sas- in video game development, which has traditionally katchewan, Canada (Figure 2) can be categorized focused on a younger audience, older adults are in- roughly as general challenges related to the evaluation creasingly considered a valuable user group. Yet, little of games with older adults, as well as challenges specif- is known about their needs regarding interface and ic to the evaluation of exergames for the elderly. Figure 1. Cinderella is a multi- game design. This is particularly difficult when design- General Challenges player game for up to three play- ing exergames, which aim to encourage players to re- ers. The game encourages move- Four general game-related challenges were observed ments according to music rhythm. main physically active [8]. Since motion detection has during our studies, ranging from issues caused by the Movements are based on kinesiatric become part of mainstream gaming hardware (Ninten- impact of age-related changes to a lack of gaming ex- exercises. It is part of the Wuppdi! do Wii controllers, Sony Playstation Move, Microsoft suite that contains five small games perience and a low level of computer literacy, which for the support of motion-based Kinect, etc.), games that motivate physical activities influences the perception of games among older adults. therapy for Parkinson‘s disease are available to a wide range of users. Although re- patients. search suggests that such games are beneficial for old- Anxiety related to technology necessary for an evalua- er adults [7], age-related changes and impairments tion: Pretests of the Wuppdi! prototype [2] showed that influence the interaction process [3], leading to a varie- an experimentation room that displays too many cables ty of problems related to the usability and accessibility and computers as parts of non-invasive research tools of such games. In this context, an important step to- (cameras, microphones, etc.) can scare and confuse wards the development of senior-friendly games is the elderly participants to the point that they withdraw establishment of a user-centered design process, which from an experiment. Negative feelings related to per- introduces evaluations at an early point of the devel- sonal performance: Failure in games is often attributed opment process. However, applying established meth- to personal failure rather than poor game design [4]. ods in games user research [6] can be difficult due to Many participants are concerned that their performance is tested instead of the quality of the game. Social fac- Figure 2. A virtual garden specifi- the impact of age-related changes and special needs of cally designed for institutionalized the target audience. tors severely influence the research of exergames for older adults using Microsoft Kinect older adults as many elderly try to be polite when ex- as input device. Gestures were In this paper, we summarize results of our own case perimenters are present. To this end, experimenter designed in collaboration with a studies to identify core challenges of evaluating exer- physical therapist. Additionally, the effects should be considered carefully as a potential game features a calibration routine games with older adults, and we address the develop- bias, especially when conducting interviews or focus which adapts interaction paradigms ment of evaluation methods that allow researchers and groups that include direct questioning of participants. to the range of motion of individual practitioners to meet the needs of older adults. There- players. Overstraining participants: Player performance and by, we hope to foster the creation of accessible exer- their ability of answering extensive questionnaires vary games and games for physical therapy for senior citi- on a daily basis [3]. During the evaluation of Wuppdi! zens. [2], therapists and caregivers pointed out that individu-

al physical fitness and cognitive awareness among older eo-stream recordings, but also in the resulting approx- * Questionnaires are crucial tools for the evaluation of games. When adults may vary strongly from one day to the next and imated skeletal postures and movements. As a result, working with older adults, standard also over the course of a day. The duration of play ses- game prototypes we used had to be adjusted to be questionnaires from games user sions and the length of asynchronous qualitative in- much more tolerant towards imprecise input. This can research are often difficult to ap- struments have to be adjusted accordingly. be challenging in combination with the differences in ply, because they do not fully match the application context and Challenges Specific to Exergames range of motion among elderly and tracking difficulties may overstrain participants. At- resulting from additional special needs, for example tempts at simplifying the answer- Evaluating exergames with older adults is particularly when participants are in wheelchairs [3]. ing process lead to contradictory challenging. In this section, we list different problems results: that we observed during our evaluation sessions. Another important aspect of physical therapy game During the evaluation of Silver- First of all, encouraging physical activity among elderly design is the involvement of therapists, family mem- Promenade, Gerling et al. [4] in- bers and nursing staff, who have additional interests, troduced smiley scales to facilitate users may increase certain health risks for frail partici- the answering process. pants. Cognitive load is increased when participants such as being able to support a player with playing the However, attempts in the Wuppdi! have to coordinate movements, interact with games game and tracking their performance (cf. [1]). While project to represent Likert scales and try to reflect on their experience. Such multi tasks most therapists were interested in our research proto- through smiley scales did not im- types and the approach of games for physical therapy, prove the understanding of the are particularly challenging for older adults but also many were cautious to point out that they would not questions. Instead, reading ques- beneficial in physical therapy [5]. Additionally, partici- tions as a structured interview pants were extremely focused on in-game events while want the games to get in the way of their work, or to improved participants‘ abilities of ”replace‘ therapeutic intervention time. Still, these third responding adequately. playing the Wuppdi! prototypes, and sometimes came close to stumbling in the real world. For older adults, party groups can be valuable to the game design pro- this may be a serious health hazard as falls can easily cess, assume roles in the game mechanics (cf. [7]) and lead to bone fractures. Furthermore, we observed in data analysis, since they are practically involved with tendencies in older adults to overexert while engaging the patients and often provide important insights into with exergames to complete all tasks within an evalua- individual participants‘ capabilities and needs. The in- tion [4]. Since they only informed the experimenter clusion of methods allowing researchers to harness the about their exhaustion after the gaming session, it was potential benefits brought in by third parties could not possible to abort it. The application of multiple tests prove highly valuable. to measure physical stress and exertion requires ex- Discussion tended ethics approval and the involvement of medical staff. In research about games for therapy, it is im- Evaluating exergames with older adults is associated Figure 3. The Frog Prince (part of portant to evaluate therapeutic effects œ long-term with unique challenges that are related to the impact of the Wuppdi! suite) is a single player studies and the involvement of therapists and physi- age-related changes and a lack of gaming experience game which requires wide circular among today‘s elderly. To obtain reliable results, the arm movements, similar to breast- cians is necessary. stroke swimming. Synchronous games user research community needs to provide On the technical side, natural user interfaces, such as movement of both arms controls methods which reduce the cognitive load of participants forward motion, movement of a the Kinect, capture continuous and noisy image data instead of overwhelming them by presenting too many single arm controls turns. and thus have problems with jitter not only in the vid- new challenges at the same time (e.g. new gaming

technology, a variety of research methods and unfamil- References J AN SMEDDINCK is a doctoral iar persons involved with the evaluation). [1] Annema, J. H., Verstraete, M., Vanden Abeele, V., candidate at the University of Bre- Desmet, S., & Geerts, D. Videogames in therapy: a men, Germany and holds a fellow- In our studies, we found that small adjustments to the therapist‘s perspective. In Proc. of Fun and Games ship of the Klaus Tschira Foundation evaluation environment (e.g. hiding technical compo- 2010, Leuven, Belgium, (2010). (KTS). Working in the fields of inter- nents) helped participants adjust to the situation. Fur- action design and serious games, he [2] Assad, O., Hermann, R., Lilla, D., Mellies, B., Mey- focuses on automated learning from thermore, providing simple, specifically designed ques- er, R., Shevach, L., Siegel, S., Springer, M., Tiemkeo, interactions in adaptive systems. tionnaires*, reading them out as structured interviews S., Voges, J., Wieferich, J., Herrlich, M., Krause, M., and keeping the session duration as short as possible and Malaka, R. 2011. Motion-Based Games for Parkin- M ARC HERRLICH is a research sonތs Disease Patients. In Proc. of ICECӂ11, Vancouver, assistant and doctoral candidate at helped reduce the cognitive load of participants experi- Canada, (2011). the University of Bremen, Germany. encing age-related changes. Additionally, allowing His research interests include inter- enough time, buffer and explanations during the proce- [3] Gerling, K., Livingston, I., Nacke, L., & Mandryk, R. active systems, new user interfaces, dures is crucial when working with special populations Full-Body Motion-Based Game Interaction for Older computer graphics and computer Adults. In Proc. of CHI 2012, Austin, TX, USA, (2012). games. such as institutionalized older adults. [4] Gerling, K., Schulte, F., & Masuch, M. Designing M ARKUS KRAUSE is a doctoral Finally, an important aspect that has to be addressed and Evaluating Digital Games for Frail Elderly Persons. candidate at the University of Bre- by future work is achieving clinically valid measures of In Proc. of ACE 2011, Lisbon, Portugal, (2011). men, Germany. His research ex- the effectiveness of games played by older adults: If plores interaction design, human [5] Hüger, D., Zieschang, T., Schwenk, M., Oster, P., computer interaction and digital exergames are to be applied in physical therapy, it is Becker, C., & Hauer, K.. Designing studies on the effec- serious games. crucial to gain insights into long-term effects and de- tiveness of physical training in patients with cognitive tailed physical effects, which requires research methods impairment. Zeitschrift für Gerontologie und Geriatrie, K ATHRIN GERLING is a PhD stu- 42(1), (2009), 11-19. dent at the University of Saskatche- that go far beyond those designed to examine game wan, Canada under the supervision usability and accessibility. [6] Isbister, K. & Schaffer, N. Matrix of Issues and of Dr. Regan Mandryk. Her research Tools. In: Isbister, K. & Schaffer, N. (Eds.): Game Usa- evolves around digital games for Acknowledgements bility. Advancing the Player Experience. Burlington, MA, older adults, as well as game usabil- USA: Morgan Kaufmann Publishers, (2008). ity and user experience. We would like to thank all student members of the Wuppdi! project, the Bremen regional group of the [7] Jung, Y., Li, K. J., Janissa, N. S., Gladys, W. L. C., R AINER MALAKA is professor for & Lee, K. M. Games for a better life: effects of playing —Deutsche Parkinson Vereinigung e.V.“ and the asso- digital media at the University of Wii games on the well-being of seniors in a long-term Bremen, Germany. He is the confer- ciated physiotherapists. Additionally, we thank Saska- care facility. In Proc. of the Sixth Australasian Confer- ence chair of the ICEC 2012 and the toon Health Region and Sherbrooke Community Centre ence on Interactive Entertainment, (2009). German representative in the IFIP for supporting our research, and the participants for TC 14. [8] Mueller, F., Edge, D., Vetere, F., Gibbs, M.R., Aga- their contribution during the evaluations. This work was manolis, S., Bongers, B., and Sheridan, J.G. Designing http://dm.tzi.de/en/people/staff/ partially funded by the Klaus Tschira Foundation Sports: A Framework for Exertion Games. In Proc. of http://hci.usask.ca/people/ through the graduate school —Advances in Digital Me- CHI 2011, Vancouver, Canada, (2011). dia“, as well as NSERC and the GRAND NCE.