Understanding Visual Interfaces for the Next Generation of Dance-Based Rhythm Video Games Emiko Charbonneau∗ Andrew Miller† Chadwick Wingrave‡ Joseph J. LaViola Jr.§ University of Central Florida Abstract We present an experimental study exploring how to best guide users when playing RealDance, a next generation dancing game proto- type. It uses four Nintendo Wii remotes, attached to the wrists and ankles, to create a 3D spatial interface utilizing the entire body to more closely mimic real dancing. Since RealDance requires a player to use both arms and legs, the player needs to know which of their four limbs to use, where they are expected to move, and when they are expected to move in the dance sequence. To understand the best way to present this information, we implemented three visual interface methods: Timeline, Motion Lines, and Beat Circles, that are based on existing rhythm video games but extended to support RealDance’s 3D interaction requirements. Our study explores each visual interface’s effectiveness in convey- ing dance sequence information and assisting the player in pro- viding a rewarding experience. Our evaluation is based on points scored in the game, and post-questionnaires used to solicit reac- tions about each visual interface including which was preferred and why. The results of the study show that players had significantly Figure 1: The RealDance system in action. The Motion Lines in- higher scores when using Motion Lines and Beat Circles than with terface is pictured. the Timeline. The results also indicate that players found Motion Lines and Beat Circles significantly easier to follow than Timeline and icon position significantly less confusing than the Timeline in- to the novelty of increased physical exertion in the case of Dance terface. From these results, we believe that Motion Lines and Beat Dance Revolution (DDR) and a greater sense of making music in Circles are more appropriate visual interfaces than the traditional the case of band games such as Guitar Hero and Rock Band. Timeline interface for full body, rhythm dance games. The advent of the Nintendo Wii remote (Wiimote) has brought 3D CR Categories: K.8.0 [Personal Computing]: General—Games; spatial interfaces [Bowman et al. 2004] to the consumer market H.5.2 [Information Interfaces and Presentation]: User Interfaces— and the forefront of gaming [LaViola 2008]. However, only a few Evaluation/methodology rhythm music games have made effective use of 3D spatial inter- action. For example, DDR uses the Wiimote and Nunchuk to add Keywords: dance-based rhythm games, dance gaming, 3D spatial other actions to its four arrow patterns, but their only function is a interaction, visual interfaces, user evaluation, games simple shake of the controller that can be completed in almost any position and in any direction. While this does require the player to use their entire body, it is not much different then pressing a button. 1 Introduction We postulate that taking advantage of the Wiimote’s potential for Rhythm music games, sometimes referred to as sight-reading mu- full body gestures, differentiating between body parts and more re- sic games, feature gameplay that incorporates eye and body co- fined detection of limb pose, direction and acceleration will lead ordination with music. To score well, a player must translate vi- to a new generation of dance-based rhythm gaming. To explore sual cues into actions and perform them at the appropriate time and how to move dance-based rhythm games to the next level, we have in rhythm. While the actions usually correspond to buttons on an developed RealDance [Charbonneau et al. 2009], a dancing game analog game controller, themed experiences through custom hard- prototype that uses four Wiimotes, attached to the wrists and ankles ware controllers such as dancing on a game pad or playing a guitar (see Figure 1). Since RealDance requires a player to use both arms shaped controller are popular as well. This is likely attributable and legs, the player needs to know which of their four limbs to use, where they are expected to move, and when they are expected to ∗e-mail: [email protected] move in the dance sequence. Thus, it is important not only to detect †e-mail:[email protected] these full body gestures, but to also understand how to best convey ‡e-mail:[email protected] the visual cues needed during gameplay. These visual interfaces §e-mail:[email protected] will have to evolve to adapt to the nuances and greater information requirements of these games, while at the same time remaining easy and fun. In this paper, we present a formal user study comparing three differ- ent visual interfaces, Timeline, Motion Lines, and Beat Circles for playing RealDance. Section 2 examines work related to full body motion games and surveys the visual interfaces used in rhythm- based games. Section 3 introduces the RealDance prototype. Sec- tion 4 discusses the three visual interfaces we developed for Real- ward and Paradiso 2006] is used to track dance movements using Dance, based on a review of existing rhythm-based games. Sections accelerometers and gyroscopes attached to the ankles and wrists. 5 and 6 present the user study and results while Sections 7 and 8 discuss future work and conclusions. Labanotation has become the most standardly used iconic repre- sentation and of dance movements. However with many differ- ent shapes, positions, coloring methods, and staffs for placement, 2 Related Work Labanotation literacy entails a large learning curve [Bureau 2007], since it was created for choreography and archival purposes, stress- 2.1 Brief Rhythm Gaming Survey ing precision and accuracy over readability. The study of rhythm games in academics has received little atten- Seventy-six rhythm-based video games from the last decade were tion until recently. One survey compared music game peripheral surveyed to identify trends and categories of interface design. Early controllers [Blaine 2004]. Another conducted an international sur- on, icons streaming across the screen in a timeline fashion were vey of the Dance Dance Revoluton community using an online used to indicate the time of a joystick tilt or a button press. This questionnaire which included questions related to physical moti- interface made up 61% of the surveyed games with little variation vations, social structure, and priorities in game experience [Hoys- among them, including Rock Band, Guitar Hero,andDance Dance niemi 2006]. Other academic research has investigated the heuris- Revolution. A similar interface style, which accounted for an ad- tics of game design [Desurvire et al. 2004]. One promising ap- ditional 18% of the games, arranged the icons radially around the proach to understanding rhythm gaming is the GameFlow model screen. In this interface, icons often emerge from the middle of the [Sweetser and Wyeth 2005], which investigates enjoyment in game- screen and project out in eight directions, splitting the player’s fo- play. To the best of our knowledge, our study is one of the first cus to multiple screen positions for increased difficulty. Gameplay to explore different methods of expressing visual information in sometimes involves an analog thumb stick. Notable examples in rhythm games. this category are Gitaroo Man and EyeToy: Groove. The latest generation of gaming devices has introduced new inter- 3 The RealDance System action. While some games continue the icon scrolling interfaces, a new form of rhythm gameplay evolved incorporating the absolute RealDance seeks to push the limits of the Wiimote hardware and positioning of the Nintendo DS touch screen. In the Japanese game to produce a full-body dancing experience. The goal is for players Osu! Tatakae! Ouendan, players must tap circles in the correct or- to feel like they are dancing naturally and improving their perfor- der and in the correct rhythm, as well as trace lines on the screen. mance. Using readily available commodity hardware they receive This was well received and allowed iNIS to create the sequels Elite feedback directly tied to how well they complete each movement. Beat Agents and Ouendan 2. However, only three other games were RealDance does not rely on button pressing, a staple of video game found to use a similar interface since its release in 2005. In total, controls which contradicts the feeling of dancing. The player is not these games account for 5% of the surveyed games. spatially tethered to a specific location either. Movements which are recognizable as gestures are scored for acting within a certain Tracing lines has been used in other control schemes as well. In We time window, closer to a realistic dancing experience. Cheer, the player holds a Wiimote in each hand, similar to pom- poms. Using color to differentiate left and right, intricate arrowed Four Wiimotes, attached to the user’s wrists and ankles, supply the lines form on the screen and an icon moves along them to indicate data for RealDance. To implement our prototype, we needed a way the timing of the motion. This interface allows for easy description to attach Wiimotes to the user. The requirements for these wearable of movement in three dimensions and presents a clear distinction attachments included comfort, adjustability and secure positioning. between large and small movements. This interface also has few We measured the arms and legs of roughly twenty people of vari- descendants: a Nintendo DS game (Princess Debut) and an upcom- able height, weight, and gender and used this data to design velcro ing pop star game for the Nintendo Wii, making their contribution straps for the forearms and shins. These straps were weaved into account for 4%. modified Wiimote jackets. Our prototype is implemented in C#, using the Bespoke XNA 3DUI Framework [Varcholik 2009] in a The remaining 12% of the games are not relevant to full body gam- Windows environment.