Mixed Reality & Playful Environments IDC 2018, June 19–22, 2018, Trondheim, Norway A conceptual framework to compare two paradigms of Augmented and Mixed Reality experiences. Laura Malinverni, Cristina Valero, Marie-Monique Schaper, Narcis Pares Universitat Pompeu Fabra Barcelona, Spain [email protected], [email protected], [email protected], [email protected] ABSTRACT 1. INTRODUCTION Augmented and Mixed Reality mobile technologies are Augmented and Mixed Reality (AR/MR) mobile becoming an emerging trend in the development of play and technologies offer the evocative potential of making the learning experiences for children. This tendency requires a invisible visible and adding new layers of meaning to our deeper understanding of their specificities to adequately physical world. These features, in the context of Child- inform design. To this end, we ran a study with 36 Computer Interaction, open the path for emerging elementary school children to compare two AR/MR possibilities related to creating novel relations of meaning interaction paradigms for mobile technologies: (1) the between the digital and the physical, between what can be consolidated “Window-on-the-World” (WoW), and (2) the directly seen and what needs to be discovered, between our emerging “World-as-Support” (WaS). By analyzing physical surroundings and our imagination. children's understanding and use of space while playing an AR/MR mystery game, and analyzing the collaboration that This potential has attracted both research and industry, which emerges among them, we show that the two paradigms are increasingly designing mobile-based AR/MR scaffold children’s attention differently during the technologies to support a wide variety of play and learning experience and elicit different forms of collaboration. We experiences. This emerging trend has shaped AR/MR as one conclude by presenting a conceptual framework to of the leading emerging technologies of 2017 [16], hence distinguish the strengths, weaknesses, and potential of the requiring practitioners to face novel ways of thinking about two AR/MR paradigms, as well as the comparison between mobile technologies and designing for them. Consequently, marker-based and marker-less technical solutions. This study a deeper and fuller understanding of AR/MR specificities is aims at helping practitioners in taking design decisions for required to design systems capable of taking advantage of its AR/MR technologies for children. affordances and potential. Furthermore, additional considerations should address how different AR/MR design Author Keywords choices can be more suitable to support specific kinds of play Mixed Reality; Augmented Reality; embodied interaction; and learning experiences for children. embodied cognition; meaning making; Window-on-the- World; World-as-Support. Starting from this perspective, the current research aims at digging deeper into the understanding of AR/MR mobile ACM Classification Keywords technologies for children to offer guidelines for researchers CCS → Human-centered computing → Human computer and practitioners. Specifically, we will focus on examining interaction (HCI) → Interaction paradigms → Mixed / and comparing two AR/MR paradigms for mobile augmented reality technologies: (1) the consolidated “Window-on-the-World” (WoW) interaction paradigm which is based on the screen of a mobile device to merge computer-generated images with the user’s view of the physical surroundings and (2) the emerging “World-as-Support” (WaS) interaction paradigm Permission to make digital or hard copies of all or part of this work for [10] which proposes the use of projective technologies to personal or classroom use is granted without fee provided that copies are project computer-generated images directly on the physical not made or distributed for profit or commercial advantage and that copies environment. bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. To compare these two paradigms, we will first overview Abstracting with credit is permitted. To copy otherwise, or republish, to AR/MR mobile technologies for children and introduce the post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. specificities of the two examined paradigms. Subsequently, we will present a study aimed at investigating how the WoW IDC '18, June 19–22, 2018, Trondheim, Norway and WaS paradigms can shape children's understandings, © 2018 Association for Computing Machinery. collaboration and use of space in a task based on playing an ACM ISBN 978-1-4503-5152-2/18/06$15.00 https://doi.org/10.1145/3202185.3202750 7 Mixed Reality & Playful Environments IDC 2018, June 19–22, 2018, Trondheim, Norway AR/MR mystery game. By contrasting the use of the same use AR to support learning are mainly designed for a application with two devices (one based on the WaS and one collaborative and shared use (e.g. [7,19]). Finally, different based on the WoW), we will show how the two paradigms designs propose different kinds of relationships between the had different implications on how children allocated their users and the physical world. For instance, some applications attention during the experience and elicited different forms do not offer, or require a tight coupling with the physical of collaboration. world (e.g. the location of Pokemons in Pokemon Go does not carry any specific meaning beyond its playability). Other To conclude, we build on the results of the study and on need to be used in a specific site to acquire their full meaning previous research in this field to propose an initial conceptual (e.g. Augmented cultural heritage experiences, which are framework aimed at helping practitioners in making design only meaningful at the specific site). decisions (and defining research questions) to support specific kinds of play and learning experiences with AR/MR Besides reflecting the richness of opportunities, the diversity technologies for children. of AR/MR-based experiences asks for a better understanding of how different configurations can support our intended 2. AUGMENTED AND MIXED REALITY FOR CHILDREN Augmented/Mixed Reality (AR/MR) technologies provide play and learning goals. Starting from this perspective, we compelling experiences that allow users to interact with the examine a consolidated and an emerging paradigm in physical world and computer-generated content. The recent AR/MR mobile technologies to better understand how they advances in the technical capabilities of smartphones and can mediate user experience and respond to different tablets have spurred the interest in AR/MR solutions for purposes, goals and specificities. mobile technologies [5]. This is making them easily 2.1 Consolidated and emerging paradigms in AR/MR accessible and very popular both in the research community technologies: The Window-on-the-World and the World- as well as in everyday experience of users. On the one hand, as-Support specific frameworks are being developed to support the From a technological perspective, most AR/MR applications creation of AR/MR applications (e.g. Google ARCore, for children are generally designed according to the Apple ARKit). On the other, both academy and industry are Window-on-the-World (WoW) interaction paradigm. This increasingly exploring the potential of using AR/MR paradigm represents a well-known approach to make the solutions in different fields such as: education [7], creativity digital and the physical worlds come together. It finds its [24], gaming [1], and cultural heritage [15,19]. This roots in early works on AR [12] and, technically, it is based panorama has shaped AR/MR solutions as one of the leading on using the display of a mobile device to merge computer- emerging technologies of 2017 [16], hence requiring a generated images with images captured from the physical specific effort to fully understand its specificities, surroundings by the camera of the mobile device. This effect affordances and potential. allows users to view virtual elements on the physical world and eventually interact with them to enable specific In the context of Child-Computer Interaction, AR/MR functionalities [14]. technologies have been employed to support a wide variety of play and learning experiences. Examples can be found in: The limited hardware requirements of the WoW have augmented reading and coloring books (e.g. [4,25]); AR transformed this solution in the dominant paradigm of videogames (e.g. Pokemon Go); applications that allow AR/MR applications. Nonetheless, as Betsworth et al. [3] augmentation and addition of multimedia contents to and Müller at al. [14] pointed out, this paradigm can present curricular materials (e.g. [26]); systems that use AR to some relevant limitations. First, by requiring users to hold support learning while visiting cultural heritage sites (e.g. their devices in front of the physical world, the system runs [7,19]); and applications that allow children to create their the risk of becoming a “digital divider” [3]. Second, the own AR contents (e.g.[27]). prominence of the digital content may reduce the attention that users pay to the physical environment during the This diverse panorama encompasses a wide variety of experience [3]. technological solutions and miscellaneous experiential goals. Some relevant distinctions can be drawn in relation to: To respond to these limitations, novel technological (1) the goals pursued by the application; (2) the social solutions are being developed. From the review of the state context for which the AR/MR activity is designed; and (3) of the art in the field of HCI, we identified the rise of a new the relation that the system establishes with the physical AR/MR mobile-based technological configuration. We world. In relation to the pursued play and learning goals, analyzed this configuration conceptually and defined its applications that allow children to create their own AR specific potential. We decided to call this novel approach the contents mainly target open and creative activities (e.g. “World-as-Support” (WaS) interaction paradigm and Aurasma).