Personal and https://doi.org/10.1007/s00779-017-1107-0

ORIGINAL ARTICLE

A method to assess pervasive qualities in mobile

Luis Valente1 & Bruno Feijó2 & Julio Cesar Sampaio do Prado Leite3 & Esteban Clua1

Received: 2 December 2016 /Accepted: 11 December 2017 # Springer-Verlag London Ltd., part of Springer Nature 2017

Abstract Software designers are facing huge challenges imposed by a new generation of applications that mix real and digital worlds, such as pervasive games. This type of has recently become a worldwide phenomenon, with thousands of people walking in the streets with smartphones to interact with the physical environment. In this paper, we propose a new method to assess pervasive qualities in pervasive mobile games, which can be customized and extended to other ubiquitous applications. This method generates a quality report, which consists of a quality spreadsheet (containing metric values and comments) and a quality vector (representing the game quality profile in the form of a bar chart). In addition, we can compare quality vectors using similarity criteria. In this paper, we apply the proposed method to commercial and academic prototype games to shed more light on their pervasive characteristics and identify ways to improve the overall quality that sets these games apart from traditional digital games—that is, pervasiveness.

Keywords Pervasive mobile games . Pervasiveness . Non-functional requirements . Ubiquitous systems .

1 Introduction games. In a broad view, we can classify these apps and systems as ubiquitous, in the sense that they can be on Software designers are facing huge challenges imposed by a everything, everywhere, and anytime, and can naturally in- new generation of apps and systems that mix real and digital tertwine real and digital worlds through multimodal inter- worlds, mostly using natural interfaces. These systems are faces. Moreover, these systems present new characteristics changing the way we work, communicate, learn, and have not found in conventional software (such as being pervasive, fun. Examples of such new systems are intelligent personal immersive, affective, and enjoyable) and in some cases, they assistants (e.g., Siri, Cortana, Google Now, and Echo), vir- bring new paradigms of UI concepts [1]. These characteris- tual systems, the internet of things, and pervasive tics are quite distinct from the more concrete non-functional requirements in software engineering (e.g., Bsafety,^ Breusability,^ and Befficiency^). They also contrast with * Luis Valente the more abstract attributes found in non-functional require- [email protected] ments. They seem to be closer to intangible and non- measurable qualities, such as Bfun,^ Baesthetic,^ Bexciting,^ Bruno Feijó [email protected] and Bflow,^ which are subjective softgoals. For example, these kinds of requirements are common in digital games Julio Cesar Sampaio do Prado Leite – [email protected] [2 4], deriving from emotional aspects regarding the game experience. Consequently, some researchers have referred to Esteban Clua these requirements as Bemotional requirements^ [2, 3], [email protected] Baffective factors^ [5], and Bexperience requirements^ — 1 Institute of Computing, UFF, Av. Gal. Milton Tavares de Souza, s/n, [6] this last one including gameplay factors, emotional fac- São Domingos, Niterói 24210-346, RJ, Brazil tors, and sensory factors. Therefore, we need to revisit the 2 VisionLab/Department of Informatics, PUC-Rio, Rua Marquês de issue of quality requirements in software engineering for São Vicente 225, Gávea, Rio de Janeiro 22451-900, RJ, Brazil ubiquitous apps, systems, and games. 3 Department of Informatics, PUC-Rio, Rua Marquês de São Vicente In this paper, we approach this line of investigation by 225, Gávea, Rio de Janeiro 22451-900, RJ, Brazil proposing a method to assess the Bpervasiveness quality^ in Pers Ubiquit Comput pervasive mobile games. These games are a subset of perva- e) Facilitate teaching, studying, and comprehending what sive games where the main game playing device is mobile pervasive mobile games are and how they are structured; (e.g., networked, portable, and context-aware devices such f) Analyze several games to create (design and/or implemen- as smartphones and tablets). Examples of pervasive mobile tation) catalogs and guidelines about how to apply specific games are Pirates! [7], Botfighters [8], PAC-LAN [9], pervasive features. REXplorer [10], Zombies Run! [11], Spellbound [12], Ingress [13], and Pokémon Go [14]. We believe that our assessment method can be adapted to Pervasiveness is the elusive quality that differentiates other ubiquitous systems, although we have not yet validated Bpervasive games^ from traditional digital games. this claim. In any case, tools, techniques, and guidelines to Pervasiveness holds several meanings, such as being ubiqui- assess pervasive qualities are highly dependent on the specific tous, permeating something, and spreading something (or type of ubiquitous system. Therefore, we dedicate this paper somewhere) in a physical space. In games (as in any other to pervasive games on mobile platforms (i.e., smartphones, application), we face enormous difficulties to define the role tablets, wearable devices), and we believe that the lessons of this elusive quality. In a previous work [15], we discussed learned here may contribute to a better practice of pervasive different (and often conflicting) approaches to the concept of quality assessment in the design of ubiquitous systems. There pervasiveness in games. Considering this scenario, instead of are different reasons for starting this type of investigation with trying to provide yet another, all-encompassing, theoretical def- pervasive mobile games. First, games are software applica- inition of pervasiveness, we opted for a more practical approach tions that face almost every aspect of computing challenges. by proposing a taxonomy of Bpervasive game qualities^—aset Secondly, games permeate many different areas (from social of quality attributes that contributetocharacterizeagameas simulation to playful learning). Thirdly, pervasive games have Bbeing pervasive.^ The driving question for the first version of recently emerged as apps with a strong (and sometimes unde- this taxonomy was Bwhat makes a game pervasive?.^ We iden- sirable) impact on large communities (e.g., Pokémon Go). tified these qualities by analyzing 24 pervasive mobile games, Some software attributes are not features of a system. reviewing the game literature, and using the authors’ experi- Instead, these attributes are qualities (also called non- ences in developing mobile and context-aware games. functional requirements); some examples are usability, reli- The main objective of our assessment process is to help ability, maintainability, and portability. They are global con- stakeholders [16] (e.g., interested parties, such as game de- straints on a software system. Software quality primarily con- signers, game developers, programmers, and researchers) in cerns Bfitness to purpose^ and is highly dependent on the the following two tasks: (1) creating new games that contain application domain [17]. This imposes a great challenge dur- the desired pervasive features and (2) evaluating existing per- ing design, because we need to predict how well the system vasive mobile games, whether commercial or academic pro- will fit its purpose without having the opportunity to test it in totypes. Considering the first task, our assessment process the future environments. This delayed feedback requires in- offers tools (e.g., Bquality vectors^ and Bchecklists,^ see tensive use of models and simulations during the design stage. Section 2.5) to help developers, programmers, and designers There is a vast literature on quality predictors and measure- to make informed decisions about which pervasive qualities ment methods for non-functional requirements [18–22]. the game should support to create a product with better qual- However, there are few studies about non-functional require- ity. Concerning the second task, we foresee the following ments for ubiquitous systems and, especially, pervasive benefits: games. Taxonomies of pervasive qualities in games are rare (see a) Discover which pervasive qualities a game contains; BRelated works^ in Section 4). In a previous work [15], we b) Learn how a game applies pervasive qualities (e.g., design proposed a conceptual map of quality requirements that helps and implementation), with the added benefit of possibly understanding Bpervasiveness^ in pervasive mobile games discovering design and/or implementations flaws that im- (see Section 2.4 below). In this map, we decomposed pair the pervasive quality; Bpervasiveness^ into two levels of non-functional require- c) Given a) and b), the stakeholder could detect issues that ments, which we named Bpervasive qualities^ (Table 1). may hinder pervasiveness in an existing game, creating Additionally, this map contains a set of questions for each the opportunity to improve how that specific game applies quality requirement, which helps designers in evaluating or pervasive qualities (in case the stakeholder is also the introducing pervasiveness (and its subcomponents) in perva- game developer and/or designer); sive mobile games. In this paper, we propose a method to d) Learn about pervasiveness in general by assessing several assess pervasiveness in pervasive mobile games, which ap- games, enabling the stakeholder to get acquainted with plies the pervasive qualities and checklists found in [15]. this concept and even to extend our taxonomy later when We consider that assessing quality features in ubiquitous she/he is more experienced; systems requires a five-step macro process. Concerning Pers Ubiquit Comput

Table 1 qualities that contribute to pervasiveness First-level qualities Second-level qualities [15] Spatiality (Spa) Mobility (), Local space redefinition (LSR) Permanence (Perm) Persistency (Per), Daily life interleaving (DLI) Communicability Connectivity (Con) (Comm) Accessibility (Acc) Access usability (Usa), Device independence (DI), Transmediality (TM), Game autonomy (GA) Context-awareness Game content adaptability (GCA), Game object tangibility (GOT) (CA) Resilience (Res) Uncertainty handling policy (UHP), Game pacing (GP) Sociality (Soc) Social communication (SC), Conformance to physical and social settings (CPS), Involving non-players (INP) pervasive mobile games, we explored the first four of these aims to answer the list of questions associated with each steps in [15]. In this paper, we enhance the fourth step and second-level quality requirement and to produce quantitative include a fifth step. We start by defining these steps for the reports. case of pervasive mobile games as follows:

1. Identify the fundamental quality. In the case of pervasive Figure 1 illustrates a simplified view of our assessment pro- mobile games, this fundamental quality is Bpervasiveness.^ cess. This assessment process allows the requirements engineer to Pervasiveness is the unique, subtle feature of pervasive generate a final quality report that summarizes the quality assess- systems, while immersion is the key concept in virtual re- ment of a particular pervasive after analyzing a large ality systems, for example. Other ubiquitous systems may collection of information about this specific game. The quality require more than one fundamental quality; report consists of a quality spreadsheet (containing metric values 2. Classify the system by boundary criteria. Boundary and comments) and a quality vector (representing the quality criteria help the requirements engineer to narrow the set profile of the game in the form of a bar chart). Also, quality of possible games to consider. For example, Table 2 pre- vectors can be used to measure quality similarities between dif- sents criteria we used to classify games as Bpervasive ferent games. When developing this assessment process, the au- mobile games^ [15]; thors produced several supporting materials (e.g., mark-ups, key- 3. Decompose the fundamental quality into refined words, and indications of information groups) that are useful for requirements. Table 1 presents a decomposition of perva- software engineers and designers. Indeed, practitioners can use siveness into two levels of requirements [15]. At the first this material to highlight important information and produce rel- level, Table 1 presents 7 non-functional requirements evant annotated material related to first-level qualities, which (qualities) that contribute to pervasiveness. On the second leads to a detailed analysis of second-level qualities. level, there are 16 non-functional requirements that contrib- This paper is organized as follows. Section 2 presents the ute to the first-level qualities. We refer to this second level as assessment process we propose in this paper. Section 3 pre- Bpervasive mobile game qualities,^ as they are qualities sents a summary of two case studies. Section 4 discusses re- more specific to pervasive mobile games. Using the lan- lated works. Section 5 presents some final conclusions and guage of the NFR framework [23], the first-level non-func- directions for future studies. tional requirements and the pervasive mobile game qualities can be considered NFR softgoals. Table 1 describes the contents of a NFR SIG (softgoal interdependency graph), 2 Assessing pervasive qualities in pervasive and all hierarchical relationships in this table are equivalent mobile games to Bhelp^ contributions in the NFR framework; 4. Define checklists (questions) for each second-level Figure 2 illustrates a BPMN diagram1 (Business Process requirement. In this step, we apply the checklists we pro- Model and Notation) that includes our six-step process to posed in [15]. However, in the present paper, we enhance evaluate pervasive game qualities (in the Bevaluator^ lane). these checklists with the following elements: clarifying This diagram features four actors: taxonomist, author, comments, a question classification system, and a set of metrics to be used in the assessment process; 1 The reader should note that we annotated the BPMN (i.e., the traced line and 5. the indication of hierarchy) to better explain the levels in the assessment pro- Apply the assessment process. This is a six-step process, cess. We chose to avoid using a swimlane in this picture (i.e., the actor), as which is the main contribution in this paper. This process there is only one actor in this case (the user of the process). Pers Ubiquit Comput

Table 2 The criteria to classify games as Bpervasive mobile games^ in specifications, scientific literature, news, product descriptions, [15]. The asterisk denotes a mandatory condition and manuals. Identification of information sources is fundamen- Condition Description tally important to elicit information about the focus of interest (pervasive mobile games in our assessment process). C1* Games that use context information We have used different information sources to elicit pervasive C2* Games that use mobile devices as the main game interfaces game qualities including research literature and media assess- C3 Games that access remote data on the move ments (e.g., informal materials, reports, non-peer-reviewed pa- C4 Multiplayer games pers, web sites, interviews, presentations). Sometimes, media as- sessments are the only available information source about a game (e.g., commercial games that are no longer available). These as- evaluator, and end user. Currently, the Btaxonomist^ actor re- sessments are usually rich, full of player and expert user knowl- fers to the present authors. Next, the Bauthor^ actor represents edge about the game world, and as such, they are valuable input to all the authors who create and publish information sources understand how a game works. Also, many pervasive mobile about the game, including the game application itself. games are research prototypes unavailable to the general public, Sections 2 to 2.5 describe each step the Bevaluator^ actor but they usually have academic papers published elsewhere. conducts to assess pervasive game qualities. The Bevaluator^ Finally, there are commercial games available in application may produce a Breport^ at the end of the process, which could stores, allowing developers, researchers, and end users to down- be accessed by Bend users^ interested in learning about per- load and play them. For our purposes, we keep all these informa- vasiveness aspects in a given game. The Bend user^ actor tion sources in a database. We identify the following types of represents game designers, developers, programmers, re- information sources that we can use in our assessment process: searchers, and requirement engineers. IS1. (Bpapers^) Academic papers; IS2. (Bofficial site^) The official game website, which may 2.1 Searching for available information sources (Steps contain information about the main game features (i.e., 1and2) commercial, selling points); IS3. (Bvideos^) Videos describing the game. These videos The process starts by selecting an information source. According may be official (e.g., created by the game developers to [24], information sources are not restricted to people (also and companies) or not (e.g., videos featuring gameplay referred as stakeholders, users, or clients) but may include differ- made by players); ent sorts of documents, such as books, internal memos,

Fig. 1 Simplified view of our assessment process applied to a game. We analyze highlighting information related to first-level qualities and then we analyze the corresponding second-level qualities using the checklists. The quality vector summarizes the final report. Some parts are computer-assisted Pers Ubiquit Comput

Fig. 2 The complete BPMN process diagram describing our process to assess pervasive game qualities

IS4. (Bnews^) Game news websites, which may contain in- IS9. (Bmarketing^) Marketing campaigns about the game. formal non-academic articles (e.g., blog posts, reports) and game reviews. These (informal) reviews may be written by players and mainly depict their impressions about the gameplay; 2.2 Determining whether a game is a pervasive IS5. (Bdev articles^) Informal, non-academic articles mobile game (Step 3) that designers/developers of a game write about that specific game. In the game industry, there The borderline of pervasive and non-pervasive games may be are Bpost-mortem^ articles that describe details fuzzy. We consider a game to be a Bpervasive mobile game^ about the overall game development process, if: which may include information about what went right and wrong about the game design, imple- & The game uses context information in game activities (i.e., mentation, and deployment. These articles may the condition C1 in Table 2). For example, the game may appear in websites dedicated to game developers; use context (e.g., location) to create game content. IS6. (Binterviews^) Interviews (e.g., text, video, and audio) Another possibility is to modify and adapt game rules with the game developers about the game; according to contextual information; IS7. (Bpresentations^) Presentations and talks in game de- & The game implements at least one game interface in veloper conferences; context-aware mobile devices (i.e., the condition C2 IS8. (Bhands-on^) Hands-on experience, which includes de- in Table 2). Examples of context-aware mobile de- veloping games and evaluating games developed by vices are smartphones and tablets. We consider a other people. In the latter case, it means downloading game a Bborderline pervasive game^ if the game is and playing the game to conduct a custom evaluation transmedia (see [15]foradiscussiononBpervasive process; games and transmediality^). Pers Ubiquit Comput

The first task to classify a game as a pervasive mobile game (concerning Step 4) before deciding whether the game is a is to explore an adequate information source by searching for pervasive mobile game or not. specific keywords and expressions (mark-ups) that are typical of pervasive games (mobile or not). The following tables can 2.3 Identifying useful information groups (Step 4) be used for this task: Table 3 lists common names (for game genres and game styles) that refer to pervasive games; Table 4 We can classify useful types of information to assess pervasive lists keywords and expressions that refer to context-aware qualities into the following groups: mobile devices that pervasive games may use (which is part of condition C2 in Table 2); and Table 5 lists general keywords IG1. The Bdesign^ group, which may be found in informa- and expressions related to pervasive games. These mark-ups tion sources under the following names: Bgame design, come from analyzes of a large number of pervasive mobile game description, game plot^. This information group games [15, 25] we conducted in other works. provides descriptions about the overall game design, Additionally, the following tables in Section 2.4 (i.e., Step main game features, game narrative, game story, game 5—identifying first-level qualities) can be useful to assist in mechanics, and game activities. This information recognizing a pervasive mobile game: Table 6 (Spatiality, group usually appears in information sources IS1 to which also strongly relates to condition C1) and Table 10 IS7 (i.e., Bpaper,^ Bofficial site,^ Bvideos,^ Bnews,^ (Context-awareness, which relates directly to condition C1). Bdev articles,^ Binterviews,^ Bpresentations^); Computer programs could assist this part of the assessment IG2. The Bimplementation^ group, which may be recog- process by highlighting the abovementioned keywords and nized in information sources by the following names: expressions (mark-ups) for visual and auditory inspections. Bimplementation, game architecture, game infra- Also, the following questions may aid in determining whether structure.^ This information group provides implemen- a game meets the C1 and C2 conditions of Table 2: tation details, system architectures, game components, and related topics. This kind of information is useful to & Which devices do players use to play the game? assess pervasive qualities that have aspects influenced & Are these devices portable and context-aware? by implementation issues, such as Uncertainty han- & Which sensors does the game use? dling policy, Connectivity, Persistency, and Game ob- & What kinds of context information does the game use in ject tangibility. This information group usually appears game activities? in information sources IS1, IS5, IS6, and IS7 (Bpaper,^ Bdev articles,^ Binterviews,^ and Bpresentations^); IG3. The Bevaluation^ group, which may appear as We claim, based on our experience, that if the information Bdeployment, experiments, validation, user tests, eval- source is an academic paper (IS1), we should first browse the uation, discussion, post-mortem.^ The evaluation pro- paper title and abstract, searching for keywords and expres- cess may be conducted by researchers with actual sions that match those in Tables 4, 5, 6,and10.Thistypeof players or with the developers themselves, constituting inspection of academic papers is usually sufficient to decide formal or semi-formal processes. This kind of infor- whether a game is a pervasive mobile game or not. mation is useful to assess aspects that are better ob- If all the abovementioned procedures are not effective to served while playing the game. Pervasive qualities that classify the game as a mobile pervasive one, we can use the include these aspects are Uncertainty handling policy, procedures of the next step (Step 4, Section 2.3), which iden- Game pacing, Social communication, Conformance to tifies useful groups of information that are helpful to assess the physical and social settings, Connectivity, Game con- first and second levels of pervasive qualities. We claim, also tent adaptability, Game object tangibility, and Local based on our experience, that if the available information space redefinition. This information group usually ap- source is non-structured or informal (e.g., IS2 to IS7 pears in information sources IS1, IS5, IS6, IS7, and IS7—Bofficial sit,^ Bvideos,^ Bnews,^ Bdev articles,^ IS8 (Bpapers,^ Bdev articles,^ Binterviews,^ and Binterviews,^ Bpresentations^), we should briefly explore it Bhands-on^);

Table 3 Common genre/style names of pervasive games Transreality game Exergame Transmedia game Urban storytelling game

Mixed-reality game Live- Mobile mixed-reality game game Location-based game Ubiquitous game Crossmedia game Ambient game Urban game Tourism game Outdoor game Computer-augmented game Context-aware game Pers Ubiquit Comput

Table 4 Expressions and keywords related to context- Mobile platforms (e.g., iOS, Android) Specific mobile device manufacturers Smartphone Tablet aware mobile devices Wearable devices Mobile phone

IG4. The Bgameplay^ group, which may appear as Bgame information source, especially concerning the quality attri- playthrough, walkthrough, hands-on.^ This informa- butes. Secondly, we want to elicit the most relevant first- tion group describes the gameplay dynamics. The main level qualities of the game. This analysis may help to conduct difference from the IG3 group is that in IG4 there is no Step 6—assessing second-level qualities using the checklists. evaluation (i.e., it is an informal process). Information This section presents common keywords and expressions sources that usually contain this group are IS3, IS4, and (mark-ups related to first-level qualities) that help in IS8 (i.e., Bvideos,^ Bnews,^ and Bhands-on^). conducting Step 5. These keywords and expressions do not represent all possibilities. These mark-ups come from our pre- If the information source is textual, we may browse it to vious works, based on the analysis of a large number of per- highlight these groups in the text (if possible). The IG1 and vasive mobile games [15, 25]. This section briefly describes IG2 groups can be useful to determine whether a game is a the first- and second-level qualities. For detailed descriptions, pervasive mobile game if the previous step did not provide please refer to [15]. enough information to classify the game. If the information source is not text-based (e.g., audio, vid- 2.4.1 Spatiality eo), we may need to browse it more carefully, taking notes to identify these information groups. Although an information Spatiality refers to aspects such as the physical place where source may have these information groups, their existence the game happens and mobility. The physical place where the does not guarantee that there will be enough information to game happens (e.g., Bthe game scene,^ Bgame area,^ Bgame assess the pervasive game qualities. arena^) usually corresponds to outdoor and public areas of Some types of information sources may not contain all urban places, but these places may also be dedicated indoor information groups—for example, a hands-on evaluation pro- physical installations customized for a specific game. cess (i.e., IS8 information source) will not provide implemen- Spatiality is composed of Mobility and Local space tation details to assess the Uncertainty handling policy quality redefinition. (Section 2.4.6) for a given game (assuming the Bevaluator^ Mobility (Mob) refers to two aspects. The first is mobile does not correspond to the game developer). In this case, this computing—using wireless connections on mobile devices. assessment may require other types of information source, The second corresponds to the requirements of movement in such as IS1, IS5, IS6, and IS7 (i.e., Bpapers,^ Bdev articles,^ the game and the physical size of the game area, which may Binterviews,^ and Bpresentations^). require players to move through great distances to complete game activities. 2.4 First-level quality analysis (Step 5) Local space redefinition (LSR) refers to how the game is able to change the meaning of the places where the game In Step 5, we browse the information groups (if any) identified happens, for example, by augmenting value to physical places in the previous step to conduct a preliminary analysis, (using virtual content), integrating the physical place in the highlighting and taking notes of some keywords, expressions, game (or objects in this place), using actors to portray non- and excerpts related to the first-level qualities. The objectives playing characters, helping the player to learn about the phys- here are twofold. The first is to get acquainted with the ical environment, and having the players perceive the

Table 5 Common keywords and expressions related to pervasive Augmented reality Live Interactions with physical Blending physical and virtual games performance objects worlds

Computer-augmented Smart objects Pervasive platforms Brings game to the physical world game Exploration and Pervasiveness Players act out in the Combination of virtual and real discovery physical world worlds Extend the virtual Play in Beyond the device screen Ubiquitous computing gaming world environment Ubiquitous computing Play in public places Pers Ubiquit Comput

Table 6 Common keywords and expressions related to Spatiality requirement for games that aim at simulating a parallel world Annotate the Collect virtual Computational Explore a that evolves by itself. This includes all games that have social physical content in the elements in the physical place networking or community aspects. However, not all pervasive environment physical world physical world games require persistency. Daily life interleaving (DLI) refers to how a pervasive mo- Use Landmarks Locality Location-based information bile game is able to diffuse through the daily lives of players, about the interleaving game playing with non-game activities. For physical example, many traditional games for mobile phones are space Bcasual games,^ meaning they have simple gameplay Place content in Physical Physical Physical place B ^ physical movement as navigation augmented and players use them as intervals of daily life. This locations game with virtual quality includes aspects such as persistent game worlds, mechanic content short-play sessions, and asynchronous communication of Physical world Play with QR codes Spatial game-related events to players. as game location expansion While assessing Permanence, we search for clues, key- board words, and expressions that describe game session details— Walk in the Wide-area game Outdoor game Physical world as environment stage for example, (1) How long a game session lasts, if completing Play in public Mobility game activities require multiple game sessions, and (2) If places game sessions are independent (e.g., they do not depend on previous game sessions). We also search for information about persistent virtual environment with alternative viewpoints. These ideas are dif- worlds and how the game communicates with players when ferent from just Bbeing in a place^ and using some physical they are not engaged in the game. Persistent worlds in perva- property (e.g., location) without explicit reference to the local sive mobile games may evolve or degrade independently of surrounding context. player input or participation. Table 7 lists common keywords A preliminary assessment of Spatiality consists of iden- and expressions to search for Permanence. tifying information that describes how the game uses physical space and its elements. For example, some related terms are Bplay area,^ Bgame area,^ Bplay space,^ 2.4.3 Communicability Bplayground,^ and Bgame space.^ The physical space where the game happens may be a public place or a Communicability concerns aspects related to how the private (and customized) place. This information may help game communicates with players (and vice-versa). to assess the second-level qualities (Local space redefini- Communicability contains one second-level quality, tion and Mobility) using the checklists. Another useful which is Connectivity (Con). This quality refers to net- information relates to specific locations and physical ob- working usage in pervasive mobile games. For example, jects (e.g., statues, landmarks) the game uses. This infor- an important aspect is the connectivity scope that a mation may help to assess Local space redefinition, game requires—global, local, or both. Global scope re- Mobility, Game autonomy (Accessibility), and Game con- fers to using networking solutions as 3G networks. tent adaptability (Context-awareness). Table 6 lists com- Local scope refers to using Wi-Fi and Bluetooth tech- mon keywords and expressions related to Spatiality. nologies. Global scope enables the game to connect re- mote peers and to use networking in wide-range areas. 2.4.2 Permanence Local scope enables the game to connect co-located peers (e.g., players and servers). The size of the physi- In an ultimate state of pervasiveness, a pervasive mobile game cal space where the game takes place directly influences becomes a game world that exists as an independent entity connectivity requirements. parallel to Breal life.^ The most prominent aspect of this idea Assessing Communicability consists of identifying the is the possibility to engage with a process that evolves contin- communication structure that the game uses. For example, uously over time (and does not depend on whether the player the game may be single-player (no connectivity), single- is connected to the game or not). Permanence denotes the idea player connected, and multiplayer. Observing the communi- of a game world being a persistent entity available anytime, cation scope (co-located, global, or both) is also important. anywhere. Permanence is composed of Persistency and Daily The information group IG2 (i.e., Bimplementation^ group) is life interleaving. useful to learn details about Connectivity, in the next step. Persistency (Per) refers to maintaining the game state over Table 8 lists common keywords and expressions to search time for future game sessions over time. This quality is a for Communicability. Pers Ubiquit Comput

Table 7 Common keywords and expressions related to Embed play with daily life Interweaving with everyday life MMORPG Persistent world Permanence Pervasive RPG Play anywhere, anytime Short-play sessions Temporal expansion Temporality

2.4.4 Accessibility aspects. Table 9 lists common keywords and expressions for Accessibility. Besides, other examples of useful clues and in- Accessibility refers to how players can access the game and formation are the following: issues that might prevent a player from accessing the game, including social, economic, and technological issues. 1. Concerns about including several groups of people in Accessibility is composed of Transmediality, Game autono- game activities. For example, integrating people with dis- my, Access usability, and Device independence. abilities in game activities; Transmediality (TM) refers to providing different game in- 2. Concerns about providing access to technology that was terfaces (i.e., modes of participation or game roles) using dif- not widely available. For example, lending devices to ferent types of devices to deliver a broader transmedia expe- players; rience. These devices might be context-aware mobile devices 3. Concerns about using devices that are already common to (such as smartphones), desktop computers, web-based inter- the general public in game activities. For example, design- faces, wearable devices, environment devices, or custom hard- ing games that use current smartphones and tablets to ware [15]. In transmedia pervasive games, each game inter- implement the main game interfaces; face is associated with a type of device, enabling players to 4. Concerns about defining a target user group and providing play in different roles in the game. These roles may be collab- amenities to reach this group. For example, designing orative or not. Epidemic Menace [26] is an interesting exam- games for children. ple of a pervasive game that uses this quality. Game autonomy (GA) refers to the degree to which a game is able to work as an Bindependent system^—requiring no specific environment setup or live game management (i.e., 2.4.5 Context-awareness orchestration). In games with high game autonomy (i.e., Bautonomous games^), players are able to readily play the Context-awareness helps create the mixed-reality that perva- game anywhere, anytime. Games with low game autonomy sive mobile games provide. In particular, this quality refers to require a specific place, often with dedicated infrastructure using acquired and context information in game activities. (e.g., sensors spread in the environment, dedicated Wi-Fi net- This quality is composed of Game content adaptability and works, public displays) to support the game activities. Also, Game object tangibility. games with low game autonomy may be considered as Bevent Game content adaptability A(GC ) relates to pervasive mo- games^ because these games have a scheduled time to start bile games that generate game content dynamically using sen- and last for a specific duration. sor and context data. This quality concerns how the game is Access usability (Usa) corresponds to traditional usability able to keep the designed functionalities anywhere regardless issues from human-computer interaction focused on mobile of context availability conditions. devices and accessibility. Device independence refers to the Game object tangibility (GOT) relates to how the game possibility of using a game interface in multiple platforms uses mobile devices as Btangible objects^—an object that (e.g., Android, iOS, and web). players grasp and manipulate in game activities, which may To assess Accessibility, we browse the information groups have a specific purpose (e.g., a role) in the game narrative. identified in the previous step looking for information and We start assessing Context-awareness by searching for clues that describe issues that might affect access to the general uses of context in game activities. Common types of game—considering social, economic, and technological context in pervasive mobile games are location (e.g., GPS,

Table 8 Common keywords and expressions related to 3G/4G networks Ad hoc networks Asynchronous communication Asynchronous gameplay Communicability Bluetooth Cellular connection Chat Client-server Co-located play Multiplayer Networking NFC SMS Text messaging Wi-Fi Mobility Communities Social network Pers Ubiquit Comput

Table 9 Common keywords and expressions related to Crossmedia Current smartphones Event-based game Haptics feedback Accessibility Multimodal user interface Public display Voice interaction Transmedia Multi-platform Game prop Orchestration Game moderators specific physical objects, QR codes), spatial area information 2.4.7 Sociality (cell-id, Wi-Fi), proximity (e.g., beacons, QR codes, RFID tags, Bluetooth, NFC). Other possibilities are physiological Sociality refers to the game’s social aspects, being composed information (e.g., heart rate), interaction context (gestures, of Social communication, Conformance to physical and social embodied interactions), traveled distance, speed, weather in- settings, and Involving non-players. formation, social context, social network profiles, and envi- Social communication (SC) refers to how game activ- ronment information (e.g., noise level, temperature, light lev- ities enable players to communicate and socialize. The el). Table 10 lists common keywords and expressions to game may enable social communication between distrib- search for Context-awareness. uted and/or co-located players. In the first case, the game mediates this process using technology artifacts, such as mobile devices and network infrastructure. In 2.4.6 Resilience the second case, players may communicate face-to-face directly (without technology artifacts) or by using the Resilience refers to the capacity of a game to keep the game as a mediator (through technology). game experience smooth despite technological issues that Conformance to physical and social settings (CPS) might break or disrupt this experience while players are refers to issues such as ethics, privacy concerns, engaging with the game. For example, technology compo- conforming to social conventions, social context, safety nents in pervasive mobile games (especially sensors and concerns (e.g., protecting players from harm while mov- networking) have inherent limitations that generate ing around), and adequacy to physical settings (e.g., Buncertainties.^ Uncertainties arise from Bseams,^ which using audio feedback in noisy places may not work Barkhuus et al. [27]definedasBabreak,gapor‘loss in well). translation’ in a number of tools or media, designed for Involving non-players ( INP) refers to integrating non- use together as a uniformly and unproblematically experi- players into the game. Montola et al. [31] identified five par- enced whole.^ Seams appear due to some of the following ticipation modes for non-players in general pervasive games: factors: limited or variable availability, limited or variable BThe outsiders can be obstacles (Cases A and D), witnesses coverage, limited accuracy and precision, intermittent op- (Cases C, F, and J), an audience (Cases G and M), or tokens to eration, sudden malfunctions, and other issues. Resilience be collected (Case I) and they can be invited to become fully is composed of Uncertainty handling policy and Game fledged player–participants (Cases B, E, and J).^ In this paper, pacing. we add four possibilities to this list: Bhuman actors as Uncertainty handling policy (UHP) refers to how a game performers,^ Baudience as co-participators,^ Bbeing handles seams and uncertainties in game activities, which gen- approached by players^ (a general case where non-players erally takes form through five general strategies: remove, hide, interact with players but do not become a player), and exploit, reveal, and manage [28–30]. Bsources of game content^ (a general case that include Btokens Game pacing (GP) refers to two aspects: (1) how the pace to be collected,^ by Montola et al. [31]).Notably,applying of game activities may disrupt the operation of technology this quality in games might raise ethical issues. components and (2) how the operation of technology compo- Assessing Sociality means searching for clues, key- nents may limit the pacing of game activities. Usually, these words, and expressions that convey how the game is technology components correspond to sensors and network- able to influence and involve people through game ac- ing infrastructure. tivities—these people may be players and non-players. In assessing Resilience, we search for general uses of Table 12 lists common keywords and expressions to sensor information and networking, which are the main search for Sociality. sources of uncertainties in pervasive mobile games. In particular, we pay attention to problems and issues re- 2.5 Applying checklists to assess second-level ported in the information source and also try to foresee qualities (Step 6) possible issues not anticipated by the game developers. Table 11 lists common keywords and expressions to In Step 5, we produced annotated material that highlights in- search for Resilience. formation about first-level qualities of a game. In Step 6, we Pers Ubiquit Comput

Table 10 Common keywords and expressions related to Beacons Bluetooth Body sensors Cell-id Cellular Context-awareness positioning

Context Context-aware computing Context-dependent Embedded Embodied gameplay interactions interactions Gesture-based GPS Implicit Location Location interaction interactions aware sensing Location-based NFC Physiological input Proximity QR codes RFID tags Change rules based on Sensor networks Sensor-based Sensor-based context information input interaction Sensors Social context Tangible objects Wi-Fi browse this material to apply the checklists [15]ofeach There is another important issue affecting quality vectors. second-level quality. In this paper, we extend the material A well-known fact in requirements engineering is that quality we described on our previous work [15] with metric rules attributes are interdependent and cannot be achieved in isola- and additional comments. To calculate quality measures, we tion. In a previous work [15], we started analyzing the inter- created a spreadsheet, which we name as Bquality dependences between pervasive quality attributes, and we spreadsheet^ in this paper.2 This spreadsheet implements the proposed some basic relationships (i R j for Bi is required by metric rules (detailed in Section 2.5.1) and calculates scoring j,^ i + j for Bi helps j,^ and i − j for Bi hinders j,^ where i and j (i.e., metric values) automatically as the evaluator (e.g., de- are two different quality attributes) that can be affected by signer, requirements engineer) answers the questions. symbols of modality (e.g., a for Balways holds^ and m for Bmay hold, in some specific cases^). Examples in [15] are UHP aR GCA (i.e., Uncertainty handling policy is always 2.5.1 Quality vector and metrics required by Game content adaptability), Usa – Mob (i.e., Access usability hinders Mobility), and GP m– TM (i.e., Each question in the checklist of a quality attribute is designed Game pacing may hinder Transmediality in some cases). to reveal a particular aspect of this attribute. We can have a Therefore, no app or game can have an Ball-ones quality quantitative assessment of these quality attributes by calculat- vector^ (i.e., a vector with all elements equal to 1.0), because ing a special vector called the Bquality vector.^ The quality some quality attributes may hinder others (e.g., Access vector V is an array of values vi such that vi ∈[0,1], i =1, n, usability hinders Mobility). We could use these interdepen- where n is the number of second-level qualities (n = 16, in this dence relationships as a consistency check for a given quality paper), and the value vi is the Bsupport level^ of the quality i, vector, but we think we need further investigation into this given by the sum of the support levels of each question asso- matter. Nevertheless, we can always use quality vectors as a ciated with the quality i. For example, if the quality tool to help us identify these relationships.

BPersistence^ has three questions and only two answers fully We can calculate vi as a simple arithmetic mean, consider- support the aspects stated by these questions, then the support ing that 0.0 means an Bunsupported^ aspect of the quality level of this quality is 2/3. attribute and 1.0 means a fully Bsupported^ aspect. Usually, Filling the quality vector is always an imperfect process the answer Bno^ corresponds to the value 0.0 and Byes^ to 1.0. because the following perfect conditions can never be Considering the checklists we elaborated in [15], this simple achieved: (1) a complete set of reliable information sources, calculation is not always possible for a number of reasons: (2) a complete and accurate set of questions (i.e., a perfect checklist), and (3) flawless human analysts. If conditions are & Some questions are eliminative, in the sense that if their perfect, the value vi would be an accurate measure of quality i. answers deny support to a particular attribute, then vi will Therefore, in practice, we have only a proxy of each quality be 0.0 regardless of the other answers. However, in the attribute. Nevertheless, this is not a matter of great concern, because the idea behind the quality vector V is not to establish an accurate way to measure quality but to propose an instru- Table 11 Common keywords and expressions related to Resilience ment to assist the evaluation process of quality attributes dur- ing the design of ubiquitous apps and games. Deal with Explore Orchestration Seamful uncertainty seams design

2 The companion website (http://www.ic.uff.br/~medialab/papers/2017/pauc) Seams Uncertainties Game contains a report with all checklists and additional comments. This website moderators also provides the quality spreadsheet for public use. Pers Ubiquit Comput

Table 12 Common keywords and expressions related to Actors Approach passersby Approach strangers Bystanders Sociality Chat Challenge social rules Invite spectators Non-players Online participants Performers Social expansion Social interaction Social interaction in public places Social networks Communities Mobile community Spectators Audience User-generated content

case of eliminative questions, if an answer gives support to latter case, the adequate aggregate function is min, which an attribute, then it can participate or not in the calculation; calculates the minimum of a set of values. Another pos- & Some answers are not simply Byes^ (1.0) or Bno^ (0.0), sibility is to assign degrees of importance for each infor- that is, they can be an expression, a list, or a number amid mation source (in this case, several aggregate functions a range of values; can be used, such as a weighted mean function, the max & Sometimes the answer Bno^ means a supported aspect function applied to the support level multiplied by a (1.0) and a Byes^ means an unsupported aspect (0.0); weight); & The support level should consider more than one informa- 3. Each question in the checklist of a quality attribute reveals tion source. a different, and independent, aspect of this attribute (there- fore, the max or min functions can be interpreted as a For these reasons, we expand the simple calculation men- union operation, which collects support evidence from a tioned above towards a more general concept, which we call number of different information sources); Bpervasive quality metrics^ in this paper. These metrics com- 4. All the answers are equally reliable; prise a set of equations, a set of hypotheses,andasetofrules. 5. All the questions for a specific quality attribute have the The simplest metric has the following equation: same importance (therefore, the simple mean of Eq. 1 is  an adequate formula). If this hypothesis does not apply, 1 ¼ α ∑N i i ; i ; ⋯; i Þ ¼ ; ð Þ vi i j¼1aggregate 1q j 2q j T q j i 1 n 1 then we should change Eq. 1 into a weighted mean, i.e., a N i weight wj should multiply the aggregate function with the … In Eq. 1, where qi means we are browsing the information following restriction: w1 +w2 + =Ni; k j 6. source k to assess the second-level quality i, looking for the The eliminative nature of a question is independent of the α answer to question j, this answer represents the support-level information sources (therefore, the parameter i of Eq. 1 value. There may be T information sources available (i.e., k = is unique for each quality attribute i); 7. 1...T). Currently, there are 16 second-level qualities (i.e., i = The database has complete information, that is, the B ^ 1...16). Finally, j varies according to the number of questions closed world assumption applies to questions (i.e., if an answer cannot be extract from an information source, for each second-level quality. Ni is the total number of answers then the aspect is considered unsupported). to be used in the calculation of vi; αi is 0.0 if there is at least one eliminative answer and 1.0 otherwise; n is the total num- ber of second-level qualities (n = 16 in this paper); and The ways of combining answers (extracted from distinct aggregate is the aggregate function, which combines the con- information sources) are more diverse than those we propose tributions of each information source. The simplest aggregate above. Thus, many other metrics can be defined. In any case, function is the max function, which calculates the maximum we should always reason in terms of the set of hypotheses. We of a set of values. Table 13 presents a case where the quality have already noted that different aggregate functions (e.g., number is 5 (i = 5), with 3 questions (e.g., a, b, c), and the max, min) represent different hypotheses. However, there are database has 2 information sources (T = 2). The aggregate more radical ways of defining aggregation. For example, if function in this example is max. hypothesis 3 does not apply (i.e., the questions do not

Then, vi = (0.8 + 1.0 + 0.3) / 3 = 0.7. The simplest set of hypotheses is the following: Table 13 Example of answers for two information sources (shaded cells mean Bunknown support^) 1. All information sources in the database are equally Answers reliable; 2. The most important source is the one that gives the Quality i Question j Info source 1 Info source 2 highest level of support (therefore, the most adequate ag- 5a 0.80.5 B gregate function is max). An alternative hypothesis is the b1.0 most important source is the one that is more restrictive, c0.3 i.e., the one that gives the lowest level of support.^ In this Pers Ubiquit Comput

Fig. 3 Quality assessment spreadsheet sample (incomplete). The input cells are marked in blue and the quality vector vi is in the last column correspond to different aspects), then we may consider aggre- We can apply several other metrics. Our fundamental rec- gating values at the information source level (instead of the ommendation is twofold: ensure that the hypotheses are clear question level). For example, in Table 13, we first calculate the and search for simple metrics—even when a metric sounds simple mean in each information source column and then oversimplified, do not give up the principle Bthe simpler, the apply the max function on the set of these mean values (i.e., better.^ This attitude is in line with the well-known Ockham’s v5 = max((0.8 + 1.0 + 0.0) / 3, (0.5 + 0.0 + 0.3) / 3) = razor principle [32], which can be restated here as Bamong max(0.60,0.27) = 0.60). In this case, the set of equations to competing hypotheses, the one with the fewest and simplest calculate vi is the following: assumptions is the best.^ In any case, we should always re- member that there are many other sources of error and noise

N i that we cannot control, such as the humans involved in the ¼ α 1 ∑ i ¼ ; ¼ ; ð Þ k Si i k q j i 1 n; k 1 T 2a process. In other words, there is no gain in searching for so- N i j¼1 À phisticated metrics when considering these issues. ¼ ; ⋯ Þ; ð Þ vi aggregate 1Si 2Si T Si 2b We implement the abovementioned metrics as a quality spreadsheet,3 which calculates the quality vector automatical- ly as the designer or requirements engineer answers the ques- where kSi is the total support level for quality i obtained from the k-th information source (where T is the total number of tions. Figure 3 illustrates this spreadsheet, in which the input information sources in the database) and the other elements cells are marked in blue and the quality vector is in the last are the same variables found in Eq. 1. The alternatives pro- column. This spreadsheet implements two different metrics posed for hypotheses 2 and 5 may also apply (e.g., aggregate using the max function: one uses Eq. 1 and the other uses as a weighted mean function). Eq. 2b. Some questions can be assessed more adequately B ^ As far as aggregation is concerned, the difference through the IG3 information group (i.e., evaluation group). between Eq. 1 and Eq. 2 is that the first is an aggre- We highlight these questions in the spreadsheet by marking gation at the question level and the latter is an aggre- them with an asterisk (e.g., the UHP1* symbol in the first row gation at the level of information source. In our experi- in Fig. 3). ence, Eq. 1 produces better results (especially because A helpful way to present the quality vector is through a bar we always try to guarantee hypothesis 3). However, chart, which we interpret as a kind of quality spectrum or more experimental work is necessary to give more sound guidelines about types of aggregation. 3 Available at http://www.ic.uff.br/~medialab/papers/2017/pauc Pers Ubiquit Comput

Fig. 4 Similar quality vectors for hypothetical games X and Y

quality profile (see Fig. 4 for an example). This bar chart & We can compare the vector qualities that come from two permits a quick global visualization of the quality attributes different information sources for the same app/game. If of a specific app or game. We may be tempted to use these the information sources are complete (in the sense of pro- charts (i.e., quality vectors) to compare two games or evaluate viding information that can answer all questions), the vec- how far a particular game is from an ideal quality profile. The tor qualities should be similar. Thus, if the similarity is too comparison of two different games may make some sense, as low, the analysts should consider the possibility of facing we explain below, but the concept of an ideal, prototypical, or errors or inadequate information sources; absolute quality profile is nonsensical. Moreover, the compar- & Whenever we have a pair of different games with similar- ison with an all-ones quality vector is wrong, because a quality ity values of 0.5 or greater, it is worth investigating wheth- vector with all elements equal to 1.0 cannot exist (as we ex- er one of the games can inspire new features for the other plained at the beginning of this section). one. This is a valid practice because they share similar The two quality vectors in Fig. 4 are visually quite similar quality concerns. For instance, we applied our method4 (almost the same bars and values). Here, we start exploring the to the games Pac Map [33] and Exploding Places [34] advantages of comparing two quality vectors, and later, we and found a similarity value equal to 0.43 (Figure 5). In present methods to calculate similarity values (which we con- fact, they have a similar quality profile, but, for example, sider as falling between 0 and 1). If the similarity is equal to 1, Exploding Places misses CPS (Conformance to physical we will have a situation in which the quality attributes are and social settings). This suggests that we could improve equally taken into consideration, as far as the information Exploding Places in some Sociality aspects by answering sources are concerned (i.e., we will have the same support two important questions: BDo game activities possibly levels). There are two basic situations in which we can check disturb non-players?^ (CPS2) and BDo game activities quality vectors: one is for the same game (using different expose players (or non-players) to embarrassing information sources) and the other is for different games. A situations?^ (CPS3). However, we should always consider high similarity value means that the quality vectors share sim- whether a missing quality (or a quality with a low support ilar quality concerns. However, in the case of comparing two level) is a consequence of a poor information source or a different games, this high value of similarity does not mean faulty design; that the two games are of the same type or provide a similar & If a specific app or game has a Bgood reputation^ among game playing experience. users,5 then we can use it as a reference and check how far The main advantage of comparing two quality profiles is to another game is from this reference in terms of quality assist designers and requirements engineers during the process attributes. In this case, we should always be aware of of evaluation and/or design. The assessment method we pro- how complete and reliable the information sources are. pose in this paper is supposed to be customized by the user, Sometimes, a low similarity value is a consequence of who should decide on which aggregate function is more ade- using a bad information source; quate and how to use the various sources of information. The quality vectors may work as an indicator, like a dashboard 4 These assessments are available at http://www.ic.uff.br/~medialab/papers/ element. Although the precise definition of similarity and the 2017/pauc 5 correlation coefficient are given in the next paragraphs, the Here, we assume that if players generally consider a given pervasive game as Bgood^ (i.e., satisfying, enjoyable), then this game should have a Bgood^ following examples may help the reader understand the utility pervasive experience. However, we are aware of how difficult it is to explain of the quality vectors: why players would consider a game Bgood.^ Pers Ubiquit Comput

Fig. 5 Quality vectors for PacMap and Exploding Places, with Euclidean similarity = 0.43

& Similarity values can be a quick way to explore a large Finally, in the metrics, we have a set of rules (Bquality i database of quality vectors, either to select a similar vector metric rules^) to calculate αi and kq j for each quality i.The given a specific one or to extract interesting pairs of games rules are the same for all types of information sources. for a more detailed analysis. Table 14 describes these rules and the types of questions. Table 15 presents a generic sample scale to assess other Btext&scale^ and Btext&scale,REV^ questions. Table 16 pre- We can compare the similarity of two quality vectors using sents a sample scale to use when evaluating Btext&scale^ a measure of similarity based on standard distances (e.g., questions related to technology uncertainties (e.g., see Eqs. 3 and 4) or using correlation methods (e.g., Pearson’s Uncertainty handling policy, Mobility, and Connectivity in correlation coefficient, Eq. 5). In this paper, we call the first [15]). type of measure BEuclidean similarity^ and the latter BPearson similarity.^ rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ðÞ¼; ∑ ðÞ− 2 ð Þ 3 Case studies dxy xi yi 3 i This section presents the summarized assessment (only 1 EsimðÞ¼ x; y ð4Þ Accessibility and Sociality due to space constraints)6 of two 1 þ dxðÞ; y pervasive mobile games, Spellbound [12] and Ingress [13]. ðÞ∑ −∑ðÞ ∑ − ixi iyi We used the quality spreadsheet with Eq. 1 to calculate the ixiyi ðÞ¼; vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffin ð Þ quality vectors (Fig. 6). The similarity values (Euclidean = rxy u ! !5 u 2 2 0.37 and Pearson = 0.29) are low (as expected) because these t ðÞ∑ xi ðÞ∑ y ∑ x2− i ∑ y2− i i games are quite different and the quality attributes are not i i n i i n equally taken into consideration. In this section, we use the quality symbols (see Tables 17 We use the Euclidean distance when we want to compare and 18) to indicate the si values of each question of a quality absolute values (i.e., in case where the quality vectors are j i—e.g., Access usability (Usa1 = 1, Usa2 = 1) for Access supposed to be nearly identical). This may be the case when usability in Spellbound. In this example, the answer to ques- we are comparing the quality vectors from two different sets tion Usa1 (BDo game activities require players to focus on the of information for the same game and we are expecting similar device screen all the time?^)isBno^ (which produces the outcomes. When we want to compare the shape of the Bquality value 1, because the question type is Byes/no,REV^). The spectrum^ regardless of their absolute values, we should use answer to question Usa2 (BDo game activities use various Pearson’s correlation coefficient. For example, Fig. 4 illus- modalities to interact with the player?^)isByes^ (which pro- trates two quality vectors with low similarity by Euclidean duces the value 1). In addition, for the sake of uniformity, we distance (Esim = 0.56) and a high correlation coefficient use B–^ in case of Bhint^ questions, as these questions have no (r = 0.86) for two different (hypothetical) games. However, support value, e.g., Transmediality (TM1 = 0, TM2 = 0, we can use both measures of similarity (Euclidean or TM3 = 0, TM4 = −,TM5=−). The reader should consult Pearson) for the same game (but from different information Tables 17 and 18 to follow the analysis below and to under- sources) or for different games—the choice depends on what stand the si values. we are looking for. For instance, we can use Euclidean dis- j tance to compare two different games if we suspect that their 6 For the complete analysis, please refer to http://www.ic.uff.br/~medialab/ quality attributes are equally taken into consideration. papers/2017/pauc Pers Ubiquit Comput

Table 14 Question types and their descriptions (where d/a Question type Rules means Bdoes not apply^ and n/a B ^ B ^ B ^ i B ^ means Bnot available^) yes/no The answers can be either yes or no, where yes means kq j = 1.0 and no means i kq j = 0.0 (e.g., Mob1 in Mobility—BDoes the game activities need to use networking i while the players are moving while playing?^). Answers Bd/a^ and Bn/a^ mean kq j =0.0. i yes/no,REV: The answers can be either Byes^ or Bno^,whereByes^ means kq j = 0.0 and Bno^ means i kq j = 1.0 (e.g., GA1 in Game autonomy—BIs the game bound to specific places or local i context?^). Answers Bd/a^ and Bn/a^ mean kq j =0.0. i text A textual answer, where any valid answer means kq j = 1.0. If the question does not apply, this i means that kq j = 0.0. For example, in BWhat are the connectivity requirements for the game?^ (e.g., Con1 in Connectivity), valid answers could be Blocal networking^ or Bglobal networking.^ In case there is no networking (e.g., it does not apply), the value i i would be kq j =0.0.AnswerBn/a^ means kq j =0.0. i text&scale A textual answer and a value for kq j (between 0.0 and 1.0) that represents the answer’s adequacy according to a Likert scale (e.g., Tables 15 and 16)—the Bmost adequate/^ i i answer means kq j =1.0andtheBleast adequate^ means kq j =0.0.TheBmost adequate^ answer means that the aspect occurs as the Bhighest value/strongest intensity^ possible. For example, BHow does the game handle player privacy?^ (CPS1 in Conformance to physical and social settings) may have the answer: Bthe game identifies players by i nicknames^ and kq j =0.5(Badequate, ok^ in Table 15). Another example is BHow does the game handle technology limitations?^ (e.g., UHP2 in Uncertainty handling policy), which may have the following answer: Bthe game identifies all uncertainties, but handles i i some^ and kq j =0.8(Table16). Answers Bd/a^ and Bn/a^ mean kq j = 0.0. i text&scale,REV A textual answer and a value for kq j (between 0.0 and 1.0) that represents the answer’s i adequacy as a Likert scale (e.g., Table 15)—the Bmost adequate/^ answer means kq j =1.0 i and the Bleast adequate^ means kq j = 0.0. Contrary to the previous type, here, the Bmost adequate^ answer means that the aspect occurs as the Blowest value/weakest intensity.^ For example, BDoes the game require players to walk long distances?^ (Mob4 in Mobility) mayhavetheanswer:Byes, they need to walk 3 km^ (which might be considered Bhigh^) i i and kq j = 0.25 (Table 15). Answers Bd/a^ and Bn/a^ mean kq j =0.0. …,ELI A suffix for any of the types above to represent Beliminative^ questions. This means that if i the answer to this question is kq j = 0.0 (e.g., Bno^ for a Byes/no,ELI^ question), then the corresponding pervasive quality is considered Bunsupported^ regardless of the answers to other questions that the given quality might have. For example, if the answer to the Btext,ELI^ question BWhat are the connectivity requirements for the game?^ (e.g., Con1 in i Connectivity) is Bno networking^ (or Bd/a^—does not apply), then kq j is calculated as 0.0 (because there is no network) and the quality BConnectivity^ is totally eliminated (i.e., αi is set to 0.0). hint These questions do not contribute to the quality metric calculation. Their purpose is to provide useful information to assess other questions and the overall quality. For example, the TM3 question of Transmediality (BDoes the the game experience for the various game roles?^) helps to identify a case where transmediality might not be supported (e.g., in case its answer is Byes^). On the other hand, answering Bno^ to TM3 does not bring insight to assess the overall quality.

3.1 Spellbound paper [12] (an IS1 information source) and used it (sec- ond step) to determine if this game is a pervasive mo- Spellbound is an outdoor pervasive mobile game that takes bile game (third step). Browsing the paper title and place in a preselected game area, where two teams compete in abstract reveals keywords and expressions related to three quests (game activities). In these quests, players are able pervasive games, such as Bmobile games beyond the to interact through proximity with virtual elements, which device screen,^ Bimmersive game experiences integrated have corresponding real locations in the game area. The game with the real world,^ Boutdoor pervasive team-based is designed to foster face-to-face communication by applying physical game,^ Breal-world actions like jumping and the Bheads up display^ paradigm and not using digital com- spinning,^ Bvoice interaction,^ Bglanceable and haptic munication channels. Spellbound uses whole-body interac- feedback,^ and Banchors enjoyment on physical action, tions (e.g., jumping, spinning, running) as the main interaction social interaction, and tangible feedback.^ This informa- mechanic with virtual elements. tion seems enough to classify Spellbound as a pervasive We begin the assessment process by searching for suitable mobile game, but we proceed to the next step for con- information sources about Spellbound. We found a journal firmation—the highlighted expressions refer to mobile Pers Ubiquit Comput

Table 15 Sample scale for generic Btext&scale^ and Assessment Grading Grading Btext&scale,REV^ questions (as (text&scale) (text&scale,REV) mentioned in Table 14) Very high, very strong, very good, all, very severe, most 1.0 0.0 adequate High, strong, good, quite a bit, severe 0.75 0.25 Average, fair, ok, moderate, adequate 0.5 0.5 Low, weak, poor, mild 0.25 0.75 None, least adequate 0.0 1.0 Does not apply, not available 0.0 0.0 devices and integration of virtual and real worlds. In the custom hardware, which means there is no device indepen- fourthstep,webrowsethepaper[12]tofindspecific dence (DI1). information groups: Regarding Game autonomy (GA1 = 0, GA2 = 1, GA3 = 1, GA4 = 0), this game is bound to a specific place—a & IG1 (Bgame design^): Section 1, 3, and 4; preselected playing field (GA1). However, this game does & IG2 (Bimplementation^): Section 3; not require preparing the physical space for game sessions & IG3 (Bevaluation^): Sections 5 and 6. (GA2). Instead, artists and designers map virtual content to physical locations using software tools. The game detects the Sections 3 and 5.1 (in [12]) describe the devices that virtual content locations using GPS. The game does not re- players use in this game: an Android smartphone, wearable quire orchestration while game sessions are happening (GA3). devices (wristband and two armbands)—two kinds of mobile Finally, the game requires custom hardware—the armband context-aware devices (i.e., the C2 condition)—and and wristband were designed solely for Spellbound. earphones. Section 1 also describes context information that Considering Transmediality (TM1 = 0, TM2 = 0, TM3 = 0, this game uses—location (i.e., the C1 condition). We conclude TM4 = −,TM5=−), in this game, players compete in differ- that Spellbound is a pervasive mobile game according to C1 ent teams (roles) using the same game interface (TM1, TM2). and C2 conditions in Table 2. BecauseTM1andTM2areeliminativequestions, Transmediality is unsupported and question TM3 does not apply. This game interface is composed of a smartphone, a 3.1.1 Accessibility wristband, and two armbands (one for each arm). The game uses the smartphone to collect sensor information to detect Spellbound is an interesting case of Access usability (Usa1 = and process whole-body interactions. The players also use 1, Usa2 = 1), as this game was designed to avoid having the smartphone to send voice input to the game and to receive players look at the smartphone screen constantly by applying audio feedback. The wristband contains LEDs and vibration Bheads up play^ (Usa1). Moreover, the player is able to use motors that the game uses to convey game state information to several modalities (e.g., voice input, haptics feedback, and players. The game uses the armbands to assist players when embodied interactions such as jumps, spins, and runs) to in- they search for virtual content in the physical place, by teract with the game (Usa2). Therefore, Access usability instructing them to turn right or left. Concerning question reaches 1.0 in the graph in Fig. 6 (i.e., Eq. 1 Vusa =1.0). TM4, there are no support modules for players (TM4). Concerning Device independence (DI1 = 0), the game in- Finally, Spellbound is not part of a broader transmedia expe- terface in Spellbound is built using Android devices and rience (TM5).

Table 16 Classification of Uncertainty handling policy considering uncertainty identification and applied policies (as mentioned in Table 14) 3.1.2 Sociality

Conditions Grading Considering Social communication (SC1 = 1, SC2 = 0, Identifies all uncertainties, handles all 1.0 SC3 = 1, SC4 = 0, SC5 = 0), Spellbound provides an interest- Identifies all uncertainties, handles some 0.8 ing case of a game design that deliberately avoids using digital Identifies some uncertainties, handles some 0.5 communication channels as a means to improve and foster Identifies none, handles some 0.4 face-to-face communication in co-located situations (SC1). Identifies all uncertainties, handles none 0.25 In this regard, Spellbound uses embodied interactions to en- Identifies none, handles none 0 courage heads up play and uses team play in a small area to accomplish this goal. By using heads up play, players are able Pers Ubiquit Comput

Fig. 6 Quality vectors for Spellbound and Ingress (using Eq. 1). Similarity values are Euclidean = 0.36 and Pearson = 0.27 to pay attention to the surrounding environment and engage in Lastly, it seems that the game does not present emergent conversations with teammates directly. gameplay (SC5). Concerning SC2, Spellbound does not transform the rela- Regarding Conformance to physical and social settings tionships among players. However, according to the game (CPS1 = 0, CPS2 = 0, CPS3 = 0.75, CPS4 = 0, CPS5 = evaluation [12], it seems that team play in Spellbound may 0.75), the information source [12] does not provide enough have provided socialization opportunities for players, as the information about privacy issues in Spellbound (CPS1). Also, following excerpt describes: BSome players said doing the because this game takes place in a preselected playground actions alone outdoors would make them feel self-conscious area, it seems that there are no issues regarding local social but doing them with their team made them feel like they were conventions and etiquette (CPS4). However, the Recruit game Bpart of a secret club that others had no idea about^ and that task requires players to approach passersby, which might dis- made playing more fun^ [12]. This is a clear way to improve turb non-players (CPS2). Also, the embodied interactions re- this pervasive quality in Spellbound. Concerning SC3, this quired to play this game (e.g., jumps, spins, and runs) might be game requires players to approach passersby and interact with an issue for non-players (CPS2), although Sra and Schmandt them in the Recruit task. Regarding SC4, the game does not [12] did not report that. We have here a clear identification of use technology to create communities around the game. quality issues that can be improved.

Table 17 Checklists of second- level qualities for Sociality.The Social Communication SC1* (text&scale) How does the game use technology as means to improve asterisk denotes questions that communication among people? can be that can be answered more SC2* (text&scale) How does the game transform the relationships among adequately through evaluation players? activities (e.g., hands-on evalua- SC3 (yes/no) Does the game stimulate players to approach/start interactions tion, user tests, and experiments) with other people? SC4 (yes/no) Does the game use technology to foster communities or social networks? SC5* (yes/no) Does the game foster emergent gameplay? (yes/no) Conformance to physical and CPS1 (text&scale) How does the game handle player privacy? social settings CPS2* (text&scale,REV) Do game activities possibly disturb non-players? CPS3* (text&scale,REV) Do game activities expose players (or non-players) to embarrassing situations? CPS4* (text&scale) Do game activities conform to local social conventions and etiquette? CPS5 (text&scale) Are game activities adequate to the physical setting of the game? Involving non-players INP1 (text.ELI) Does the game involve non-players? How does it do it? INP2 (yes/no) Does the game have activities where players need to find out who the other players are? INP3 (yes/no) Does the game generate/use content that is based on other (non-player) people? INP4 (yes/no) Does the game use human actors for non-player characters? Pers Ubiquit Comput

Table 18 Checklists of second- level qualities for Accessibility. Access usability Usa1* (yes/no,REV) Do game activities require players to focus on the device screen all the The asterisk denotes questions time? that can be that can be answered Usa2 (yes/no) Do game activities use various modalities to interact with the player? more adequately through evalua- Device DI1 (yes/no) Is it possible to use the game interface across multiple mobile device tion activities (e.g., hands-on independence platforms? evaluation, user tests, and experiments) Game autonomy GA1 (yes/no,REV) Is the game bound to specific places or local context? GA2 (yes/no,REV) Does the game require configuring the physical space for a game session? GA3 (yes/no,REV) Does the game require any kind of supervision (orchestration) when players are playing it? GA4 (yes/no,REV) Does the game require custom hardware, human actors, or other kind of related resources? Transmediality TM1 (yes/no,ELI) Does the game offer different participation modes (game roles)? TM2 (yes/no,ELI) If the game offers different modes of participation, are these roles played through different media (devices)? TM3* (text&scale) Does the game balance the game experience for the various game roles? TM4 (hint) Does the game need to use other media (e.g., desktop PCs) as support modules? TM5 (hint) Is this game part of a broader transmedia experience?

Concerning CPS3, Sra and Schmandt [12] reported that real landmarks and objects such as monuments, historical some players felt uncomfortable performing the embodied sites, statues, stores, churches, train stations, and public art. interactions when alone but not when they were in groups. An important player goal is to capture these portals and claim Some players also felt uncomfortable when using certain re- them for their team—an action that requires players to go the quired keywords in voice input (such as Bhootie^)inthe physical location that corresponds to the portal. Players are BFind^ game task, as passersby would be able to hear it. able to link portals located in different areas to create Bcontrol Voice input is an important interaction mechanism in fields.^ The control fields are important, as they are able to Spellbound. This game takes place in outdoor places, where generate a significant amount of XM (Bexotic matter^), which ambient noise is likely to be high. The game developers in- is a kind of energy particle players need to collect while they cluded some measures to adapt this interaction mechanism to wander around. Some player actions, such as interacting with the physical setting where this game takes place (CPS5). The portals (i.e., Bhacking portals^ in the game’s idiom), require following excerpt illustrates this issue: BIn outdoor spaces variable amounts of XM that depend on the portal status. with high ambient noise, there is always a possibility for the These portal links may span varying distances, from a hundred player to not hear the spoken dialog or remember what was meters to several kilometers (i.e., their length depends on the spoken so the system needs to be designed such that repetition distance between two portals). is possible on demand^ [12]. Players interact with the game through the BIngress Finally, when evaluating Involving non-players (INP1 = 1, scanner^ (i.e., the smartphone). Besides interacting with por- INP2 = 0, INP3 = 0, INP4 = 0), the answer to question INP1 tals, this scanner enables players to perform other actions, is Byes,^ because players need to approach passersby in the such as chatting with other members of their team, managing Recruit task. However, Spellbound does not have activities game item inventory, and receiving news about live game where players need to find out who the other players are events (e.g., a player of the opposite team is attacking a portal (INP2). Also, this game does not generate content based on held by the player) that happen locally, regionally (e.g., city- non-players (INP3), and Spellbound does not use actors nor and state-wide), or worldwide. performers (INP4). We begin the assessment process by searching for suitable information sources about Ingress. There are several informa- 3.2 Ingress tion sources about this game (e.g., official websites, videos, interviews, academic and non-academic papers), including the Ingress is a multiplayer mobile location game where players game application itself, which is available for download in the (the Bagents^) take part in two opposing teams (BEnlightened^ Google and Apple application stores. Browsing the official and BResistance^). The game creates a mixed-reality world website (https://www.ingress.com/) is enough to understand based on maps of real locations around the players. This game that Ingress is a pervasive mobile game, as it runs on populates this mixed-reality environment with Bportals,^ context-aware smartphones (the C2 condition) and it relies which are virtual objects created according to the location of on context (i.e., location) information (the C1 condition). Pers Ubiquit Comput

We chose to use the game itself in a custom evaluation portals, banning misbehaving users). Finally, the game re- process to assess it (IS8 information source). In this process, quires only off-the-shelf smartphones to play (GA4). a researcher played the game on an Android device (Motorola Considering Transmediality (TM1 = 1, TM2 = 0, TM3 = 0, Moto X, 2nd Gen) for six days (one hour each day) in Rio de TM14 = −,TM5=−), there are two roles in Ingress Janeiro, Brazil. In the first three days, the process consisted of (BEnlightened^ and BResistance^)—therefore, the answer to free-play—the researcher played the game as a casual player. question TM1 is Byes.^ However, these two roles use the same In the next three days, the researcher played the game while game interface (question TM2 is Bno^), which means that observing aspects about the pervasive qualities and taking Transmediality is totally unsupported (because TM2 is an notes about them—this means playing the game while focus- eliminative question). Yet, question TM3 does not apply. ing on each question pertaining to the given quality at a time. Furthermore, Ingress does not use other media as support On the last day, the researcher wrapped up the assessment modules (TM4), and it is not part of a broader transmedia process with a final review of the written material generated experience (TM5). on the previous days. Compared to the general process we described in Fig. 2, we summarize this hands-on process as 3.2.2 Sociality follows: Step 1: Searching for available information sources—the Regarding Social communication (SC1 = 1, SC2 = 0, SC3 = results include the game website, videos, interviews, reports, 0, SC4 = 1, SC5 = 1), Ingress is a game that encourages team academic papers, and the game itself; play (SC1). The game provides an internal chat interface Step 2: We selected the game website for initial impres- where players of the same team may start communicating sions about the game—we learned that Ingress is a pervasive among themselves. The game developers also use existing mobile game (Step 3). We discarded this information source social networks (e.g., Facebook, Google+) to create commu- and selected the game application itself; nities around the game, which means that players on the same Step 4 and 5: Casual game playing, where the goal was to team are able to discuss game tactics inside and outside the get acquainted with the game (i.e., the information source) and game. There are live events announced in advance at the offi- start observing aspects related to the first-level qualities. cial game website (https://www.ingress.com/events), where Step 6: Focused game playing, considering each second- the goal is to gather people to play the game together. These level quality at a time and taking notes about the relevant events include a meet-up after the playing is over (SC1, SC4). aspects (i.e., using the checklists as a guide). At the end of For SC2, we were unable to determine whether Ingress trans- this step, we filled out the quality assessment spreadsheet, forms the relationship among players, but we believe that the reviewed the produced material, and wrapped up the process. game may help in this issue due to its aforementioned aspects. As far as SC3 is concerned, players are not required to ap- proach strangers to start interaction. Lastly, Ingress presents 3.2.1 Accessibility emergent gameplay (SC5)—for example, players are able to create several missions that are available to other players (the Considering Access usability (Usa1 = 0, Usa2 = 0), Ingress Bmission game mode^). requires players to focus extensively on the device screen Concerning Conformance to physical and social settings (Usa1, Usa2). The game uses audio for output but it does (CPS1 = 0.5, CPS2 = 0.75, CPS3 = 0.75, CPS4 = 0.75, not play a relevant role in player interaction. This assessment CPS5 = 0.75), we evaluated question CPS1 (about player pri- analysis encourages the designer to look after game activities vacy) as 0.5 because although Ingress identifies players that require players to look less at the device screen and more through nicknames, we do not know how this game handles at the real world. other player information (e.g., location). According to our Ingress game has device independence (i.e., Device inde- experience, game activities in Ingress were not designed to pendence (DI1 = 1)) because it is possible to play Ingress in disturb non-players nor embarrass players (CPS2, CPS3), Android and iOS devices. and they seem to be fine with local conventions and etiquette Regarding Game autonomy (GA1 = 0, GA2 = 1, GA3 = 1, (CPS4). Also, game activities in Ingress seem to be adequate GA4 = 1), Ingress is not bound to a specific game area but to the physical setting where the game takes place (CPS5), instead depends on location-specific content—Bportals^ are which are mostly urban environments. Nevertheless, players real landmarks (GA1). Game administrators create portals might be discouraged from going after portals that are located using software tools and players detect these portals through in dangerous neighborhoods. However, conformance to phys- GPS coordinates—therefore, it is not necessary to set up the ical and social settings (CPS1 to CPS5) is a complex issue, physical space to play (GA2). Ingress does not require orches- because it depends on the player’s actual behavior while tration or supervision in live game play (GA3), but game playing, which may be unpredictable. In fact, the media has administrators carry out some tasks periodically (e.g., creating been reporting several safety and privacy problems with Pers Ubiquit Comput

Table 19 BTechnological requirements^ from Broll et al. [35] (first row) mapped to our taxonomy

Localization Communication Crossmedia Devices Authoring and Orchestration and surveillance augmentation game engine

Context-awareness Connectivity Transmediality Transmediality – Game autonomy (Accessibility) (Communicability) (Accessibility) (Accessibility) Mobility Conformance to physical and social (Spatiality) settings (Sociality)

Ingress—such as players being caught by the police, being Bemotional requirement^ [2]. On the other hand, we focused robbed in remote areas, and fatalities due to distraction.7 on what makes a game pervasive—i.e., Bpervasiveness.^ Therefore, we do not consider Ingress as fully supporting Guo et al. [38] proposed a conceptual framework for CPS. This quality assessment may encourage the game de- pervasiveness in games containing four pervasive quali- signer to think about important improvements. For instance, ties (referred as Bperspectives^): Btemporality,^ the designer may incorporate the need to pay attention to dan- Bmobility,^ Bperceptibility,^ and Bsociality.^ These per- gerous places (e.g., road crossings, cliffs, remote areas) as part spectives have some properties they identified as impor- of the gameplay, with rewards to the player. tant.Guoetal.[38] also proposed a simple metric to Finally, Ingress does not support the quality Involving non- evaluate these perspectives and presented a scoring of players (INP1 = 0, INP2 = 0, INP3 = 0, INP4 = 0). several pervasive games according to their metric. The work by Guo et al. [38] is geared towards design stud- ies independent of technical implementation, whereas in 4 Related work our taxonomy, we are motivated by technological (computing) aspects. As far as we know, there are no works in the literature that Nevelsteen [39] analyzed several pervasive games and B ^ propose methods to assess pervasive qualities in games. Also, summarized his findings as a component feature set, which there are few works that discuss or present taxonomies of corresponds to nine aspects he found important to create per- B ^ pervasive qualities. We describe these works in this section. vasive games: virtual game world with world persistence, B ^ B However, although we share some common concerns with the shared data space(s) with data persistence, heterogeneous ^ B ^ B authors of those works, we are pursuing a different research devices and systems, context-awareness, roles, groups, ^ B ^ direction on a deeper assessment level. hierarchies, permissions, current and historical game state, B ^ B Broll et al. [35] discussed technological challenges to im- game master intervention, reconfiguration, authoring and ^ B plement pervasive games and presented some technological scripting in run-time, and bidirectional diegetic and non- ^ requirements, such as Blocalization,^ Bcommunication,^ diegetic communication. Nevelsteen [39] created his taxon- — Bcrossmedia augmentation,^ Bdevices,^ Bauthoring and game omy in a bottom-up fashion i.e., he was concerned with engine,^ and Borchestration and surveillance.^ The main con- game engines, middleware, and game architectures. By ana- cern in Broll et al. [35] was to discuss the actual technology lyzing these elements, he identified his component feature set. components to implement these requirements and middleware On the other hand, we are concerned with how a pervasive support. We were also motivated by technological quality would impact the game implementation (i.e., a top- (computing) issues to create our taxonomy but we did not down concern) and the use of technology in games. Our ap- focus on how to implement the pervasive qualities. proaches seem to complement each other, as we discuss Jegers [36] proposed a model of player enjoyment in per- vasive games that extends an existing playing enjoyment Table 20 BPervasive characteristics^ from Jegers [36](firstrow) model for traditional games (the Bgame flow model^ [37]). mapped to our taxonomy In this model, Jegers [36] identified distinguishing Bpervasive ^ B Mobile/place- Social interaction Integration of the features (i.e., qualities), such as mobile/place-independent independent gameplay between players physical and virtual gameplay,^ Bsocial interaction between players,^ and worlds Bintegration of the physical and virtual worlds.^ Jegers fo- cused his work on Bplayer enjoyment,^ which is a kind of Mobility (Spatiality) Social Game content communication adaptability (Sociality) (Context-awareness) 7 See https://www.oodaloop.com/osint/homelandsecurity/2014/04/03/google- ingress-game-players-subject-of-law-enforcement-fusion-center-alert/ Game content Game autonomy concerning homeland security, and http://www.irishtimes.com/news/ireland/ adaptability (Accessibility) irish-news/online-gamer-died-on-poolbeg-pier-capture-mission-inquest- (Context-awareness) hears-1.2645636 reporting a fatal case. Pers Ubiquit Comput

Table 21 TeMPS’s perspectives from Guo et al. [38] (first row) mapped In this paper, we proposed a new method to assess perva- to our taxonomy sive qualities in pervasive mobile games, in line with the con- Temporality Mobility Perceptibility Sociality text we mentioned above. This method generates a quality report, which consists of a quality spreadsheet (containing Daily life interleaving Mobility Context-awareness Sociality metric values and comments) and a quality vector (Permanence) (Spatiality) (representing the quality profile of the game in the form of a bar chart). Also, we can compare quality vectors by calculat- design and requirements issues and Nevelsteen’s work dis- ing similarity values for two basic situations: one is for the cusses middleware issues. same game but from different information sources, and the We are not working with user experience (UX), as it is other is for different games. ’ B ^ usual in HCI (human-computer interaction). In the light of Although our paper s title emphasizes assessment, UX, quantitative and qualitative studies with a pervasive game the knowledge to assess a pervasive game is also avail- can be found elsewhere [40]. able to designers and developers interested in improving Tables 19, 20, 21,and22 describe mappings of our taxonomy existing games or creating new games. An important to the works we described in this section [35, 36, 38, 39]. These aspect of improving a game is to understand where mappings arrange concepts that are somehow related. However, the game lies concerning known and desired qualities these mappings do not mean that the other taxonomies cover all (as the ones this paper provides). The main benefit of aspects that we described for the qualities in [15]. our process will come from the encoded knowledge re- use by software developers, game developers, and game designers. The encoded knowledge we provide as taxonomies of per- 5 Conclusions vasive games qualities aims to improve the quality of the final product. This encoded knowledge contains different quality A pervasive game is very different from traditional mobile categories—some qualities relate to software development games because it brings the game experience out of a device and other relate to end users or players. For example, and into the real world. Although this type of game has been Sections 2.4.4 and 2.4.7 describe qualities related to user ex- around for some time (at least since 2001), its characterization perience (e.g., Access usability, Conformance to physical and and design methods are scarce in the literature. Pervasive social settings). The knowledge we provide in some cases is games are no longer restricted to laboratories of game studies; Bmeta knowledge,^ which is knowledge that will help practi- they now require practical guidelines for better designs. For tioners to get to the desired knowledge. This is exactly the example, Ingress (and Pokémon Go, a more recent case) be- point of providing questions (checklists), as in the came a worldwide phenomenon, with thousands of people Goal/Question/Metric (GQM) paradigm [41]. walking in the streets with smartphones to interact with the As far as we know, no other work in the literature has physical environment. proposed a method to assess pervasive qualities in games.

Table 22 Part of our taxonomy (first row) mapped to the Persistency Game autonomy Transmediality and Context-awareness Social Bcomponent feature set^ in (Permanence) (Accessibility) device independence communication Nevelsteen [39] (Accessibility) (Sociality)

Virtual game Roles, groups, Heterogeneous Context-awareness Bidirectional world with hierarchies, devices and diegetic and world permissions systems non-diegetic persistence communication Shared data Game master Roles, groups, space(s) with intervention hierarchies, data permissions persistence Current and Reconfiguration, historical authoring and game state scripting in run-time Bidirectional diegetic and non-diegetic communication Pers Ubiquit Comput

Indeed, we did not find a similar work that delves further into from the perspective of user experience in HCI (human-com- the analysis of pervasiveness in games, provides checklists for puter interaction). However, from the perspective of games, a detailed assessment task, offers a systematic way to explore we would have the additional issue that usability and playabil- several sources of information, and proposes a set of custom- ity have distinct characteristics [44]. ized metrics. There are very few works dedicated to explore A future work of utmost importance is a more thorough pervasive qualities in games, as we described in Section 4. study on non-functional requirements related to safety [45], Although in this paper we focused on pervasive mobile privacy, and health aspects of players and passersby [46]. The games, we believe that the lessons learned here may contribute numerous reports of unexpected drawbacks about these issues to a better practice of pervasive quality assessment in the de- in recent pervasive games (e.g., Pokémon Go) reinforce the sign of ubiquitous systems in general. need for this study. Lastly, we envisage the evolution of our The application of our method to some well-known games method towards other types of ubiquitous applications, in may shed more light on their pervasive characteristics and which abstract qualities are at a premium, such as being fun, identify ways to improve their quality. For example, the as- immersive, empathetic, or affective. sessment of Conformance to physical and social settings (CPS) reveals important design issues for improving current Acknowledgements This research was supported by the following and future design of location-based games (see the summary Brazilian government agencies: CAPES (Federal Agency for Support and Evaluation of Graduate Education, linked to the Ministry of of the case studies in Section 3). Education); CNPq (National Council for Scientific and Technological Our method relies on several types of information sources Development) and FINEP (Brazilian Innovation Agency), which belong we identified in Section 0: Bpapers,^ Bofficial site,^ Bvideos,^ to the MTCIC (Ministry of Science, Technology, Innovation and Bnews,^ Bdev articles,^ Binterviews,^ Bpresentations,^ Communication); and FAPERJ (Research Support Foundation for the B ^ B ^ State of Rio de Janeiro). The authors are also thankful to NVIDIA hands-on, and marketing. This is not a complete set of Corporation for the financial support. possible information sources and we understand that in prac- tice some of these sources may provide better information (i.e., more accurate, more complete, and/or more reliable) than References others. The literature on knowledge elicitation usually as- sumes information sources as trustful. However, it is possible 1. Valente L, Feijó B, Ribeiro A, Clua E (2016) The concept of per- that an information source may not be reliable, and as such, it vasive and its application in digital entertainment systems. may provide inaccurate or wrong information. A solution to In: Wallner G, Kriglstein S, Hlavacs H, Malaka R, Lugmayr A, and avoid this problem is to rely on different information sources, Yang H-S (eds) Entertainment computing - ICEC 2016. Springer International Publishing, pp 187–198 which will provide different viewpoints and perspectives [42] 2. Callele D, Neufeld E, Schneider K (2005) Requirements engineer- on the context at hand. This solution may raise information ing and the creative process in the industry. In: 13th conflicts, and in these cases, conflict resolution [43] will be up IEEE International Conference on Requirements Engineering, – to the engineer who is building the encoded knowledge. 2005. Proceedings, pp 240 250 3. Alves C, Ramalho G, Damasceno A (2007) Challenges in require- We acknowledge the fact that the proposed method has a ments engineering for mobile games development: the meantime high cost in manpower and time. However, we claim that this case study. In: Requirements Engineering Conference, 2007. RE cost will be a wise investment to improve quality and reduce ‘07. 15th IEEE International, pp 275–280 rework. Moreover, we believe that future works may contrib- 4. Furtado AWB, Santos ALM, Ramalho GL (2010) Streamlining ute to reduce the cost by providing automated support for the domain analysis for digital games product lines. In: Bosch J, Lee J (eds) Software product lines: going beyond. Springer, Berlin proposed assessment process. Heidelberg, pp 316–330. https://doi.org/10.1007/978-3-642- There are still many possible future research works. For 15579-6_22 instance, a more complete validation of our method should 5. Bentley T, Johnston L, von Baggo K (2002) Putting some emotion be conducted by a large number of game designers. We are into requirements engineering. In: Proceedings of the 7th Australian Workshop on Requirements Engineering granting public access to our spreadsheet template and internal 6. Callele D, Neufeld E, Schneider K (2010) An introduction to expe- 8 reports to encourage this validation work. Moreover, de- rience requirements. In: Requirements engineering conference signers of current pervasive games are invited to apply the (RE), 2010 18th IEEE international, pp 395–396 method and identify new ways to improve the quality of their 7. Björk S, Falk J, Hansson R, Ljungstrand P (2001) Pirates! Using the physical world as a game board. In: Proceedings of Interact 2001. products. Also, we expect that discovering new qualities, IOS Press, pp 9–13 checklist questions, and improving the current ones might 8. Struppek M, Willis KS (2007) Botfighters. In: Borries F, Walz SP, soon be undertaken. and Böttger M (eds) Space time play. Birkhäuser Basel, pp 226–227 Another future work is investigating how aesthetic aspects 9. Rashid O, Bamford W, Coulton P, Edwards R, Scheible J (2006) PAC-LAN: mixed-reality gaming with RFID-enabled mobile (i.e., how the artifact is perceived) might impact pervasiveness phones. Comput Entertain CIE 4:4 10. Ballagas R, Kratz SG, Borchers J, Yu E, Walz SP, Fuhr CO, 8 Please refer to http://www.ic.uff.br/~medialab/papers/2017/pauc Hovestadt L, Tann M (2007) REXplorer: a mobile, pervasive Pers Ubiquit Comput

spell-casting game for tourists. CHI 07 Ext. Abstr. Hum. Factors now? ACM Trans Comput Hum Interact TOCHI 13(1):100–133. Comput Syst 1929–1934 https://doi.org/10.1145/1143518.1143522 11. S.T. Start: Zombies, Run!, https://zombiesrungame.com/ 31. Montola M, Stenros J, Wærn A (2009) Pervasive games: theory and 12. Sra M, Schmandt C (2015) Expanding social mobile games beyond design. Morgan Kaufmann thedevicescreen.PersUbiquitous Comput 19(3-4):495–508. 32. Good IJ (1977) Explicativity: a mathematical theory of explanation https://doi.org/10.1007/s00779-015-0845-0 with statistical applications. Proc R Soc Lond Ser Math Phys Sci 13. Inc: Ingress, https://www.ingress.com/ 354(1678):303–330. https://doi.org/10.1098/rspa.1977.0069 14. Niantic Inc: Pokémon Go, http://www.pokemongo.com/ 33. Chatzidimitris T, Gavalas D, Kasapakis V (2015) PacMap: 15. Valente L, Feijó B, Leite JCS (2015) do P.:mapping quality require- Transferring PacMan to the physical realm. In: Giaffreda R, ments for pervasive mobile games. Requir Eng 1–29 Vieriu R-L, Pasher E, Bendersky G, Jara AJ, Rodrigues JJPC, 16. Ryan MJ (2014) The role of stakeholders in requirements elicita- Dekel E, and Mandler B (eds) Internet of things. User-centric IoT. tion. INCOSE Int Symp 24(1):16–26. https://doi.org/10.1002/j. Springer International Publishing, pp 139–144 2334-5837.2014.tb03131.x 34. Flintham M, Greenhalgh C, Lodge T, Chamberlain A, Paxton M, 17. Freeman P (1987) Software perspectives: the system is the message. Jacobs R, Watkins M, Shackford R (2011) A case study of explod- Addison-Wesley Longman Publishing Co., Inc., Boston ing places, a mobile location-based game. In: Proceedings of the 8th 18. Boehm BW, Brown JR, Lipow M (1976) Quantitative evaluation of International Conference on Advances in Computer Entertainment software quality. In: Proceedings of the 2Nd International Technology. ACM, New York, NY, USA, pp 30:1–30:8 Conference on Software Engineering. IEEE Computer Society 35. Broll W, Ohlenburg J, Lindt I, Herbst I, Braun A-K (2006) Meeting – Press, Los Alamitos, CA, USA, pp 592 605 technology challenges of pervasive augmented reality games. In: 19. Boehm B, In H (1996) Identifying quality-requirement conflicts. Proceedings of 5th ACM SIGCOMM Workshop on Network and – IEEE Softw 13(2):25 35. https://doi.org/10.1109/52.506460 System Support for Games. ACM, New York, NY, USA 20. Fenton NE, Neil M (1999) A critique of software defect prediction 36. Jegers K (2007) Pervasive game flow: understanding player enjoy- – models. IEEE Trans Softw Eng 25(5):675 689. https://doi.org/10. ment in pervasive gaming. Comput Entertain CIE 5 1109/32.815326 37. Sweetser P, Wyeth P (2005) GameFlow: a model for evaluating 21. Fenton NE, Neil M (2000) Software metrics: roadmap. In: player enjoyment in games. Comput Entertain 3(3):3–3. https:// Proceedings of the Conference on The Future of Software doi.org/10.1145/1077246.1077253 Engineering. ACM, New York, pp 357–370 38. Guo H, Trætteberg H, Wang AI, Zhu M (2010) TeMPS: a concep- 22. Jiang Y, Cuki B, Menzies T, Bartlow N (2008) Comparing design tual framework for pervasive and social games. In: Digital Game and code metrics for software quality prediction. In: Proceedings of and Intelligent Toy Enhanced Learning, IEEE International the 4th International Workshop on Predictor Models in Software Workshop on. IEEE Computer Society, Los Alamitos, CA, USA, Engineering. ACM, New York, pp 11–18 pp 31–37 23. Chung L, Nixon BA, Yu E, Mylopoulos J (1999) Non-functional 39. Nevelsteen KJL (2015) A survey of characteristic engine features requirements in software engineering. Spring for technology-sustained pervasive games. Springer International 24. Leite JCSP, de Moraes EA, de Castro CEPS (2007) A strategy for Publishing, Cham. https://doi.org/10.1007/978-3-319-17632-1 information source identification. In: Anais do WER07 - Workshop em Engenharia de Requisitos, Toronto, pp 25–34 40. Blythe M, Reid J, Wright P, Geelhoed E (2006) Interdisciplinary 25. Valente L, Feijó B (2013) A survey on pervasive mobile games. criticism: analysing the experience of riot! a location-sensitive dig- – Departamento de Informática, PUC-Rio, Rio de Janeiro ital narrative. Behav Inf Technol 25:127 139 26. Lindt I, Ohlenburg J, Pankoke-Babatz U, Ghellal S (2007) A report 41. Basili V (1992) Software modeling and measurement: the on the crossmedia game epidemic menace. Comput Entertain CIE goal/question/metric paradigm. University of Maryland 5(1):8. https://doi.org/10.1145/1236224.1236237 42. Leite JCSP, Freeman PA (1991) Requirements validation through – 27. Barkhuus L, Chalmers M, Tennent P, Hall M, Bell M, Sherwood S, viewpoint resolution. IEEE Trans Softw Eng 17(12):1253 1269. Brown B (2005) Picking pockets on the lawn: the development of https://doi.org/10.1109/32.106986 tactics and strategies in a mobile game. In: Beigl M, Intille S, 43. Easterbrrok S, Nuseibeh B (1996) Using ViewPoints for inconsis- Rekimoto J, Tokuda H (eds) UbiComp 2005: ubiquitous comput- tency management. Softw Eng J 11(1):31–43. https://doi.org/10. ing. Springer, Berlin, p 903 1049/sej.1996.0004 28. Benford S, Anastasi R, Flintham M, Drozd A, Crabtree A, 44. Sánchez JLG, Vela FLG, Simarro FM, Padilla-Zea N (2012) Greenhalgh C, Tandavanitj N, Adams M, Row-Farr J (2003) Playability: analysing user experience in video games. Behav Inf Coping with uncertainty in a location-based game Technol 31:1033–1054 29. Bell M, Chalmers M, Barkhuus L, Hall M, Sherwood S, Tennent P, 45. Leveson NG (2012) Engineering a safer world: systems thinking Brown B, Rowland D, Benford S, Capra M, Hampshire A (2006) applied to safety. The MIT Press, Cambridge Interweaving mobile games with everyday life. Proc SIGCHI Conf 46. Chappell D, Eatough V, Davies MNO, Griffiths M (2006) Hum Factors Comput Syst:417–426 EverQuest—it’s just a computer game right? An Interpretative 30. Benford S, Crabtree A, Flintham M, Drozd A, Anastasi R, Paxton Phenomenological Analysis Online Gaming Addiction. Int J M, Tandavanitj N, Adams M, Row-Farr J (2006) Can you see me Ment Health Addict 4:205–216