Sjalvst¨ andigt¨ arbete i informationsteknologi 13 juni 2021

An Interactive Online Application for Active Learning using Escape Rooms

Andreas Bleichner Christoffer Nyberg Nils Hermansson Sebastian Fallman¨

Civilingenjorsprogrammet¨ i informationsteknologi Master Programme in Computer and Information Engineering Abstract

Institutionen for¨ An Interactive Online Application for Active informationsteknologi Learning using Escape Rooms

Besoksadress:¨ ITC, Polacksbacken Lagerhyddsv¨ agen¨ 2 Andreas Bleichner Christoffer Nyberg Postadress: Box 337 Nils Hermansson 751 05 Uppsala Sebastian Fallman¨

Hemsida: https://www.it.uu.se As the pandemic of 2020 broke out there was an increase in the use of virtual meeting rooms as a platform for holding online lectures. The use- fulness of being able to attend a lecture comfortably from home cannot be denied, however participants report feeling less engaged in lectures when studying remotely. By involving and engaging the participants in online lectures with the help of virtual Escape Rooms the partici- pants can feel more engaged and perform better in their schoolwork. The result is a prototype that showcases how virtual Escape Rooms can be used for educational purposes. The prototype consists of a room that connects to a group of Escape Rooms. The participants can interact with each other and objects as well as communicate through text and voice chat. Further work is necessary for the application to be an effective tool for education.

Extern handledare: Matthew Davis Handledare: Mats Daniels, Anne Peters, Bjorn¨ Victor och Tina Vrieler Examinator: Bjorn¨ Victor Sammanfattning

Nar¨ pandemin brot¨ ut 2020 okade¨ anvandningen¨ av virtuella motesrum¨ som en platt- form for¨ onlineforel¨ asningar.¨ Anvandbarheten¨ av att bekvamt¨ fran˚ hemmet kunna delta i forel¨ asningar¨ kan inte fornekas,¨ men undersokningar¨ visar pa˚ att deltagare i virtuella motesrum¨ kanner¨ sig mindre engagerade i forel¨ asningar¨ nar¨ de deltar pa˚ distans. Ge- nom att involvera och engagera deltagarna i onlineforel¨ asningar¨ med hjalp¨ av virtuella Escape Rooms kan deltagarna kanna¨ sig mer engagerade och prestera battre¨ i sitt skol- arbete. Resultatet ar¨ en prototyp som visar hur virtuella Escape Rooms kan anvandas¨ i utbildningssyfte. Prototypen bestar˚ av ett rum som ar¨ sammankopplat till en grupp Escape Rooms. Deltagarna kan interagera med varandra, samt interagera med objekt och kommunicera via textchatt och rostkommunikation.¨ Ytterligare arbete behovs¨ for¨ att applikationen ska vara ett effektivt verktyg for¨ utbildning.

ii Contents

1 Introduction 1

2 Background 3 2.1 Virtual Meeting Room ...... 3 2.2 Escape Room ...... 4 2.3 Active Learning in Escape Rooms ...... 4 2.4 External Stakeholder ...... 5

3 Purpose, Aims and Motivation 5 3.1 Purpose ...... 6 3.2 Aims ...... 6 3.3 Motivation ...... 6

4 Sustainability and Ethics 7 4.1 Sustainability ...... 7 4.2 Ethics ...... 7 4.2.1 Availability ...... 7

5 Related Work 8 5.1 Unlock the Future - An Educational Escape Game ...... 8 5.2 Similar Products ...... 8 5.2.1 Engage ...... 8 5.2.2 Breakout EDU ...... 9

6 Method 10 6.1 Game Engine - Unity ...... 10

iii 6.2 Networking - Mirror ...... 10 6.3 Communication - Vivox ...... 11

7 System Structure 11 7.1 Clients ...... 11 7.2 Server ...... 12 7.3 Voice Communication ...... 12

8 Application Usage 12

9 Implementation 16 9.1 Managing the Escape Rooms ...... 16 9.2 State Synchronization ...... 17 9.3 Handling Commands ...... 17

10 Requirements and Evaluation Methods 18 10.1 Usability Requirements ...... 18 10.2 Evaluating Usability ...... 19 10.3 Requirements and Evaluation of Communication Functionality . . . . . 20

11 Evaluation Results 21 11.1 Initial Usability Test Results ...... 21 11.2 Evaluation of Communication Functionality ...... 22 11.3 The Second Usability Test Results ...... 22

12 Results and Discussion 23

13 Conclusion 24

iv 14 Future Work 25

Appendix 29

A First User Survey Results for Initial Prototype 29

B Second User Survey Results 33

v 1 Introduction

1 Introduction

The development of the internet over the last few decades has provided us with many new and exciting tools that we can use to improve our everyday life. One such tool is the virtual meeting room. The usage of Zoom for example, an online virtual meeting room [38], skyrocketed in 2020 going from 10 million daily meeting participants in December 2019 [36] to 300 million daily participants in April 2020 [37] due to the pandemic that occurred and prevented people from physically meeting in large groups. Virtual meeting rooms offer both advantages and disadvantages compared to physical meetings. They can offer an easy way of sharing links to websites and documents with participants during a meeting and increase the availability to work from home. Students and other users have reported that they experienced complications after transitioning to these meeting rooms [18]. The usage of the meeting rooms might cause tiredness [14] and a reduced sense of engagement [1], resulting in students not learning as much from lectures in virtual meeting rooms as they could have [9, pages 10-12]. Our proposed solution to this issue is a virtual meeting room in 3D (Figure 1) that has tools which can increase the engagement of the students, from now on referred to as participants, in the meeting room. By having the participants discuss and cooperate with each other we want to achieve more interactions among the participants compared to other virtual meeting rooms. The user, meaning the host or a participant, starts in a meeting room referred to as the ”main room”. Each user controls a virtual avatar. The participants can go through a doorway in the main room to move to a room called an Escape Room. In the Escape Room the participants have to complete tasks in order to be able to exit the room. There can be many different types of tasks such as finding the correct password to a keypad or pushing buttons in a specific order. It was intended for Escape Rooms to be customized by the host to fit the material being discussed or taught however this feature was eventually scrapped due to time constraints. Multiple Escape Rooms have been created and linked together in succession so that when the students complete one Escape Room, they walk through a door in the completed room to get to the next Escape Room. The participants can move to the next room when they complete the given tasks. There is a timer in each room counting down from a set time limit to induce a sense of urgency upon the participants.

1 1 Introduction

Figure 1 Two players standing in the main room.

In order to move around and interact with objects within the 3D room, the participant can look around by moving the mouse and use a set of commands from the keyboard and mouse to move and interact with objects. The set of commands are shown in Table 1. The resulting application was evaluated by doing usability tests. These tests showed us that the application was generally easy and enjoyable to use. Some usability issues where found as some test participants reported that they did not immediately understand the controls and sometimes had minor issues understanding each other using voice com- munication. However, despite this all test participants were successfully able to learn the controls, interact with the tasks in the Escape Rooms and communicate with each other using both voice and text chat.

Command Key Command Key Move forward W Jump Space Move left A Open Menu Esc Move backwards S Interact Left Mouse Button Move right D Table 1 Set of commands to be used as a user

Declaration of Division of Labor All group members have been involved in both the development of the application, research as well as writing the report. We felt like having all group members involved in all aspects of the project was a good idea in order for each group member to gain insight into all parts of the system. In each area of the project there were tasks that we distributed as evenly as possible.

2 2 Background

Nils focused on integration of voice communication as well as designing the audio menu. Sebastian designed and developed the keypad component (including network synchronization) in the escape rooms, implemented the text chat and assisted in syn- chronizing physics objects across the network. Andreas aided Nils with the integration of voice communication, designed the picture board component in the escape rooms and conducted the user tests. Christoffer designed the main menu, implemented the player controls and handled network synchronization of players, physics objects, and components within the Escape Rooms. All group members have been involved with reviewing and incorporating feedback into each section of the report. While each member has contributed to all sections in some capacity, there are sections which each member focused on. Nils focused on writing about the Purpose, Aims and Motivations of the project, Sus- tainability and Ethics as well as Results and Discussion. Sebastian focused on parts of the Background, Purpose, Aims and Motivation, Related Work, parts of Requirements and Evaluation, Evaluation Results and Results and Discussion. Andreas focused on the Introduction, parts of the Background, parts of Sustainability and Ethics, parts of Requirements and Evaluation Methods, Evaluation Results and the Results and Discus- sion. Christoffer wrote primarily about Method, System Structure, parts of Application Usage, Implementation, Requirements and Evaluation Methods and Future Work.

2 Background

This project focuses on creating a platform that uses a virtual meeting room combined with Escape Rooms to engage students in active learning in a 3D environment. Virtual meeting rooms, Escape Rooms and active learning are aspects that are central in our project and therefore presented in this section. We will also talk about our external stakeholder.

2.1 Virtual Meeting Room

A virtual meeting room is a digital environment that allows participants to meet and connect to each other in real-time online. Virtual meeting rooms of today utilizes for ex- ample video conferencing, teleconferencing, screen sharing, whiteboards, instant mes- saging, breakout rooms, polls and questionnaires to conduct lectures online. The aim of a virtual meeting room is to make it possible to communicate online even if

3 2 Background participants are located in different geographical locations.

2.2 Escape Room

An Escape Room is a game where players are locked into a room with the objective to get out of the room. To achieve this objective the players need to finish tasks. The type of tasks can vary in kind. Examples of such tasks include but are not limited to: discovering clues, solving math problems, finding hidden objects or a combination of these. You cannot exit or proceed to the next room until you have solved the tasks [15]. An Escape Room can take place in real life or digitally. The real life Escape Room industry has been growing rapidly for the past years, with over 2300 Escape Rooms in the US, and almost 1500 Escape Rooms in the UK 2019 [22] [34]. A precursor to modern Escape Rooms can be found in the 80s gaming industry where an early form of virtual Escape Rooms made their entrance. During this time text- based interactive games became popular. These types of games are built upon giving the computer commands by typing them in on your keyboard, and required the player to solve the same kind of tasks as the earlier mentioned ones. Text-based games evolved into Point-and-Click games as computers and graphics evolved and with the addition of the computer mouse [16]. Since the 1980s the gaming industry has kept advancing along with the advancement of computers, and the concept of Escape Rooms is still a commonly used element in video games [15].

2.3 Active Learning in Escape Rooms

Active learning is the process of learning where the students get engaged through ac- tivities and/or discussion in class, the opposite of passively listening to the lecturer [8]. It emphasizes higher-order thinking and often involves group work where the students can learn from each other [8] [20]. According to a meta study [8] the average failure rates of the courses that were examined in the study were 21.8% for students that attended active learning lectures and 33.8% that attended traditional lectures. Failure in this case meaning the grades D, F or with- drawing from the course in question. The study also showed that average examination scores improved by 6% in courses using active learning, compared to courses that did not use active learning. The concept of educational Escape Rooms favors active learning [20] which in turn

4 3 Purpose, Aims and Motivation increases student engagement. Educational Escape Rooms employ a technique called gamification [21] which draws from elements commonly used in games to motivate and engage students. Using active learning within Escape Rooms has proven to be effec- tive in increasing engagement for the participants and therefore also increasing concen- tration and learning [20]. However the Escape Room has to be designed with active learning and the course material as main aspects, which can be very complex and time consuming if the designers are unaware of the strategies to design educational Escape Rooms [7] [20]. To facilitate creation of Escape Rooms with education in mind, Jessica Reuter et al. [20] made a study with the aims to contribute to the body of knowledge about the process of applying Escape Rooms for educational purposes. They came to the conclusion of five main steps to take when creating an Escape Room, these are in order: Identify learning needs, define clear objectives, develop the Escape Room narra- tive, elaborate the challenges and puzzles and lastly, test and apply. These steps were taken into consideration when creating the prototype Escape Room.

2.4 External Stakeholder

This project is conducted in collaboration with Matthew Davis who is a Ph. D. student at Uppsala University as the external stakeholder. He wanted a better and more engag- ing way to conduct his sustainability course and meet the students all over the world in a more interesting way than Zoom and the other alternatives that exist today. A devel- opment that Davis wished for was to build a platform that utilized active learning in an Escape Room kind of manner. The idea is partially based on work performed by Emmy Pater [17] who created an educational escape game to promote sustainable thinking. Pater’s work is further discussed in section 5.1.

3 Purpose, Aims and Motivation

The goal of this project is to create an easy to use alternative to other virtual meeting rooms intended for education with features designed to engage its participants. We will further discuss the problems we aim to solve as well as the aims and motivations for the project below.

5 3 Purpose, Aims and Motivation

3.1 Purpose

The problem this project aims to solve is the lack of engagement people have reported when they participate in online learning through virtual meeting rooms like Zoom and Microsoft Teams [35]. According to perceived engagement reported by teachers and student self-reported engagement online teaching resulted in lower student engage- ment [35]. In that poll students and teachers were asked about their level of engagement in online lectures. 60% of teachers and 57% of students reported that students felt at least “a little less engaged” than in face to face learning. No teachers and only 13% of students reported that students felt more engaged. Therefore, a new solution for this is needed. The purpose of this project is to increase student engagement by creating a virtual meet- ing room that has Escape Rooms. Escape Rooms encourage active learning [20] through discussions between the student and interactions with puzzles directly related to the course material inside of the Escape Rooms.

3.2 Aims

This project aims to provide a platform where it is easier to engage the participants by employing active learning through Escape Rooms in a virtual meeting room. To achieve this we will create a meeting room in a 3D environment where students and a lecturer can move around and communicate by voice and text. A prototype will be created, consisting of a main room and three educational Escape Rooms that the students can complete to showcase how Escape Rooms can be used for educational purposes in a virtual meeting room. Furthermore the lecturer and students should be able to communicate through voice and text to promote discussions and teamwork.

3.3 Motivation

As mentioned earlier student engagement has decreased among students who are study- ing remotely due to the pandemic. But why is student engagement important? En- gagement among students has proven to contribute to ability and performance, school completion, reduced anxiety, reduced stress, reduced depression and reduced delin- quency [9, page 26]. Due to the Covid-19 pandemic there has been a massive increase in the amount of stu- dents who study remotely [36] thus it makes sense to provide a platform that assists in

6 4 Sustainability and Ethics making remote studies easier on the users. For example a survey [18] where a group of undergraduate students were asked about their remote learning experience was con- ducted. 94% reported having minor to considerable difficulties when learning remotely compared to their previous in-person classes meaning only 6% of students preferred studying remotely.

4 Sustainability and Ethics

In this section we will briefly discuss some of the ethical and sustainability aspects of this project.

4.1 Sustainability

This project primarily works towards the United Nations global sustainable develop- ment goal dedicated to quality education [24], goal number four. Since the purpose of this project is to be used by lecturers for educational purposes the goal of quality education is very closely linked to it. By providing teachers and students with a plat- form that promotes engagement through active learning this project contributes to this sustainability goal.

4.2 Ethics

There are some ethical aspects that had to be considered when working on this project. Usually when it comes to developing applications one of the main ethical aspects is how user data is handled. This however won’t be an issue since we do not store personally identifiable information.

4.2.1 Availability

The general design philosophy of this project is to make a lightweight application mean- ing an application that is easy to pick up and use while not requiring expensive hardware. This helps making the application available to more people. However in order to use this platform you need a computer and a stable internet connection which not all people have. These requirements are however hard to avoid. Perhaps in the future a phone application could be created to increase the availability. Being visually impaired makes

7 5 Related Work it harder to use our platform as it relies on a graphical interface that the user can interact with. Being audibly impaired makes it harder to communicate with other users through voice chat however there is a text based chat that one can use instead.

5 Related Work

This section contains products and methods worth mentioning that in some way are similar to this project as well as how they are different.

5.1 Unlock the Future - An Educational Escape Game

An influence for this project is the paper “Unlock the Future”: An Environmental Es- cape Game and its Development, Evaluation and Impact by Emmy Pater, teaching as- sistant at Uppsala University’s Department of Earth Sciences, Centre for Environment and Development Studies. She co-designed an environmental escape game called Un- lock the Future. The purpose of her thesis ”was to investigate how an escape game can be a catalyst for such a sustainable mindset change, from a theoretical, practice-based and experimental perspective” [17, page 41]. Pater explores the concept of using an Escape Room environment to influence and edu- cate its participants which is similar to what this project wants to achieve. The difference between Pater’s and our project is that Pater wants to influence the user’s values and the way the user thinks while we want to create a tool which can be used for education.

5.2 Similar Products

There exists multiple similar products that offers the ability to host some sort of virtual meeting room where a host can invite participants to listen to a lecture or presentation and interact in some ways. In this section some of these products will be discussed and compared to our own solution.

5.2.1 Engage

Engage is a virtual reality platform developed by Immersive VR Education [5], their product offers immersive virtual environments where a user can host a virtual meeting

8 5 Related Work for up to 70 people. In this virtual meeting room environment, the host can play a video on a screen, either in a predetermined position or if the users want to add more screens the video will also be played on the added screens. To play a video, the user has to provide a YouTube link or if you are a paying user the host can also utilize a share screen feature. The host can also do questionnaires, in the form of quizzes, tests and feedback reports, that can be prepared beforehand and then send it out to all the clients where the clients get a prompt to answer the questionnaire and submit the answers, which gets corrected immediately. Engage aims to improve the quality of online learning [6] but has a different method of doing so than this project. It aims to increase interactivity in online lectures and to promote experiential learning which is the process of learning by doing [11], while our method is to facilitate active learning. Engage focuses more on the aspects of recreating a regular in-person lecture in a 3D virtual meeting room, they differ from our proposed solution in their functionalities as we can not play YouTube videos or share the screen. A functionality we have that Engage does not is the Escape Room aspect.

5.2.2 Breakout EDU

Breakout EDU’s aim is to increase engagement during lessons by letting students play breakout games. The breakout games are simple 2D games where you need to use a computer mouse to select the correct answer after solving the problem that is presented on the screen. Their games helps students practice skills like critical thinking, creative thinking and collaboration skills by working together to solve academical puzzles [3]. On their platform they offer over 1800 digital games [4] as well the ability to create your own games using the platform. Breakout EDU also offers a physical lockbox with a set of different tools and locks. There are games on Breakout EDUs platform that are specially designed to utilize these tools to unlock the lockbox. The users will work to unlock the box by solving the puzzles in these games [4]. Some of these tools are printable puzzle compononents, red lens viewer, UV flashlight and an invisible ink pen. The games are then designed to be solved by using a set of these tools to discover the clues required to open the lockbox. Breakout EDU is similar to our project since it focuses on teaching methods that pro- mote active learning Section 2.3. There is however differences between this platform and ours. Breakout EDU is not a platform for online lectures and is primarily intended to be a tool to be used in a physical classroom. Breakout EDU utilize physical tools such as the mentioned lockbox while our application is entirely digital.

9 6 Method

6 Method

To speed up the development process, an existing platform for development of real-time 3D applications was preferred. It was also necessary for this platform to either provide or through other means have a networking solution available to facilitate the multiplayer capabilities of the application. A framework or service for voice communication was also needed, since implementing this on our own would be too time-consuming.

6.1 Game Engine - Unity

Unity, developed by Unity Technologies, is a cross-platform game engine used for cre- ating 2D and 3D applications. The engine offers support for over 20 different plat- forms, ranging from mobile to console and desktop platforms [30]. Unity uses a component-based scripting system; all entities are represented by a fundamental ob- ject called GameObject [29], to which components are then added to define its behavior and functionality. Unreal Engine is a competing platform offering a similar set of fea- tures [33], including an integrated multiplayer framework. It also comes bundled with many high-end visual resources, which may assist in quickly producing prototypes with high visual fidelity. Both platforms have an extensive feature set, documentation and a supporting community. Unity is however regarded by some developers as easier to use, has a wider range of supported platforms [32], and also offers a student license via the GitHub Student Developer Pack [31] for which we already qualified. These were the primary reasons we chose to use Unity for development.

6.2 Networking - Mirror

Mirror [13] is an open-source networking library for use with Unity, and the networking solution that was chosen. It is used to establish and manage connections between the hosts and the clients. It has extensive documentation and it is free. There are a num- ber of different alternatives to Mirror. One such is Unity’s own Multiplayer High Level API [25]. However, this system has been deprecated and is no longer maintained, which made it unsuitable for our purposes since we wanted a system that was still in active de- velopment. Unity is replacing this system with the Unity Mid level API [26]. This new system is still in an experimental stage [27] and is likely to have more implementation issues associated with it, such as bugs or API changes, which makes it an unsuitable choice for us.

10 7 System Structure

6.3 Communication - Vivox

Vivox [28] is a platform offering voice and text communication services, for example chat and voice channels in video games. It integrates well with Unity and is free for up to 5000 concurrent users. There are a few alternatives to Vivox, such as Dissonance Voice Chat [19] and Game Multimedia Engine [23]. Both of these can be integrated with Unity. Unfortunately neither offer the same services as Vivox for free, which is why we chose to discard them as solutions.

7 System Structure

An overview of the system structure can be seen in Figure 2. In the figure the clients represent the participants and the server is representing the host of the session. The Voice Service is provided by Vivox. The system is primarily based on a client-server model. The server runs a simulation of the virtual world, communicating any changes to the clients. This includes information such as the current positions of all the players or positions of different objects.

Figure 2 Overview of the system structure.

7.1 Clients

A client is a user joining a session created by a host. The client runs the application and sends information to the server about its actions. This could for example be moving around in the room or sending a text message.

11 8 Application Usage

7.2 Server

If a client performs some action, such as moving a character, this is communicated to the server. The server is the primary communication facilitator between the clients. Information is relayed by the server to the clients, to synchronize the client’s actions over the network. The clients in turn send information about their actions to the server to ensure that all participants experience the same actions on their screens. The main motivation for this type of architecture is that of security; it lets the system perform verification of a client’s actions on the server. In this sense, the server has authority over what changes are made to the virtual world. If a client attempts to alter the virtual world in unintended ways, such as moving around objects that they’re not supposed to be able to move, this will not be reflected on the other clients.

7.3 Voice Communication

Voice communication between clients is handled via the voice communication service offered by Vivox. The service is hosted on a separate cloud server which the clients connect to. This reduces the bandwidth consumption associated with hosting our ap- plication server, since voice communication traffic doesn’t need to be handled by the server host. Vivox also offers proximity voice chat that is easily integrated with Unity. Proximity voice chat is a technique where audio is altered based on positional data, to emulate real life conversations. This is handled by Vivox and it just needs the clients to submit their positions for their voices to be adjusted accordingly.

8 Application Usage

When the user starts the application they will be prompted to enter a name that will appear over their avatar and will then be given the option to either create a session or join an existing one (Figure 3). The options button seen in the picture is not enabled yet. When joining a session, a user must submit the IP-address of the host they wish to join (Figure 4), including a password if a password has been set by the host. When hosting a session, the host gets an option to add a password to the session. If the host chooses not to add a password the session will not be password protected and anyone can join the session.

12 8 Application Usage

Figure 3 The menu the user is presented with after entering their name.

Figure 4 The menu where the user is prompted to enter a network address and password in order to join an existing session.

The prototype consists of a main room that all the users start inside, and three Escape Rooms in which users can walk around and interact with each other through voice and

13 8 Application Usage text chat. The main room doesn’t have any features besides a whiteboard containing movement control instructions and a teleporter to the first Escape Room. The Escape Rooms are linked together by a door and a light that visually indicates whether the Escape Room is complete or not. The tasks all have the same theme; sustainability. The first room (Figure 5) contains pictures of everyday items which participants have to sort into recycling bins. The second room (Figure 6) contains a keypad that the user has to input the correct numerical code into, with hints to the solution spread and hidden around the room. You need information from all the hints to calculate the correct keypad code and continue. One hint is hidden on top of a staircase, one hint is hidden on the side of the staircase, and another hint is hidden behind an object you need to move. The third and final room contains a board with a question and five pictures. One of the pictures represent the correct answer to the question. The correct picture needs to be marked by clicking on it. The answer is then submitted by clicking on a button below the pictures. Once participants are done with the task in the last room the last door will open up and when the participant goes out of the room they’ll be teleported back to the main room.

Figure 5 A player completing a task in the first part of the Escape Room prototype.

14 8 Application Usage

Figure 6 The keypad and staircase in the second Escape Room.

If a participant presses the Escape key a Pause Menu will appear (Figure 7a). The Pause Menu contains the following options: ”Resume” resumes the participant to the game. ”Settings” lets the participant adjust the mouse speed. ”Quit to Main menu” takes the participant back to the menu where they can choose to join or create a game. ”Create Escape Room” is not yet implemented. ”Audio” lets the participant change input device and mute/unmute microphone (Figure 7b). ”Exit Game” closes the application.

15 9 Implementation

(a) The menu that appears when the user (b) The menu for adjusting audio settings. presses the Escape key. Figure 7 Pause menu and audio menu

9 Implementation

In this section we will describe in more detail how the Escape Rooms are managed, how remote actions are performed across the network and briefly how we have approached the security aspect of certain remote actions.

9.1 Managing the Escape Rooms

Each room in the Escape Room is managed by what we call a ”subroom manager”. A subroom manager is a system that keeps track of when a single Escape Room is completed and handles the transition to the next Escape Room. It uses a point system where each puzzle is defined to give some arbitrary amount of points when the puzzle is completed. The subroom manager also defines a minimum points requirement to unlock the next room. When the accumulated points reach this level, the Escape Room is considered completed. The manager will then change the color of the indicator light

16 9 Implementation above the door, which can be seen in (Figure 6), to green and open the door to the next room.

9.2 State Synchronization

The multiplayer aspect of the application requires that the state of the world simulation remains synchronized between the server and the clients. State, in this regard, is any in- formation required to reproduce the server’s world simulation on a client. For example, an individual user’s state is defined by their current position and rotation. Mirror, our networking library, has components that assist with synchronization. To synchronize variables, we only need to mark the variable with a tag in the code. For more complex synchronization, such as the state of a light, a door, or when a player picks up an object, we had to utilize Mirror’s remote procedure calls. In Mirror, a remote procedure call is a way to perform an action remotely. One such type is Command, which is a call from a client that makes the server perform some action, such as setting the position of an object. The other type is ClientRpc, which is a call from the server that makes the clients perform an action. An overview of how this works can be seen in Figure 8 below.

9.3 Handling Commands

By default, Commands may not be called by a client unless the client has been given explicit authority to do so beforehand by the server. This restriction exists as a security measure to prevent clients from altering the virtual world’s state in unintended ways. ClientRpcs do not have this restriction, since the server has full authority over the world state and is regarded as trusted. In our application, a user only has authority over their own avatar and may perform actions such as moving or jumping freely. For objects that can be interacted with by multiple users, we have disabled this restriction and instead perform verification on the server to ensure that the action the client is trying to perform is a valid one. The keypad component in the second Escape Room (as described in section 8) is a good example of how we perform such verification. A client is free to send a Command indicating that they pressed the button to submit the answer on the keypad, and the server verifies it by comparing it to the correct answer. If the server determines that the answer was correct, multiple ClientRpc calls are made by the server to ensure each client has the keypad interface updated (to reflect that this is indeed the correct answer), and also to update the state of the Escape Room (opening doors or switching the indicator light on).

17 10 Requirements and Evaluation Methods

Figure 8 Example overview of a communication exchange between the server and two connected clients, using remote procedure calls. In this example, Client 2 executes a Command call on the server, which in turn causes a ClientRpc call on every connected client.

10 Requirements and Evaluation Methods

Based on the wishes of our stakeholder, we put together some requirements for our project, both on usability and functionality. To evaluate the usability of the product we brought in test participants that answered surveys before and after performing a user test. These requirements and evaluation methods are discussed below.

10.1 Usability Requirements

To avoid interruptions in immersion, it is important that the application has a high us- ability. The International Organization for Standardization (ISO) defines usability as extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use [10]. Effectiveness can be defined as the accuracy and completeness with which users achieve specified goals. An example of this could be a measure of the amount

18 10 Requirements and Evaluation Methods of participants that were successful in connecting to a remote host. Efficiency can be defined as the relationship between resources used and the results achieved. Typically such resources include time, human effort, costs and materials. For example a partic- ipant should quickly understand how to maneuver the application or join a given IP- address. Satisfaction includes the extent to which the user experience that results from use of our application meets the user’s needs and expectations. This could be measured by surveying the participants on how enjoyable they found the application. The first time a participant enters the application they should quickly understand how to navigate the start menu interface and set their name. It should be easy for the participant to choose between joining and hosting a session, they should also find it easy to connect to a given IP-address. They should quickly understand the movement controls and how to get into the first Escape Room. The participant should quickly realize what to do, preferably through discussion with other participants. When the participants solve a puzzle they should feel a sense of satisfaction. After completing the last Escape Room the participants should understand how to get back to the main room.

10.2 Evaluating Usability

The structure of the test for evaluating usability is inspired by Mattias Arvola’s book Interaktionsdesign och UX: om att skapa en god anvandarupplevlse¨ [2]. Arvola is a researcher in cognitive science at Linkoping¨ Universitet [12]. Testing usability was done by having test participants fill out surveys before and after entering the application. They were surveyed about their earlier experiences with Escape Rooms before entering the Escape Rooms. After they had completed the tasks, they were given questions regarding what they felt about the Escape Rooms in the prototype. In the mentioned book Arvola says that test participants should represent the target group of the application tested [2], therefore students were picked as test participants. In total there were 8 test participants, who were divided into four test groups consisting of two test participants per group. Two of the test groups tested the application initially, and after resolving the given feedback from the first two test groups another two test groups tested it so that we could ensure that the same issues were not brought up again. A test leader was assigned from the project group. The test leader’s role was to provide the test participants with necessary information before the start of the test, introduce the tasks of the test and to instruct the test participants of what to do in case they got stuck for too long. User tests were conducted by giving the test participants a survey before entering the prototype. In the first part of the survey which they answered before going into the

19 10 Requirements and Evaluation Methods prototype, the test participants were asked if they consented to the use of their responses in the survey in this paper as well daily use of computers and earlier experiences with Escape Rooms. The tasks given to the test participants were the following:

1. Create a name for your avatar.

2. Connect to the given IP-address.

3. Move to the front of the big screen and await further instructions.

4. Enter the first Escape Room.

5. Get through all of the Escape Rooms by solving the tasks in each room and get back to the main room.

While the test participants were doing the given tasks, notes were taken about how the they handled the tasks, for example if there was anything in particular that they had problems with to understand. This was done by the test leader. The second part of the survey was to be answered after the test participants had completed the tasks they were given. It asked about their overall impression of the Escape Rooms, if they had any especially bad or good impressions of the application, if anything felt unclear when going through the rooms, what could be improved, if they felt like some function or content was missing or if they had any ideas of their own about tasks that could be added to the Escape Rooms. The survey questions and answers can be found in Appendix A and B.

10.3 Requirements and Evaluation of Communication Func- tionality

An important part of working together is to be able to communicate. We chose voice communication as the primary means of communication. This would let users commu- nicate with each other without relinquishing control of their avatar. Text communication was chosen as a fallback in case a participant did not have access to a microphone. For voice communication, users should be able to clearly understand each other without noticing any disruptions in communication. Such disruptions include delayed sound and poor sound quality that may make users unable to understand each other. In cases where participants have no access to a working microphone, text communication should be available as a fallback solution. This will be evaluated during user testing where the test participants will be surveyed on how well they perceived both voice and chat com- munication.

20 11 Evaluation Results

11 Evaluation Results

Applying the methods of evaluation mentioned above we could observe how well our product performed. The first survey results can be found in Appendix A and the second survey results can be found in Appendix B.

11.1 Initial Usability Test Results

All test participants were successful in creating a name for their avatar as well as con- necting to the given IP-address, as passwords on rooms were not implemented for the first user test session, we could not test the usability of connecting with a password. One test participant was confused about what an IP-address is, but managed to enter it successfully and connect anyway. All test participants could navigate to the big screen to await further instructions from the test leader, some test participants gave constructive feedback about the movement controls. One test participant said that they were not used to controlling an avatar with a keyboard and mouse at the same time and therefore felt like it was hard to control the avatar, but they quickly learned how to control their avatar. Another test participant experienced that the movement of the avatar felt a bit unresponsive where when you stop moving you do not stop immediately. No test participant had problems with entering the first Escape Room from the main room. The task in the first room was completed quickly by both test groups. One test participant did at first not understand how to pick up objects, but understood how to pick it up after discussion with the other test participant. The second room was completed with ease for both groups, but in one test group there was a misunderstanding regarding a staircase that the players need to climb in order to get information required to solve the task. The test group thought it was supposed to be a team effort to climb up the staircase since one of them did not manage to get up at first. This was due to an inconsistency in the avatar’s jump height, meaning sometimes the avatar was unable to jump high enough to get up the staircase. A bug was found and fixed after receiving feedback from the test participants. In the third room both groups experienced problems with understanding if anything happened when they picked the wrong answer twice in succession, since the message telling them ”Wrong answer” did not disappear or change between the answer submissions. Adding a timer on how long the text ”Wrong answer” is visible after submitting the wrong answer was added and fixed this issue. The effectiveness aspect of our product was partly satisfied. All test participants con- nected to the session successfully however one test participant could not communicate

21 11 Evaluation Results through the built-in voice chat. This was due to a bug in the code where if the test par- ticipant’s computer clock was not set correctly and therefore did not match the server time, this made them unable to connect to the voice chat. The efficiency aspect of our product was partly satisfied. Three out of four test participants had little trouble moving around and navigating the application. One test participant reported not knowing the controls for movement and picking up items at first. To clarify how you control your avatar we added a board with instructions in the first room. In the part of the survey that was to be filled after completion of the given tasks, all test participants said that their overall impression of the Escape Rooms was that it was fun and enjoyable which is tied to the satisfaction aspect of the ISO definition of usability in section 10.1.

11.2 Evaluation of Communication Functionality

Evaluation of voice functionality was conducted concurrently with the user tests. The presentation and assignment of the user test was held inside the application, and the test participants were encouraged to communicate with each other while solving the puzzles. None of the test participants reported that they were unable to understand each other. One test participant reported that they perceived brief cuts in the voice audio and that there were moments were the sound quality was regarded as poor. However, this did not disrupt their ability to communicate with the other test participant.

11.3 The Second Usability Test Results

A second test session was performed to evaluate the changes made from the results of the first user tests. The test was performed in the same way as the first one with the same survey and tasks. We wanted the first and the second test to be as similar to each other as possible so we could evaluate the changes we had made. All test participants successfully created a name and connected to the given IP-address. They proceeded to move to the big screen in the main room to await instructions. This time no test participants experienced problems with the movement controls and reported the controls being responsive. The board with information of how to control the avatar that was introduced since the first test might have had a positive impact in this regard. All test participants proceeded to the first Escape Room without any issues. All test participants understood how to pick up objects. One pair of test participants had some issues completing the first room. This was due to them having trouble with sorting some of the items into the correct bin as well as in one case having trouble identifying the object to be sorted.

22 12 Results and Discussion

One test participant had an issue where their headphones were partially damaged, re- sulting in one side of their headphones not playing any sounds. This is interesting since a person with impaired hearing on either ear could have a similar experience. They expressed that whenever the other test participant moved to the side where their head- phones did not work, they could not hear what the other test participant were saying. This is a result of the proximity voice effect adjusting audio volume based on direction, meaning that if the audio source is in the direction of the damaged earphone, the overall volume will be drastically lowered. A fix to this would be to lessen the proximity effect of the voice communication or implement an option to turn it off. Compared to the first test there were less issues with player controls and overall the second test group found the application enjoyable. To conclude the second test session, the usability was improved but could be further improved since test participants still experienced some issues with the voice communication.

12 Results and Discussion

Most of the project goals were completed, although due to time constraints one of the main desired functionalities was left out. The feature that was left out was the ability for the host to create and customize Escape Rooms within the application. It proved challenging to create such a tool that is easy to use by people with different levels of computer knowledge. For this reason we had to abandon the feature a few weeks into the project in favor of focusing on the other core features such as picking up objects to be used for puzzles in Escape Rooms, voice communication and text chat. The interactable objects and voice chat were especially of high priority. Without objects that the partici- pants can interact with there can not be any Escape Rooms since a required component of Escape Rooms is having various puzzles to solve. Voice communication is of great importance since that is the primary means of communication between participants in the application. Looking at the results of the usability tests one can see that the overall response was positive. More testing and evaluation is however needed in order to further investigate the effects of Escape Rooms in virtual meeting rooms. At the start of the project it was our intention to also evaluate how using our platform impacts the engagement of the users, however due to time constrains we were not able to do so. If the application had features such as teachers being able to make Escape Rooms themselves one could conduct user tests that are more relevant. We would need large control groups, a course to base the teaching material on and preferably a teacher that can hold the lectures as well as guide us in evaluating the performance of the students. For example a teacher

23 13 Conclusion could hold a lecture for two classes; one with Escape Rooms and the other without. Afterwards the students could be quizzed on the material and both students and the teacher could be polled on how they experienced the lecture. It is also important to make evaluation tests aimed at the instructor who creates the Escape Rooms. We believe that the idea behind the application has potential and that the application could be used for lectures given that the feature for the host to be able to create Escape Rooms is implemented. It is important in order for the application to be a practical tool for lecturers since it is not realistic for a team of programmers to handle potentially hundreds of requests of customized Escape Rooms from customers. The result of this project is an application that can be used to showcase how Escape Rooms can be used in an educational context. From the tests, we could see that the participants could engage and interact with the Escape Rooms and complete the tasks given to them and use all the desired functionalities. However some had problems connecting to the host with the given IP-address, connecting to a host could perhaps be done differently in the future, like using room codes or links that directly connects to a specific host. The evaluation of the prototype also provided some feedback on how controls could be improved, for example the control to make the avatar jump was not consistent at all times and made it hard to jump up on a staircase to find information to complete the second puzzle.

13 Conclusion

This report describes how a prototype has been developed that can be used to showcase how Escape Rooms in a virtual meeting room can be used for educational purposes by utilizing active learning. The prototype consists of a main room and three connected Escape Rooms. Participants can move and interact with objects in the Escape Rooms, as well as communicate with each other through text and voice communication. The results from evaluating the prototype showed some shortcomings in the usability. One of our requirements for the application is that participants should quickly under- stand the controls but some test participants expressed issues with this. Despite some usability issues, all test participants completed the given tasks, expressed enjoyment and wanted to see more of the product. Due to time constraints the functionality to create Escape Rooms within the application was not implemented. Implementing and testing the ability to create Escape Rooms within the application is necessary for the application to be used in an educational con- text.

24 References

14 Future Work

There are a number of potential improvements and features that could be added to ex- pand the application’s usefulness in an educational environment. One such feature is the ability to create customized Escape Rooms, which was discussed in section 12. This could make it possible for lecturers or course coordinators to adapt the Escape Rooms to any content without the assistance of a development team. An example of how this could work would be to provide a set of ready-made components, such as the keypad mentioned in section 8, that the Escape Room creator could configure and connect to other components. The user interfaces for these tools should have a high degree of us- ability and creating a custom Escape Room should be possible to do in a short enough time frame to be able to meet the time requirements of different creators. Another improvement would be to allow hosting via dedicated servers. A central server could keep track of ongoing sessions and users would be able to join a session via, for example, a room code instead of a network address. This would improve both usability and allow more users to host private sessions, regardless of their network configuration or any potential network bandwidth limitations that they might have. The ability to load and display PDF files inside the application is another feature that we regard as important. This could potentially expand the use of the application even further by allowing hosts to hold entire lectures or presentations inside the application.

References

[1] R. D. Adams. (2020, Jul.) Zoom fatigue by the numbers: A new poll looks at video conferencing engagement. Accessed 2021-04-26. [Online]. Tillganglig:¨ https://www.techrepublic.com/article/zoom-fatigue-by-the-numbers- a-new-poll-looks-at-video-conferencing-engagement [2] M. Arvola, Interaktionsdesign och UX: om att skapa en god anvandarupplevelse¨ . Studentlitteratur AB, 2014. [3] Breakout EDU. Accessed 2021-06-07. [Online]. Tillganglig:¨ https://breakoutedu. com/ [4] Breakout EDU. Accessed 2021-06-07. [Online]. Tillganglig:¨ https://store. breakoutedu.com/ [5] (2021, Apr.) Engage VR Homepage. Engage. Accessed 2021-04-12. [Online]. Tillganglig:¨ https://engagevr.io/

25 References

[6] EngageVR. Accessed 2021-06-07. [Online]. Tillganglig:¨ https:// immersivevreducation.com/about-us/

[7] P. Fotaris and T. Mastoras, “Escape rooms for learning: A systematic review,” in Proceedings of the 13th International Conference on Game Based Learning, ECGBL 2019, ser. Proceedings of the European Conference on Games-based Learning, L. Elbaek, G. Majgaard, A. Valente, and S. Khalid, Eds. Academic Conferences and Publishing International Limited, Oct. 2019, pp. 235– 243. [Online]. Tillganglig:¨ https://www.academic-conferences.org/wp-content/ uploads/dlm uploads/2019/10/ECGBL-Abstract-booklet-2019.pdf#page=81

[8] S. Freeman, S. L. Eddy, M. McDonough, M. K. Smith, N. Okoroafor, H. Jordt, and M. P. Wenderoth, “Active learning increases student performance in science, engineering, and mathematics,” Proceedings of the National Academy of Sciences, vol. 111, no. 23, pp. 8410–8415, 2014. [Online]. Tillganglig:¨ https://www.pnas.org/content/111/23/8410

[9] K. Hughes, “Impact of student engagement on achievement and well-being,” pp. 26 – 26, 08 2015.

[10] ISO, “Ergonomics of human-system interaction — Part 11,” International Organization for Standardization, Standard ISO 9241-11:2018, 2018, Accessed 2021-04-20. [Online]. Tillganglig:¨ https://www.iso.org/obp/ui/#iso:std:iso:9241:- 11:ed-2:v1:en:term:3.1.1

[11] Kent State University. Accessed 2021-06-09. [Online]. Tillganglig:¨ https: //www.kent.edu/community/what-experiential-learning-and-why-it-important

[12] Linkoping¨ Universitet. Accessed 2021-05-04. [Online]. Tillganglig:¨ https: //liu.se/en/employee/matar63

[13] Mirror Homepage. Mirror Networking. Accessed 2021-04-26. [Online]. Tillgang-¨ lig: https://mirror-networking.com/

[14] R. Nadler. (2020, Dec.) Understanding “zoom fatigue”: Theorizing spatial dynamics as third skins in computer-mediated communication. Computers and Composition. Accessed 2021-04-12. [Online]. Tillganglig:¨ https://doi.org/10. 1016/j.compcom.2020.102613

[15] S. Nicholson. (2015) Peeking behind the locked door: A survey of escape room facilities. Accessed 2021-04-26. [Online]. Tillganglig:¨ https: //scottnicholson.com/pubs/erfacwhite.pdf

26 References

[16] S. Nicholson. (2015) Peeking behind the locked door: A survey of escape room facilities. Accessed 2021-04-26. [Online]. Tillganglig:¨ https: //scottnicholson.com/pubs/erfacwhite.pdf [17] E. Pater, “Unlock the future: An environmental escape game and its development, evaluation and impact,” Ph.D. dissertation, Uppsala University, 2020. [18] E. Peper, V. Wilson, M. Martin, E. Rosegard, and R. Harvey. (2021) Avoid Zoom Fatigue, Be Present and Learn. NeuroRegulation. Accessed 2021-04-9. [Online]. Tillganglig:¨ https://www.neuroregulation.org/article/view/21206/13976 [19] Dissonance Voice Chat. Placeholder Software. Accessed 2021-04-27. [Online]. Tillganglig:¨ https://assetstore.unity.com/packages/tools/audio/dissonance-voice- chat-70078 [20] J. Reuter, M. Ferreira Dias, M. Amorim, C. Figueiredo, C. Veloso, and C. Figueiredo, “How to Create Educational Escape Rooms? Strategies for Creation and Design,” in Eighth International Conference on Technological Ecosystems for Enhancing Multiculturality, ser. TEEM’20. Association for Computing Machinery, 2020, p. 694–698. [Online]. Tillganglig:¨ https: //dl.acm.org/doi/10.1145/3434780.3436664 [21] A. Sanchez-Mena and J. Marti-Parreno.˜ (2017, Oct.) Drivers and barriers to adopting gamification: Teachers’ perspectives. Accessed 2021-04-26. [Online]. Tillganglig:¨ https://www.researchgate.net/publication/320735386 Drivers and barriers to adopting gamification Teachers’ perspectives [22] R. Sugar. (2019, Aug.) The great escape. Accessed 2021-04-20. [Online]. Tillganglig:¨ https://www.vox.com/the-goods/2019/8/7/20749177/escape-room- game [23] Game Multimedia Engine. Tencent Cloud. Accessed 2021-04-27. [Online]. Tillganglig:¨ https://www.audiokinetic.com/products/plug-ins/gme/ [24] United Nations. Accessed 2021-05-13. [Online]. Tillganglig:¨ https://sdgs.un.org/ goals/goal4 [25] The Multiplayer High Level API. Unity. Accessed 2021-04-26. [Online]. Tillganglig:¨ https://docs.unity3d.com/Manual/UNetUsingHLAPI.html [26] Unity MLAPI. Unity. Accessed 2021-04-26. [Online]. Tillganglig:¨ https: //github.com/Unity-Technologies/com.unity.multiplayer.mlapi [27] Unity Multiplayer Release Notes. Unity. Accessed 2021-04-26. [Online]. Tillganglig:¨ https://docs-multiplayer.unity3d.com/docs/release-notes/introduction

27 References

[28] Vivox voice and text comms. Unity. Accessed 2021-04-27. [Online]. Tillganglig:¨ https://unity.com/products/vivox

[29] (2021) Unity GameObject. Unity. Accessed 2021-04-21. [Online]. Tillganglig:¨ https://docs.unity3d.com/Manual/GameObjects.html

[30] (2021) Unity Homepage. Unity. Accessed 2021-04-12. [Online]. Tillganglig:¨ https://unity.com/

[31] (2021) Unity Student. Unity. Accessed 2021-04-20. [Online]. Tillganglig:¨ https://store.unity.com/academic/unity-student

[32] (2021) Unity vs Unreal Engine for XR Development: Which One Is Better? Unreal Engine. Accessed 2021-04-20. [Online]. Tillganglig:¨ https: //circuitstream.com/blog/unity-vs-unreal/

[33] (2021) Unreal Engine Features. Unreal Engine. Accessed 2021-04-20. [Online]. Tillganglig:¨ https://www.unrealengine.com/en-US/features

[34] S. Usborne. (2019, Apr.) Get me out of here! Why es- cape rooms have become a global craze. Accessed 2021-04-20. [On- line]. Tillganglig:¨ https://www.theguardian.com/games/2019/apr/01/get-out-how- escape-rooms-became-a-global-craze

[35] K. A. Walker and K. E. Koralesky, “Student and instructor perceptions of engage- ment after the rapid online transition of teaching due to covid-19,” Feb. 2021.

[36] E. S. Yuan. (2020, Apr.) A Message to Our Users. Zoom. Accessed 2021-04-20. [Online]. Tillganglig:¨ https://blog.zoom.us/a-message-to-our-users/

[37] (2020, Apr.) 90-Day Security Plan Progress Report: April 22. Zoom. Accessed 2021-04-20. [Online]. Tillganglig:¨ https://blog.zoom.us/90-day-security-plan- progress-report-april-22/

[38] (2021, Apr.) Zoom. [Online]. Tillganglig:¨ https://zoom.us/about

28 A First User Survey Results for Initial Prototype

A First User Survey Results for Initial Prototype

Below are the results from the user surveys, including the questions of the survey. The translations of the survey questions are provided in the captions of the figures. The responses are provided verbatim.

Figure 9 What was your overall impression of the Escape Rooms? (Was it fun, difficult, boring, unclear what was supposed to be done?)

29 A First User Survey Results for Initial Prototype

Figure 10 Was there something that gave you a positive impression, and if so, what?

Figure 11 Was there something that gave you a negative impression, and if so, what?

30 A First User Survey Results for Initial Prototype

Figure 12 Was it unclear what was supposed to be done in each room? If so, which room(s) and why?

Figure 13 Apart from aesthetic aspect such as object and room materials, what could be improved upon?

31 A First User Survey Results for Initial Prototype

Figure 14 Do you think that there is any content or functionality missing in the Escape Rooms, and if so, what?

Figure 15 Have you tested Escape Rooms in reality or virtually before?

32 B Second User Survey Results

B Second User Survey Results

Figure 16 What was your overall impression of the Escape Rooms? (Was it fun, diffi- cult, boring, unclear what was supposed to be done?)

Figure 17 Was there something that gave you a positive impression, and if so, what?

33 B Second User Survey Results

Figure 18 Was there something that gave you a negative impression, and if so, what?

Figure 19 Was it unclear what was supposed to be done in each room? If so, which room(s) and why?

34 B Second User Survey Results

Figure 20 Apart from aesthetic aspect such as object and room materials, what could be improved upon?

Figure 21 Do you think that there is any content or functionality missing in the Escape Rooms, and if so, what?

35 B Second User Survey Results

Figure 22 Have you tested Escape Rooms in reality or virtually before?

36