DEPARTMENT OF INFORMATICS

TECHNISCHE UNIVERSITÄT MÜNCHEN

Bachelor’s Thesis in Informatics: Games Engineering

A Serious Game about the Battles of the Second Punic War in VR

Moritz Rocksien DEPARTMENT OF INFORMATICS

TECHNISCHE UNIVERSITÄT MÜNCHEN

Bachelor’s Thesis in Informatics: Games Engineering

A Serious Game about the Battles of the Second Punic War in VR

Ein Serious Game über die Schlachten des Zweiten Punischen Krieges in VR

Author: Moritz Rocksien Supervisor: Prof. Gudrun Klinker, Ph. D. Advisor: Dipl.-Inf. Univ. David A. Plecher, M.A. Submission Date: 15.02.2019 I confirm that this bachelor’s thesis in informatics: games engineering is my own work and I have documented all sources and material used.

Munich, 15.02.2019 Moritz Rocksien Abstract

With more and more consumer grade virtual reality systems reaching maturity and becoming affordable, new opportunities in teaching and education emerge. This thesis will discuss the possibility of teaching players about the historic battles of the Second Punic War with a serious game in virtual reality. Important terms and concepts are briefly defined and existing related work analyzed. The requirements as well as the possibilities for such a serious game will be discussed and the approach and decisions for the subsequent implementation that was done for this thesis will be explained. The final game is evaluated in terms of performance as well as with regards to reaching its goals of teaching players about the actual history. For this, a user study was conducted and assessed.

iii Kurzfassung

In den letzten Jahren entstehen immer mehr technisch ausgereifte Virtual Reality-Systeme für den Heimgebrauch, welche völlig neue Möglichkeiten in Lehre und Bildung eröffnen. In dieser Abschlussarbeit soll die Möglichkeit erörtert werde, mittels eines Serious Games in Virtual Reality Spieler über die historischen Schlachten des Zweiten Punischen Krieges zu unterrichten. Dafür werden zuerst wichtige Begriffe und Konzepte kurz definiert und anschließend bereits vorhandene inhaltlich verwandte Arbeiten analysiert. Die Anforderungen an und die Möglichkeiten eines solches Spiel werden erörtert. Der gewählte Ansatz für die Implementierung und die Entscheidungen, die bei der Umsetzung des Prototypen getroffen wurden, werden im Anschluss erklärt. Um den Erfolg des Prototypen zu evaluieren, wird seine technische Leistung gemessen und eine durchgeführte Nutzerstudie ausgewertet um festzustellen, wie groß der Lernerfolg der Spieler ist.

iv Contents

Abstract iii

Kurzfassung iv

1. Introduction 1 1.1. Goals ...... 1 1.2. Outline ...... 2

2. Terms and Definitions 3 2.1. Virtual Reality ...... 3 2.2. Serious Game ...... 4 2.3. Game and Visualization ...... 4

3. Related Work 6 3.1. Rome: Total War and Total War: Rome II ...... 6 3.2. Serious Games in VR ...... 8 3.3. Virtual Reality Applications with History Content ...... 9 3.4. "Serious Game: Ancient Battles in VR" by Steen Müller ...... 10

4. Analysis and Definition of Requirements 12 4.1. Transfer of Knowledge ...... 12 4.2. Historical Battle of the Trebia ...... 12 4.2.1. Sources ...... 12 4.2.2. Background ...... 13 4.2.3. Course of Events ...... 14 4.3. User Guidance ...... 15 4.4. Tutorial ...... 15 4.5. Different Perspectives ...... 15 4.6. Gameplay ...... 16 4.7. Performance ...... 16

5. Approach 17 5.1. Existing Implementation ...... 17 5.1.1. Features ...... 17 5.1.2. Issues ...... 18 5.2. Improving Legibility ...... 24

v Contents

5.3. Enhancing the Replay Mode ...... 26 5.3.1. User Guidance ...... 26 5.3.2. Background Information ...... 27 5.4. Adding the Battle of the Trebia ...... 28 5.4.1. Introducing War Elephants ...... 29 5.4.2. Adding Mechanics to Represent History ...... 30 5.4.3. A New Map for a New Battle ...... 31 5.5. Evolving the Main Menu ...... 31 5.5.1. Concept ...... 31 5.5.2. Model Display ...... 31 5.5.3. First Person Weapon Interaction ...... 32 5.5.4. Information Displays ...... 33 5.6. Tutorial Level ...... 33 5.7. Performance Improvements ...... 34 5.7.1. Reducing Clone Instancing ...... 34 5.7.2. Reducing the Amount of Detail in the Scene ...... 34

6. User Study 36 6.1. Questions ...... 36 6.2. Results ...... 37

7. Evaluation 41 7.1. Performance ...... 41 7.2. Goals ...... 43

8. Conclusion 49

9. Future Work 50 9.1. User Experience ...... 50 9.1.1. Controls ...... 50 9.1.2. Tutorial ...... 52 9.1.3. Ergonomics ...... 52 9.1.4. User Guidance ...... 52 9.1.5. Legibility ...... 52 9.2. Performance ...... 52 9.3. Main Menu ...... 53 9.4. Content ...... 53

A. User Study Questionnaire and Results 55 A.1. Demographic Questions ...... 55 A.2. Previous Knowledge ...... 55 A.3. Game Experience ...... 59 A.4. Retention ...... 63

vi Contents

List of Figures 70

List of Tables 71

Bibliography 72

vii 1. Introduction

New technical possibilities created by the progression of Virtual Reality (VR) open up novel ways in education and teaching. Using the immersive and intuitive qualities of VR and combining them with elements of modern video game design, serious content can be presented in new immersive and engaging ways. The intuitive controls of modern virtual reality solutions also help to make these experiences accessible for a wide variety of audiences, instead of just seasoned video game players. With this in mind, this bachelor’s thesis tried to create a serious game in virtual reality that teaches users about the historic events of the Second Punic War in a playful and fun way to evaluate whether a serious game in virtual reality can effectively teach players about the battles of the Second Punic War.

1.1. Goals

The goal of this thesis is to create an educational but at the same time engaging experience for players. To do this, the resulting game has to satisfy two main aspects: Playing the game has to be fun in and of its own, and the game has to provide a high degree of historical accuracy and transport its content effectively. The knowledge this game aims to transfer is about the Second Punic War, specifically about two if its battles - the already implemented Battle of Cannae and the newly implemented Battle of the Trebia - as well as background information about the war and the armies involved. Therefore, adding the Battle of the Trebia to the game is paramount. To successfully convey this new and additional information, players have to be given access to all the necessary information and be able to freely explore but at the same time sometimes need guidance to correctly interpret and understand the presented information. To ensure that the game is accessible to a wide variety of audiences, a tutorial level that explains the control scheme and introduces the basic mechanics of the game is needed. Because of potentially game-ending problems that players could encounter with virtual reality sickness, the performance of the game is also an important concern and has to be taken into account. In the end, a user study has to be conducted to examine how well the game does at satisfying its two main aspects: the transfer of knowledge and the engagement of the audience. After all, if the game does not provide a better and/or more engaging way for audiences to learn about the Second Punic War compared to traditional methods of teaching, the game does not fulfill its purpose.

1 1. Introduction

1.2. Outline

Before discussing the specific implementation of the new content of the game, this thesis will start by defining terms and concepts that are important for this thesis. Following this, related work from the fields of traditional video games, serious games in virtual reality and serious VR applications with historic contents will be analyzed and their applicability to the tasks of this thesis examined. The fourth chapter will discuss the requirements for this thesis. This includes examining the historical Battle of the Trebia, as well as defining the necessary requirements the game needs to fulfill to be able to accomplish its goals of teaching players about the Second Punic War in engaging and immersive ways. Afterwards, the approach that was chosen for the game’s implementation will be detailed. This chapter starts by illustrating issues with the existing implementation as well as some of the measures that were undertaken to alleviate or solve them. Following this, the enhanced replay functionality with the newly implemented user guidance measures as well as the provided background information are highlighted. Adding the Battle of the Trebia as well as the necessary additions to the game that came with it are discussed. The improvement of the main menu and the concept behind it is the next point, with the implementation of the tutorial level and the measures undertaken to improve the performance of the game ending the chapter. After the approach, the conducted user study and its results are the content of the next chapter. The evaluation of the game in terms of performance as well as meeting its goals is the last chapter of this thesis, before reaching a conclusion and discussing potential future work.

2 2. Terms and Definitions

In this chapter, some of the necessary terms and concepts for this thesis shall be defined. A short definition for the terms virtual reality and serious game will be introduced and a distinction between games and visualizations will be made.

2.1. Virtual Reality

Figure 2.1.: The Vive head-mounted display with controllers. [Pes16]

In this thesis, virtual reality will be defined as a "computer-generated digital environment that can be experienced and interacted with as if that environment were real" [Jer15]. However, this definition still leaves a lot of questions unanswered: How will this virtual reality be experienced? And how can it be interacted with? Following J. Steuer [Ste92], the omission of these technical aspects is wanted for the definition, for the purposes of this thesis, though, it is hindering. In this specific instance, the virtual reality will be experienced and interacted with by using a HTC Vive [HTC] head-mounted display (HMD) in combination with two motion controllers. Both the HMD and the controllers are being tracked in the real world with movement being

3 2. Terms and Definitions translated into the virtual world. For interaction purposes, the controllers also feature four different buttons as well as a touch pad.

2.2. Serious Game

To be able to properly define the term "serious game", it is first necessary to split it up into two parts: the "serious" part, and the "game" part [DAJ11]. A very good definition for games can be found in David R. Michael’s and Sandra L. Chen’s book Serious Games: Games that Educate, Train, and Inform [MC05]:

[...] games are a voluntary activity, obviously separate from real life, creating an imaginary world that may or may not have any relation to real life and that absorbs the player’s full attention. Games are played out within a specific time and place, are played according to established rules, and create social groups out of their players.

It may be noted that enjoyment is explicitly missing from this definition. This is because, in their eyes, enjoyment is not an ingredient of a game, but rather a result of playing the game [MC05]. Now, after having defined the term "game", we can go on to take a look at what makes a game a serious game. According to Tarja Susi, Mikael Johannesson and Per Backlund, a game becomes a serious game when it "contribute[s] to the achievement of a defined purpose other than pure enter- tainment" [SJB07], with which both D. R. Michael and S. Chen [MC05] as well as D. Djaouti, J. Alvarez and J. Jessel [DAJ11] generally concur. This definition, however, does not exclude entertainment. Instead, it simply states entertainment is not the main goal of a serious game. In the context of this thesis, this defined purpose will be education. Another important distinction between serious games and gamification is made by S. Deter- ding, D. Dixon, R. Khaled and L. Nacke. They state that "whereas ’serious game’ describes the design of full-fledged games for non entertainment purposes, ’gamified’ applications merely incorporate elements of games" [Det+11]. However, they also note that such a distinction between the two concepts is often vague and always highly subjective.

2.3. Game and Visualization

For the purpose of this thesis it is also important to differentiate between the terms "game" and "visualization". The main distinction that can be made between a game and a visualization lies in their interactivity. For visualizations, their contents are entirely predestined, with users having no direct way to change or influence content between different sessions. For games, contents change dynamically corresponding to the users input and decisions, meaning that the experience between two distinct sessions will be completely different if the user chooses different actions. In games, the user has direct influence over the content presented to him.

4 2. Terms and Definitions

Or, as Juul puts it: "A game is a [...] system with a variable [...] outcome, where [...] the player exerts effort in order to influence the outcome [...]" [Juu11]. Another important distinction to make is that games - opposed to visualizations - have goals. Because visualizations are predetermined, users can’t actively try to achieve goals, whereas in a game, the motivation of users is to fulfill a self-imposed or external goal. This also leads to games being more engaging, as the users input actively shapes the events of the application and demanding their focus to successfully achieve their goals, whereas user input is entirely uninfluential in a visualization. However, depicting very specific events in a strict order can be difficult in a game, where the user has the option to shape the events on screen at any moment. That is why for strict reconstructions, visualizations with their limited interactivity can be better suited. In the context of the game proposed by this thesis, the replay mode constitutes a visual- ization, whereas the play mode is a game. In the replay mode, the interactive elements the player has at his disposal are limited to changes of perspective and jumping back and forth in time steps, with no way to actively change the events presented to him. On the contrary, in the play mode, the user actively shapes the events on screen with his actions and the goal to win against the AI.

5 3. Related Work

After extensive research, a combination of a serious game with Virtual Reality integration to teach about ancient battles in general and the Second Punic War specifically could not be found (of course with the exception of "Serious Game: Ancient Battles in VR" by Steen Müller). However, a combination of only two of the three categories (serious) game, Virtual Reality and ancient history leads to far more results. That is why on the following pages the three categories games with history as content (specifically Rome: Total War and Total War: Rome 2), serious games in VR and VR applications with history content will be examined more closely to determine the benefits and shortcomings of these works and how they relate to the goal of this thesis to provide a learning environment for the battles of the Second Punic War. The master’s thesis "Serious Game: Ancient Battles in VR" by Steen Müller, upon which this bachelor’s thesis builds, will also be analyzed and briefly presented.

3.1. Rome: Total War and Total War: Rome II

Figure 3.1.: The historical Battle of the Trebia as depicted in Rome: Total War.

6 3. Related Work

Out of the field of classic desktop computer games, Rome: Total War [Cre04] and the second installation, Total War: Rome II [Cre13] are both related to the game proposed for this thesis. They are set in the time of the Roman Empire from roughly 300 B.C. until 50 A.D., which covers the time frame of the Second Punic War which is the subject of the game for this thesis. The basic gameplay of both titles - old and new alike - consists of a turn based campaign map of the ancient world centered around the Mediterranean - reaching from the British Isles to the shores of northern Africa and from the Atlantic coast of the Iberian peninsula to the Near East - for troop movements and city administration as well as real time battles. Of special interest are the real time battles. Their gameplay tries to depict large clashes of the ancient world authentically, however not historically accurate as concessions have to be made either because of technical limitations or for the sake of gameplay. Due to showing individual soldiers of every unit, both games have to reduce the number of combatants significantly below the levels of typical battles at the time due to resource limitations. Typical battles in both games consist of only a few thousand soldiers, if not less, in contrast to real battles of the time period, for example the Battle of the Trebia, in which around 80,000 men fought [Pol89]. In the first title, every soldier of a unit also looks exactly alike - a flaw that has been removed in the later version, however it proved to have some useful consequences for gameplay as will be discussed later. The goal of battles in the Total War titles is to either kill every enemy or drive them from the field by lowering their morale. Each unit has a morale value and different values representing their fighting capabilities as well as their ammo should they have a ranged attack. Units that leave the field of battle or who lose all their men are defeated. The player assumes command over one army at a time, however it is possible to face multiple enemies and also have allied armies controlled by the AI all in the same battle. The maximum amount of units that the player can control in one battle is 20 for Rome: Total War and 40 for Total War: Rome II. Both titles also offer a historic battle mode. In this mode, the player assumes command of one predetermined side in an encounter that tries to replicate a historical battle of the time period. This is were the first installment deserves a special mention because it features the Battle of the Trebia as one of the historic scenarios (as can be seen in figure 3.1). However, rather than being accurate reconstructions of the events, these battles are merely inspired by the actual historic events and provide, apart from a short intro sequence and a rough reconstruction of army compositions and initial deployment, no real knowledge about the actual events. Due to the nature of the games as entertainment products, the real historic events get distorted by the means of the game, as the course of these historic battles were adapted to fit the constraints of the game rather than the other way around. For example, units that are not featured in the game but were present at the historical battles are substituted. These changes compared to the real historic events and the fact that any background information about the battles is only sparse, with the actual course of events not being explained or shown at all, stems from the fact that these games are not serious games and do not have the purpose of teaching players about the actual historic events of the time. They take inspiration from the events, but their main focus is to provide an enjoyable game

7 3. Related Work experience to players, and not educate them. In general, both titles provide a very good depiction of how tactics and battles in the ancient world might have looked like at a smaller scale, though some aspects of their design as well as their intended use as entertainment products keep them from being a historically accurate depiction. They also do not feature a representation of the actual events of the battles they show and only provide an initial setup. This, in addition to missing the benefits VR brings, unfortunately makes them unsuitable for being a learning environment about ancient battles.

3.2. Serious Games in VR

Figure 3.2.: The serious game to explore the solar system by Detlefsen. [Det]

Another type of game that was analyzed for the purpose of this thesis were serious games that utilize Virtual Reality that do not feature history or battles at all. In particular, a closer look was taken at the master’s thesis by Jan Detlefsen [Det14] to determine it’s usefulness and applicability to this project. For his thesis, Detlefsen developed a Virtual Reality game with the goal of teaching school children about the planets of our solar system. The information Detlefsen tried to convey were size, distance and order of the planets of our solar system. To achieve this, the users were able to travel from planet to planet with a spaceship and inspect them closer upon arrival with the possibilities of the VR implementation. Detlefsen notes that, after an initial period of high interest, the engagement of the users with the application dropped significantly after only a few minutes. He goes on to recommend a higher degree of gamification to keep user interest up and the users engaged with the content of the game. In the end, it was found that subjects had not significantly improved their knowledge of relative size and distance between the planets, but showed better knowledge of existence and order of the planets of our solar system.

8 3. Related Work

Due to the higher degree of gamification of the game of this thesis compared to the one developed by Detlefsen, making this game more clearly a game than just a gamified application than Detlefsen’s game on the spectrum between them (see 2.2), better engagement of subjects was expected for the results of this thesis.

3.3. Virtual Reality Applications with History Content

Figure 3.3.: Inside view of one of the buildings in Virtual Rome. [Nic]

Another field of related applications are VR visualizations of historic content. Two of these were examined closer, Virtual Rome [Nic] and Rome Reborn [Fri], which both have the same goal of delivering a historic reconstruction of the city of Rome which can be experienced in VR. Neither of those experiences, however, have any gameplay elements incorporated and only feature a virtual tour of the reconstructed city. Users go to different points in a reconstructed Rome and explore the ancient architecture, but they have no goals to fulfill or ways to interact with their environment. This makes these applications visualizations rather than games. They also do not show war or battles in any kind of form, providing different content and a learning environment aimed more towards ancient culture, architecture and urban life, making them distinct from the contents and experiences this thesis aimed to deliver.

9 3. Related Work

Figure 3.4.: The VR battle simulation game developed by Steen Müller. [Mül17]

3.4. "Serious Game: Ancient Battles in VR" by Steen Müller

The foundation this thesis will build upon will be the master’s thesis "Serious Game: Ancient Battles in VR" by Steen Müller [Mül17]. In the context of his master’s thesis, Müller conceptu- alized and developed a serious game for the HTC Vive to teach about ancient battles. The game features a single battle - the Battle of Cannae - as well as a main menu to start the game from. Players start in the main menu and can choose between three different options for the Battle of Cannae: they can either choose to spectate the battle, play as the commander of the Roman army or try to follow in the footsteps of Hannibal and command the Carthaginian force in the field. Once a selection has been made, the according game mode for the Battle of Cannae will be loaded. The graphical representation of the armies is inspired by chess or similar board game pieces with a symbolic representation instead of showing every single man of a unit. Information - like troop strength, armor and more - about a selected unit is displayed on an UI element attached to the controller in the secondary hand of the player, with the controller in his primary hand serving as the main input device. The pieces stand on a playing field that looks like a map of the historic battle site, which is placed on a table. Due to the free movement tracking of the HTC Vive, the pieces and the map can easily be viewed from different angles and positions. Additionally, a special camera icon can be placed on a unit to zoom the camera to its position and experience the game from its point of view or the player can point at an arbitrary point on the map and switch his perspective to that point. If a player chose to spectate the battle, they will not be able to command any of the units on the battle field but rather will be able to follow the historic events of the famous engagement of 216 B.C. through a series of steps to illustrate how both armies fared that day. Every step is divided into three parts, the first being the initial setup at the beginning of each individual step, the second showing the positions the units will move to, and the third being the actual movement of the troops. The pace is always being controlled by the user as the replay will

10 3. Related Work only progress with their input and they can always go back and take a look at earlier steps if they so wish. This part of the game is mostly a visualization of the actual tactics being used. If players chose to command one of the two sides into battle, the level will be loaded in the play mode and they can immediately start giving orders to their troops to maneuver and attack the enemy units with their controller. Units have different strengths and weaknesses and have to be used with care to be able to apply their full potential. Whether a player wishes to follow the historic course of the battle or tries to find a new strategy remains at their discretion. Once the player ends their turn, all units will be moved to their designated positions and will subsequently fight with their designated opponents. A morale system was planned, however it was not implemented. Once the fighting is done, the AI starts their turn and begins computing their next move. Once the AI has finished its turn, the pieces will move and fight once more and the player will have the opportunity to give commands to their units again. This goes on until one of the two sides has won the battle. The goal of Müller’s thesis was to create a serious game that was able to teach about ancient battles with higher levels of immersion and engagement than could be found in more standard mediums like books or documentaries, which he successfully achieved.

11 4. Analysis and Definition of Requirements

This chapter details the requirements the implementation of the game needs to fulfill to achieve its goal of being an educational and engaging experience. Additionally, the historical Battle of the Trebia will be analyzed.

4.1. Transfer of Knowledge

Of utmost importance for the game is its serious purpose, which in this case is the transfer of knowledge about the battles of the Second Punic War. It aims to educate players about the events of the two battles implemented, as well as about the strategies and tactics employed by their commanders. To provide as complete of a picture as possible, players also need to learn about the backgrounds of the battles: the events leading up to them, the composition of the armies involved, their arms and armor, their typical strategies in battle as well as the experiences of soldiers in the field. To reach this goal, it is important to make the game as historically accurate as possible. It is also important to keep the limitations in mind that are imposed by the medium chosen. Because the game needs to tell a coherent and complete story, a discussion of topics where no consensus has been reached as of yet in the scientific community is not easily possible. Possible gaps in historic records also have to be bridged with plausible solutions, instead of just omitting them. In general, one version of history to be told has to be found, even though this version might not necessarily exist or be available.

4.2. Historical Battle of the Trebia

4.2.1. Sources The main source for the time period of the Second Punic War is the Greek historiographer Polybius. Being born around the time of the end of the conflict, and having been acquainted with the family of one of the Roman consuls of the battle, he had access to eye witnesses from both sides as well as to the actual site of the battle [Man17]. Paired with his own demands of historiography to be truthful and unbiased, which he tried to meet himself as best as he could [Man17], he can be considered the best (almost) contemporary source for the Second Punic War, with only small restrictions.

12 4. Analysis and Definition of Requirements

4.2.2. Background The Battle of the Trebia marked the first major battle between Hannibal’s army and Roman forces in the Second Punic War. To evade a Roman army that blocked his way into Italian lands [Man17], Hannibal had previously led his men on an exhausting march over the Alps [Bar98]. After some small skirmishes between Roman and Carthaginian units and a first engagement at the Ticinus [Pol89], two Roman armies united to finally meet Hannibal’s force in battle. The spot that the Carthaginian general had picked for this was near the river Trebia, in northern Italy. The Carthaginians had their camp on one side of the river, with the Romans being encamped across [Hun17]. On the Carthaginian side of the river lay a flat and treeless plane, with a small river bed on their right side. This plain plane was ideally suited to allow Hannibal’s superior cavalry to maneuver and played to their strength, while it also led the Romans to believe they did not need to be afraid of any ambushes because of the open field without trees that would offer concealment [Pol89]. However, Hannibal had devised a plan to hide 1,000 infantry men and 1,000 cavalry mean each under the command of his brother Mago in an ambush in the river bed at the side of the destined field of battle the night before [Pol89]. According to Polybius, on the day of battle in December of 218 B.C. [Bar98], both sides had amassed around 40,000 men [Pol89], however, Seibert questions these numbers as Polybius appears to have only calculated them based on the number of legions present and doesn’t seem to have taken into account casualties from previous engagements [Sei93]. In any way, the composition of both armies differed greatly. Where Hannibal’s force consisted of 21,000 men heavy infantry from Iberian, Libyan and Celtic tribes, 11,000 men cavalry and 8,000 light infantry soldiers plus an additional unknown [Sei93] number of war elephants (Hunt speaks of 37 [Hun17], whereas Barcelo doubts any elephants survived crossing the Alps at all [Bar12]), the Romans led 36,000 infantry men and 4,000 cavalry men into battle [Pol89].

The Composition of a Roman Army in the Time of the Punic Wars

To fully understand these differences in army composition, it is important to know what a Roman army in the times of the Second Punic War looked like. The most important organizational structure of a Roman army during this period was the legion.A legion typically consisted of a total number of 4,200 infantry soldiers plus an attached force of 300 cavalry men (equites) [Man17]. The number of cavalry men is so low because Roman soldiers had to provide their own equipment and only wealthy citizens could afford a horse [Man17]. Of the 4,200 infantry men, 1,200 belonged to the light infantry, the so called velites, and 3,000 were heavy infantry soldiers [Sei93]. The Romans additionally distinguished between three different heavy infantry unit types: the hastati, the principes and the triarii. The hastati were the youngest soldiers of a Roman army and made up the first line of battle. They were armed with javelins, so called pila, and a short sword (gladius) for melee fights. For protection, they had a large shield (scutum), a helmet and a breastplate made from bronze or leather [Man17]. As every soldier had to provide his own equipment, small differences in armament were normal [Man17]. The principes made up

13 4. Analysis and Definition of Requirements the second line of battle. Their equipment was largely identical to that of the hastati, with the only difference being that their members were older and more experienced [Sei93]. The triarii were the most veteran soldiers of a Roman army and made up the last line of battle [Man17]. Because of their seniority, they were often able to afford better armor, like chain mail instead of a simple breast plate, and their weapons were only a single long spear (hasta) instead of a pilum and a gladius [Man17]. Every legion consisted of 1,200 men hastati and pricipes and 600 men triarii [Sei93]. In battle, the heavy infantry was divided into ten maniples à 120 men each [Sei93] for hastati and principes and 60 men each for the triarii. These maniples were then deployed in a checker board formation, leaving gaps the width of a maniple between them [Man17]. In front of this formation the light infantry was deployed and the cavalry was split evenly and placed on both wings [Man17].

4.2.3. Course of Events On the early morning of the day of the battle, Hannibal ordered his Numidian cavalry to cross the Trebia and attack the enemy camp. His plan was to lure the Roman army out and across the river [Pol89]. Tiberius Sempronius Longus, Roman consul and general in command, took the bait, blinded by some small victories in the previous days, and ordered his soldiers to chase the quick Numidians [Sei93]. The Numidians, following Hannibal’s plan, retreated back across the river, with the Roman army in pursuit [Pol89]. Reaching the other side of the river, the Roman troops were already exhausted from chasing the Numidians, wet and freezing from the ice cold water of the river and on top of that they didn’t have had any opportunity to eat a breakfast in the morning, as the Carthaginian attack took them by surprise, and so they found themselves at a major disadvantage right from the start [Man17]. The cold weather conditions - the day is being described as "snowy and excessively cold" [Pol89] - also meant that they could not cross the river a second time [Hun17]. Meanwhile, the Carthaginians had time to eat a breakfast and prepare themselves for the cold with anointments [Pol89]. Realizing the futility of their chase of the Numidian cavalry, the Romans regrouped into their typical battle formation and marched towards the now deployed Carthaginian battle lines [Pol89]. The Carthaginians had deployed with their cavalry split evenly on both wings, their light infantry in front and their heavy infantry in a long line in the center [Pol89]. Hunt places the Celts and Iberians in the center of the Carthaginian heavy infantry formation, with the Libyan fighters on the flanks [Hun17]. The position of the war elephants is by Polybius simply described in front of the wings [Pol89], however, whether this meant in front of the cavalry or in front of the infantry wings is still debated in the scientific community [Sei93]. The engagement was opened up by the light infantry contingents of both sides, showering their opposition with projectiles [Pol89]. After they had expended their ammunition, they fell back behind the lines of their heavy infantry [Pol89]. At this point, the Carthaginian war elephants started attacking the wings of the Roman heavy infantry [Pol89]. On the wings, the cavalry forces of both sides met, while in the center, the heavy infantry units of both sides

14 4. Analysis and Definition of Requirements started their fight[Pol89]. Almost immediately, the Roman equites lost the engagement and fled the field of battle [Sei93]. The Carthaginian cavalry forces turned around and, together with their light infantry units, started attacking the exposed flanks of the Roman heavy infantry formation [Pol89]. At this point, Mago and his troops come out of the previously prepared ambush and fell into the rear of the Roman triarii [Pol89]. Under the pressure, the Roman flanks and rear faltered, but in the center, around 10,000 hastati and principes managed to break through the Carthaginian lines and fled to the nearby town of Placentia [Pol89]. The remaining Roman forces were trapped between the Carthagini- ans and the Trebia, with most either falling or getting captured [Man17]. Exact casualty numbers are not known [Sei93], but Hunt estimates around 15,000 killed for the Romans [Hun17].

4.3. User Guidance

To clearly make the most important events of the game stand out to viewers, a system has to be implemented that guides users through the game. The goal is to implement a way to actively guide the user’s attention to the most important events. The use of virtual reality complicates this, though, as players can move and look around freely in all directions and typical overlays, like in desktop applications, are not well suited for VR. This requires novel ways to approach the problem of user guidance for virtual reality applications.

4.4. Tutorial

To make the game easily accessible for all users, a tutorial should be implemented. The tutorial should explain the control scheme of the game as well as the main mechanics of the gameplay. It should subsequently enable players to easily navigate the rest of the game. To achieve this, the control scheme of the game should be explained by the tutorial step by step, guiding the users through the process and enabling them to familiarize themselves with the concepts behind the controls. Additionally, basic mechanics like moving, attacking and ending a turn have to be introduced and explained to players.

4.5. Different Perspectives

The game proposed in this thesis aims to make use of the possibilities of virtual reality applications to the most of their potential. To do this, it should feature two different perspectives: One perspective that is removed from the battles and resembles a birds eye view for better overview and another one that shows the experiences of a foot soldier on the ground. The tracking capabilities of the Vive system provide a unique potential to make the experiences of a common foot soldier in ancient times tangible for modern audiences. For this, a variety of period weapons should be implemented that can be wielded by players in

15 4. Analysis and Definition of Requirements

first person with a degree of realism not possible in other types of medium like books or documentaries. The overview perspective, on the other hand, provides the perspective of a general planning a battle beforehand or commanding from a raised location like a hill. From this perspective, players have a great overview over the whole battlefield and can easily follow the events of the battle.

4.6. Gameplay

To ensure players sustained attention on the contents of the game and amplify their learning experience, the gameplay needs to be engaging and enjoyable. To achieve this, gameplay mechanics and controls need to be intuitive and the gameplay well balanced. The immersive qualities of virtual reality should be utilized to create an interesting experience that allows players to fully concentrate on the contents of the game. The gameplay should also provide a challenge, so that users will have to remain focused and not get bored, but not be too hard as to not frustrate players. To ensure a good learning environment for players, it should also always be possible for them to explore the historic contents of the game without having to worry about their progress in the game suffering.

4.7. Performance

Because of the refresh rate of the display of the head-mounted display of the Vive system of 90Hz [HTC], the game’s performance should reach 90 frames per second or more at any given time. This provides the best experience for players [BG16].

16 5. Approach

In this chapter, the approach that was chosen for the implementation of the game will be discussed. The chosen solutions will be explained, as well as the reasoning behind them.

5.1. Existing Implementation

Because of time constraints, it was not possible to create a game for the purpose of this thesis from scratch. Instead, the existing game implemented by Steen Müller for his master’s thesis [Mül17] was used as a foundation upon which this game builds.

5.1.1. Features To get a better understanding of the already existing implementation, some of the most important features need to be introduced. Additionally, a short description of what they do and how they work will be given.

Play Mode

The play mode is one of the two different modes of the game. In this game mode, players can choose one of the two sides involved in the battle, taking over as commander of that faction. The game starts with the historically correct initial deployment of the two armies and the player can then order his units to maneuver and attack the enemy. The goal is to defeat the enemy army, which is being controlled by an AI. As the game is turn based, players first choose the desired positions and targets for their units and upon ending their turn, the units automatically move to their goal positions and attack any targets that have been assigned.

Replay Mode

In the replay mode, players can observe the historical events of the battle to learn about the history behind the game. The historic battle is divided into several steps. Players can freely jump around between steps to go forward and backward in time. For every step, first the initial positions of all the units will be shown. After that, a ghost is being placed for every unit on the battlefield that shows where the unit will move. Finally, the units will move to these positions and, once they reach them, the next step will begin.

17 5. Approach

Menu

The game features a menu from which players can either choose on which side they want to play or whether they want to spectate the historical battle in the replay mode. Three different battles can be selected, but only the Battle of Cannae can be (re)played. For all of them, a short description is given upon selection.

Camera Positions

To give players a more detailed view of their armies, different options for camera positions exist. When players click on the ground somewhere on the map, they can observe the battlefield from that position. To return to the default perspective, the grip button of the Vive controller has to be pressed. Players can also attach a camera to one of their units. Once they end their turn, the camera perspective will change to the perspective of that unit and will follow its movements. When the unit reaches its goal, the camera is returned to the default position. When players take one of the special perspectives, they can no longer control their units. This is only possible from the default perspective.

5.1.2. Issues As far as the original game got in regard to reaching its goals, it still had some issues. These issues ranged from simple bugs with no real impact on the gameplay and knowledge transfer of the game, being merely a nuisance, to problems impacting both gameplay and knowledge transfer massively and in some cases even making finishing the game impossible.

Legibility

One of the most glaring problems with the initial implementation was legibility, that is to say identifying individual units and the side they belonged to. Due to the resolution of the Vive and the scale of the units, details that made them distinguishable from one another were almost impossible to recognize from the normal overview perspective. These details include, for example, different weapons and equipment, which were too small to be seen from the normal perspective during gameplay. As can be seen in figure 5.1, these differences between units were only visible from a very close distance. This meant that players were unable to quickly identify different units at a glance, leading to confusion. To communicate the necessary information more clearly and quickly to the player, a new approach had to be found.

Teaching of Controls

In the initial implementation of the game, there was no way for players to learn the controls on their own other than trial and error. This, coupled with the relatively complex control scheme, meant that the entrance barrier for the target audience of first time players was extremely high. It also meant that an introduction by a person that was familiar with the

18 5. Approach

Figure 5.1.: The old color scheme of the game. No distinction in color was made between different unit types. controls of the game was almost mandatory to play the game and players could not play on their own, which limits the ease of access, which should be as low as possible to allow as many people as possible to experience the game and learn about the historic topic of the game.

Attack Range Calculation

A further problem of the initial game manifested itself in the calculation of the attack range for the units. After moving a unit, if an enemy is in the attack range of the moved unit, players can choose a target, out of every enemy in range, to be engaged at the end of the turn. However, the calculation of this attack range was not done from the newly chosen position, but instead from the initial location at the start of the round. To make the situation even more complicated to assess for the player, the area used to highlight the attack range was not drawn with its origin at the initial location of the unit, which was used for the calculation, but incorrectly at the newly chosen one. After spotting this mistake, first attempts to correct the error were unsuccessful. Changing the game to use the correct calculation with the new position of the unit broke the AI and made the game unplayable. Due to time constraints, a band-aid fix was applied that did not fix the calculation to use the new position of the unit, but instead changed the displayed attack range to correctly use the initial location as its origin. This, however, was highly unintuitive, as sometimes a unit moved a great distance on the battlefield but still attacked an enemy unit close to their position at the start of the round that should have been out of their attack range after moving.

19 5. Approach

This version of the game was also the version used for the user study. The solution that was finally implemented and featured the correct calculation for the attack range, starting from the new position instead of the initial one, without causing problems with the AI, was only implemented later and was not available in the game at the point of the test sessions. The final way in which the problem was fixed should be more intuitive to players and cause less frustration and confusion, providing a better learning experience.

Performance

Figure 5.2.: The frame times measured during different situations in the game.

The performance of the game was initially struggling mainly in two situations, which would both occur during the replay mode. The first of those problems were large spikes in frame times every time a new replay step was loaded. These spikes could make the game freeze almost completely. Their impact on performance can be seen in figure 5.2, in the coloumn "Replay Mode (Cannae) (normal usage)" where they are shown by the extremely high maximum frame times. The trigger for these huge spikes in frame times was the instantiation of the clones in the replay mode. To show the positions to which every unit would move, a ghost object was created for each. However, in Unity, instantiating an object is a very resource intensive action [Uni]. During normal gameplay, this would not cause any issues, as both the player and the AI would typically only set one clone at a time. In the replay mode, though, the clones for every unit were instantiated all at the same time, causing a lot of strain on the CPU. The second problem was a low amount of frames per second whenever the goal position for the units were displayed in the replay mode. This can be seen in figure 5.2 under the point "Replay Mode (Cannae) with clones". For this type of benchmark, the replay mode

20 5. Approach was set to show the clones of every unit, however, during the benchmark no input was given and only the performance of the static scene was measured. In this situation, the game had to render not only the units in the game, but also a transparent ghost for every unit on the battlefield. This effectively doubled the amount of units that had to be displayed at the same time. Because the units were quite detailed and complex, with a high vertex count per unit, and the clones were transparent, which is more costly than simple opaque objects, this put a lot of strain on the GPU. This problem was not as prevalent outside of the replay mode because only one side - the player or the AI - is able to move its units at a time and so at worst only the ghosts for every unit of one side and not both had to be displayed at the same time. This can be seen by comparing the values to that of "Play Mode (Cannae)" in 5.2.

Unit Sizes and Values

Another problem with the original implementation were the values that were assigned to each unit. In the game, every unit has values to represent different aspects of its fighting strength, for example armor or movement range, and also a value that shows how many men are being represented by one unit on the map. The values chosen for these aspects, however, were problematic. The values that caused the most issues were the ones representing the amount of men that each unit had. These values were initially fluctuating immensely between units and made the actual amount of troops on the battlefield almost impossible to discern for the player. To give an example, the "Heavy Libyan Infantry" units and the standard "Heavy Infantry" units of the Carthaginians were represented by the exact same models, however the former had 1,666 men per unit whereas the latter only had 525 men. Due to having the same representation, this was impossible to recognize for players at a glance. Only comparing the values listed in the UI that were displayed once a unit had been selected showed this huge difference. This meant that players were unable to accurately tell the size of the battle by simply looking at the battlefield in front of them, but rather had to take a look at the number of men in each type of unit individually and add them up. As overlooking this could happen quite easily, players were at risk of getting a distorted impression about army sizes and compositions that was undesirable. To tackle this, the amount of men that each unit represented in the newly implemented Battle of the Trebia were changed. Both the standard "Heavy Infantry" units as well as the "Heavy Libyan Infantry" now represented 500 men each. The Roman heavy infantry units were also adjusted. Instead of representing 1,250 men like before, they now represented 650 men per unit. This was done to bring the value closer to that of their Carthaginian counterpart to help players discern the proportions of the battle more easily. The specific value of 650 was chosen due to multiple reasons. The first of those reasons was the need to adhere to the historical sizes of the armies and their compositions. The heavy infantry of a Roman army in the time of the Second Punic War consisted of three different types of units: the hastati, the principes and the triarii. Additionally, the soldiers were divided into legions (and even further into maniples etc. [Man17], but this was not within the scope of the game). The amount of hastati and principes in a legion was

21 5. Approach the same, with the amount of men that belonged to the triarii being half that of the hastati or principes. Furthermore, a typical Roman legion consisted of 3,000 heavy infantry soldiers and 1,200 light infantry soldiers (or velites) [Sei93]. This meant that a legion had 1,200 men each belonging to the hastati and principes and 600 to the triarii. Considering this, choosing 600 men per unit on the Roman side and using two hastati and principes units and one triarii unit would have been the first choice. This, however, caused problems with the historical amount of Roman soldiers involved in the Battle of the Trebia and brings us to the second reason. The number of Roman soldiers involved in the Battle of the Trebia is not completely certain and the historical accounts vary in their numbers. For this implementation, a number of 36,000 men infantry and 4,000 men cavalry was chosen, as described by Polybius [Pol89]. Of those 36,000, 10,000 soldiers were velites, leaving 26,000 men of hastati, principes and triarii. This number, however, was not evenly divisible by the 3,000 men heavy infantry in a typical legion. However, in wars or other special circumstances, the number of soldiers in a Roman legion could vary and additional men could be added to a legion [Man17]. That is why 650 men per unit was chosen as a plausible number for a slightly strengthened legion, while still being close to the amount of 600 men that a unit should have representing a typical Roman legion.

The second problem with the values lay not in a misleading representation of the historical battles, but rather with gameplay. The chosen values for some of the units either made no sense or caused the gameplay to be unenjoyable. An example for values that made no sense was the value for the attack range of the triarii. The initial amount of 1.2 meant that this unit could attack enemies almost at the other side of the map, despite historically being a melee unit. Obviously, this had to be changed. Other values that were altered included movement speed and turn rate and, most importantly, the attack damage dealt by every unit. The initial damage values were significantly too low and meant that battles could easily last more than an hour, being unnecessarily long and making the game very exhausting. As exhaustion could prove detrimental to learning about the historic events, changes had to be made. Therefore, the attack damage for every unit has been raised considerably. While making these changes to the unit values, the strengths and weaknesses of the historical units always had to be considered. For example, because of their large shield and armor in real life, the units representing the heavy infantry got considerably higher armor values than the light infantry units, whereas their movement ranges where lower due to the added weight of their equipment. Another example is the difference in attack damage between hastati units and principes units. Even though their arms were historically identical, the men of the principes were older and more experienced. This is represented in the game by a higher attack damage that signifies their higher proficiency with their weapons. Keeping these values accurate to the historical strengths and weaknesses of units is important to teach players about the units playfully.

22 5. Approach

Descriptions

The descriptions in the game and in some parts the lack thereof also were an issue with the initial implementation. The only description texts in the game initially were a short paragraph explaining the currently selected battle in the main menu and in the play or replay mode a short text in the hand-held UI that gave a bit of background information for the currently selected unit. This, however, meant that users were unable to get the bigger picture of the whole war and were limited to the events of the Battle of Cannae. But even there, the replay mode provided no explanation for the shown steps whatsoever, leaving the interpretation of the shown course of battle completely up to the player. In order to not only teach players about the events of a single isolated battle, but rather the intricacies of the whole war, this had to be changed. Additional descriptions are also able to provide more background information about the battles that could not directly be shown otherwise. Also, some of the description texts in the game had some minor issues with regards to historical accuracy or grammar - the latter mostly regarding the Latin names of units and equipment. In some cases, plural forms were used instead of the singular and vice versa. These were fixed wherever they were spotted.

Attacking Enemy Units

Initially, units could only be ordered to attack an enemy unit after being placed at a new position and optionally being rotated. This meant that it was impossible to leave a unit at the same location while also giving it an order to attack a specific unit. After feedback from the user study, new functionality was added to the game to enable this. It is now possible to select a unit and directly select an enemy unit to attack without having to first choose a new position for the unit. This way, units can now attack a specifically selected target without having to be moved first, making the gameplay more intuitive and enjoyable, which has a positive effect on learning.

Additional Problems

Unfortunately, the original game also had a plethora of minor additional problems, however some were so severe that they made finishing the game impossible. One of those was a problem that occurred every time a unit had no health left and "died". After the game object was subsequently destroyed, some functions still tried to access the object and caused a null reference exception. Even though checks for the object being null had been implemented, the problem persisted. Only after a few different solutions were tried, a workaround was found that fixed the problem and made the game progress normally. Another problem was caused by an incorrect query to check whether the grip button of the controller had been pressed, making the grip button and its functionality unusable. This meant that players could get stuck in the detailed view when they pointed at the map and pressed the trigger because they had no way of returning to the overview perspective from which they could control the game, forcing a restart of the game. After correcting the query and recognizing the input of the grip button correctly, the issue was solved.

23 5. Approach

Due to an oversight in the controller schematics, it was possible for players to select another unit with their second controller while they were already in the process of placing a unit with their first controller. This caused unwanted behavior, like the movement range of the unit selected with the first controller not being removed again correctly and staying visible on the map until the end of the turn. This issue was solved by adding a new controller state to the game. Whenever one of the two controllers would select a unit and start the placement process, the second controller would be set to this new state in which it could not do anything. Once the placement had been finished, both controllers were reset to their default state. A similar error occurred whenever the player ended his turn. To make a player unable to control any units during the AI turn, their controllers were set to a certain state that did not allow any selection of units. However, this state was only set for the controller with which the signal to end the players turn was given. The other controller still remained in its default state and was able to give commands to the player’s units even if it wasn’t the players turn. This meant that players could get confused as they were still able to control their units after ending their turn, making them believe they hadn’t in fact correctly ended their turn. After setting both controllers to the right state at the end of a turn, this issue was solved. Because of some missing calls to a clean up function for the attack arrows indicating the chosen targets of a unit, the arrows stayed until the end of the game. This did not have any significant impact on gameplay, but it caused some visual clutter and wrong information presented to players. Adding the calls to the situations in which they were missing solved the issue. Another bug that was in the game initially were some materials on the prefabs for the units that were broken. This may have been caused by upgrading the Unity version used to create the game, however it was fixed by recreating and reapplying the broken materials.

5.2. Improving Legibility

Tackling the issue of poor legibility, a new color scheme was implemented that assigned a specific color to both the side a unit belonged to as well as the type of a unit. To stick with the already established colors for the banners, Roman units got an outer rim in red, while Carthaginian units got one in white. However, the original implementation already tried showing this by having Roman and Carthaginian units use a different shade of brown. This was still used, but the difference in shading was amplified. The real issue, though, was making unit types more distinguishable from afar. The solution to this problem was in fact inspired by the first Rome: Total War game [Cre04]. In this game, every individual soldier of a unit looked exactly the same, owed to the fact that the game was first released in 2004 and the technology could not handle more diversity yet. This changed in the second installation, Total War: Rome II [Cre13], and made the game more realistic. However, it also had an unintended impact on gameplay. In the first game, even with the camera removed far away from the battle, units were still easily recognizable due to the color scheme and palette that each soldier shared with his fellow men in the same unit. In the second game, this was no longer possible, and players had to rely on UI icons or text.

24 5. Approach

Figure 5.3.: The new color scheme of the game. The outer rim shows the side the unit belongs to, while the inner color signifies the type of the unit.

Figure 5.4.: The new color scheme of the game when viewed from further away. The different types of units and the side they belong to are clearly distinguishable.

25 5. Approach

With this inspiration, a color coding scheme was devised for the type of units involved in the battle. Heavy infantry units stayed in the brown base color of the wood they initially had, light infantry units got a shade of green and cavalry units got a slightly red shade. The newly introduced elephants also got a shade of red, however a marginally different one, to underline their role as special units. This new color coding of unit types enabled players to distinguish quickly between melee infantry, ranged infantry and cavalry without the help of UI elements. This enables players to navigate the game more easily with less need to inspect UI elements foreign to the actual events on the battlefield. As a result, immersion was increased.

5.3. Enhancing the Replay Mode

The replay mode is the main device of the game to teach players about the actual events that took place in December 218 B.C.. To further facilitate the already existing replay functionality and amplify the learning experience, important changes were made to the game by enhancing the replay mode. The goal was to clearly show the players which events in each step where the most important ones and to also provide background information for every step of the replay to give them an even better understanding of the actual historic course of battle.

5.3.1. User Guidance

Figure 5.5.: The highlighted cavalry fight on the wings.

One of the goals of this thesis was to improve the already implemented replay mode by making the important events of each step more clear to players to aid their understanding of the actual historic events. To do this, a way had to be found to focus players attention on

26 5. Approach these important events. However, the use of virtual reality brought some challenges with it in this regard. In virtual reality, forcibly changing the orientation or field of view of the camera of the game can easily lead to virtual reality sickness symptoms and is therefore not advisable. This causes problems, though, if players have to be made aware of specific locations in the virtual environment, because if they are able to freely move and look around, it is easy for them to miss these specific locations. Because of this, a different approach to alert users to important events had to be found. The implemented solution does not rely on forcibly moving the camera, but instead clearly highlights the important areas on the battle map while players can still move and look around without restrictions. To highlight the relevant areas, an icon floats over the map above the specific locations with a light cone and a particle effect on the map itself drawing a circle around the relevant portion of the replay. The highlighting of specific areas on the battlefield was done to facilitate a better under- standing of the key events and areas at certain points in the historic battle. For example, if just watching the replay of the battle without such a system, the important cavalry engagement on the wings of the battle could easily be missed by players focused on the action at the center of the battlefield. Missing this key engagement, however, would make fully understanding the battle impossible.

5.3.2. Background Information To give players more background information about the battles that may not be possible to show by gameplay alone and to also explain what is going on in each step of the replay, a display was installed on the wall with descriptive texts that change depending on the context. In the play mode, the text on these displays is static and gives background information about the events leading up to the battles as well as other important information like the size of the armies or the date of battle. This was especially necessary for the newly implemented Battle of the Trebia as Hannibal’s strategy relied heavily on his preparation. He had scouted the surrounding countryside before meeting the Romans and had prepared an ambush that would prove critical for the outcome of the battle[Pol89]. In the game, it was possible to show the units in their hideout, however the circumstances that had placed them there were not possible to explain without the addition of the extra description. Another important factor in the historical battle was that the Romans had left their camp in pursuit of the fast Carthaginian cavalry without breaking their fast and without preparing for the cold [Pol89]. This was also impossible to display in the game and therefore made the use of background texts necessary. During the battle, the text changes and gives a more detailed description of each step. Not only are the events depicted explained, but also the thoughts and plans of Hannibal outlined. This gives players a more complete picture of the events of the battle and the strategies employed by both sides than would otherwise be possible. Whenever the description changes, a particle effect is triggered in front of the text to grab the users attention and make them aware of the new text.

27 5. Approach

The texts used for every step of the replay had to fulfill two requirements: They had to be sufficiently extensive to provide players with the necessary details, while at the same time staying short enough to not bore players or break their immersion.

5.4. Adding the Battle of the Trebia

To provide a more complete experience and explain the Second Punic War better, a second battle was implemented: the Battle of the Trebia. This engagement in December of 218 B.C. was the first major encounter between the forces of Hannibal and a Roman army. Some gaps in the historical records meant that the exact deployment of the Carthaginian side is not completely known today. For the most part, I went with the version of P. Hunt [Hun17], who put the Libyan infantry on the wings and the Celts and Iberians in the center, with the war elephants in front of the wings. This is also supported by Polybius’ Histories, where he notes that the Celts had taken the heaviest casualties following the battle, making them more likely to have been in the center where the Roman forces broke through the Carthaginian lines [Pol89]. For the replay mode, the battle was split into seven different steps, each showing a new important development in the course of the battle. This ensures that players will be able to understand every key event of the historic battle. The first step is the initial deployment of both armies before the battle. This initial deployment tried to follow the historic accounts as closely as possible, with the Roman heavy infantry force being divided into three distinct lines of battle with the hastati in the front and the triarii in the back. The Carthaginian heavy infantry is deployed in a long line, with both sides placing their light infantry units in front and their cavalry on the flanks. In the second step, both armies advance. Meanwhile, the light infantry of both sides starts the battle with their javelins. After showering their opposition with projectiles, both sides skirmishers fall back behind friendly lines while the heavy infantry keeps advancing. On the flanks, the cavalry battle begins. The elephants start their attack on the Roman infantry. In step four, the Carthaginian cavalry has driven its opposition from the field and turns to engage the flanks of the Roman heavy infantry together with the Carthaginian skirmishers. In the center, both formations of heavy infantry meet and start fighting. Following the attack of the cavalry on the flanks, in step five the previously hidden units come out of their ambush and assault the Roman rear. In the center, the Carthaginian forces suffer casualties at the hand of the excellent Roman heavy infantry. Creating a gap in the Carthaginian battle lines, in the next step a force of roughly 10,000 Roman hastati and principes manages to break through and escape. The last step sees the Carthaginians closing their lines again, trapping the remaining Roman forces. Almost all of them are killed or captured, which, however, is not being shown in the replay.

28 5. Approach

5.4.1. Introducing War Elephants

Figure 5.6.: The new war elephant unit as shown in the game.

The implementation of the Battle of the Trebia made it necessary to add a new unit to the games roster: war elephants. After crossing the Alps with his army, Hannibal still had a powerful force of war elephants at his disposal [Pol89]. The exact number, however, is unknown [Sei93], with the only information available being that it were less than 37 [Hun17]. For the purposes of the game, the war elephants were divided into four units with a strength of eight animals each, so that the game features 32 elephants in total. This number was chosen because it seemed very plausible with regards to the historical records and was also very fitting for gameplay purposes. The gameplay values chosen for the elephants were chosen mainly with the values for cavalry units in mind. Their movement speed was set slightly below that of cavalry units as well as their turn rate. Their armor rates were set to be very high to account for their impressive physique, with their attack damage also fairly high. The historical records do not describe the role of the war elephants in great detail. It is disputed whether their placement in the battle deployment of the Carthaginian army was on the wings in front of the cavalry or in front of the wings of the heavy infantry formation [Sei93]. For their role in the fighting, Polybius then describes their attack on the flanks of the Roman heavy infantry [Pol89], on which most secondary sources seem to agree. In the game, the elephants start on the wings of the Carthaginian formation. Following the clash of the light infantry of both sides, the elephants attack the wings of the Roman infantry formation. After the first cavalry engagement and subsequent defeat of the Roman riders, two units of elephants chase the fleeing velites - in line with Polybius descriptions of cavalry and elephants chasing down fleeing Romans on the shores of the Trebia [Pol89] - while the

29 5. Approach other two units keep on harassing the wings of the heavy infantry. The model of the elephant used in the game was kindly provided by fellow student Andreas René Keller.

5.4.2. Adding Mechanics to Represent History

Figure 5.7.: Three hidden units in an ambush.

To faithfully reconstruct the battle, two new mechanics had to be implemented. The first is a mechanic to hide units from the enemy and the second mechanic is one that gives certain units a disadvantage, in the case of the Battle of the Trebia caused by the cold water of the river and the insufficient preparation of the Roman troops. As the units under Mago that were laying in ambush were historically a major factor in Hannibal’s victory, a mechanic was implemented that allows units to be set as hidden. If a unit is hidden, it becomes invisible to the enemy player until it either moves or attacks a hostile unit. When units from the players side are hidden or the battle is being observed in the replay mode, all hidden units are visible but an icon floating above the hidden units will indicate their special statuses as well as the same icon on the hand-held UI. Once a unit has been discovered, it can not go back into hiding. This mechanic to hide units was necessary to accurately recreate the ambush that Hannibal had prepared the night before the battle. The second new mechanic that was implemented was one to decrease the fighting strength of units. It introduced the option to "debuff" units by a specified amount. Affected values were attack damage, attack range and armor as well as the currently unused values moral and hit probability. This mechanic was necessary to represent in the gameplay the impact that the exhaustion and unpreparedness for the cold had on the Romans. To show whether a unit is suffering from any "debuffs", the icon of a snowflake is displayed on the hand-held UI when the unit is selected.

30 5. Approach

The icons used were by David Merfield [Mer].

5.4.3. A New Map for a New Battle To represent the terrain at the Trebia, a new battle map had to be created. As the exact location of the battle is unknown today, only a relatively broad description of the terrain by Polybius could be used to recreate the historic site. Behind the Roman lines, the river Trebia flows from one side of the map to the other. On one of the flanks, a river bed was placed where the units that lay in ambush were positioned. To represent this river bed, a gravel texture was used [Mys]. Because of the time of year at the Battle of the Trebia and the described snowy conditions [Pol89], the grassy ground texture that had been used for the Battle of Cannae had to be replaced with a snow texture [Blu]. Through the use of a snow texture, the impact that crossing the freezing river had on the Romans is also better visualized and more intuitive to understand for players.

5.5. Evolving the Main Menu

In the initial implementation of the game, the main menu was relatively plain and did not offer any other functionality than to choose the play mode from. For the implementation within the scope of this thesis, more functionality and content was added to provide players with a more information about the historic events of the Second Punic War.

5.5.1. Concept The concept behind the new main menu was to not only provide a hub to start the different battles from, but to also add content to it. The goal was to make a distinction in perspective between the battles and the main menu. The battles were supposed to provide the perspective of a general ordering his units around or planning his strategy at a table beforehand respec- tively. On the other hand, the main menu is supposed to provide two different perspectives: That of an individual soldier in one of the two armies and an overview perspective over the whole course of the war and the previous events.

5.5.2. Model Display In the play mode, it is hard for players to closely inspect the models of the soldiers in the battle. From the normal perspective from which players play the game, the models are too small. If players use the camera feature, they are not able to move around freely and so can only inspect the models from a fixed position. In the new main menu, four models - two from each side - were placed at the sides of the table together with their faction banner. Because they are significantly larger than in the battles and because the movement of the players is not restricted, inspection of the models is significantly easier and more comfortable.

31 5. Approach

Figure 5.8.: The main menu of the game. Behind the table, the weapons that can be wielded in first person can be seen. At the left and right side of the table are the models that can be inspected.

This means that players can take a detailed look at the recreated units from all sides and learn about their weapons, armor and equipment.

5.5.3. First Person Weapon Interaction To use virtual reality’s possibilities and unique opportunities, models of the weapons that the units in the game use were hung up on an invisible wall behind the table. These weapons can be selected and will then replace the controllers in the hand of the player. This creates an experience unique to virtual reality where players can get to know what it was like to wield the weapons of a soldier in ancient times from a first person perspective - an experience books or movies are unable to provide. Players can choose between two shields - one Roman and one Carthaginian - two swords (the Roman gladius or the Carthaginian falcata), a spear (Latin: hasta) and the javelin of the Roman hastati and principes, the pilum. The order in which they are mounted on the wall is specifically chosen so that players intuitively choose a shield for the left hand and a weapon for the right hand, just like they would have been carried in ancient times. This one of a kind perspective increases immersion for players and let’s them learn about the arms of the Second Punic War with a first-hand experience which can not be experienced in other mediums.

32 5. Approach

5.5.4. Information Displays To give more background information about the conflict, a display was installed that provides information about the structure of the Roman army in the period of the Second Punic War and about the journey of Hannibal up to the Battle of the Trebia. The former explains the different types of units in a Roman army, the division of the Roman heavy infantry into the three different classes hastati, principes and triarii as well as their typical deployment strategy. The latter gives some background information about Hannibal’s siege of the city of Sagun- tum on the Iberian Peninsula and on its route across the Alps that led him into the heart of the Roman countryside. With this additional information, players are able to better place the events presented to them in the battles and gain a broader understanding of the Second Punic War going beyond just the events on the battlefields.

5.6. Tutorial Level

To not have players rely on a second person that already has experience with the games control scheme and to minimize frustration and confusion when starting the game, a tutorial level was implemented to teach them the basics. The actions that had to be covered in the tutorial were placing a unit and changing its orientation, attacking an enemy unit, attaching the camera to a unit, changing the camera perspective to an arbitrary point on the map and ending a turn. To teach all these commands, the tutorial is split into three different steps. An explanation for each step of the tutorial with a description of what to do is given at the side of the table at the position where, in a regular battle, the background information would be given. In the first step, players learn about placing and rotating a unit. The table is empty except for one unit controlled by the player and a highlighted area in front of it. The description explains to players the necessary input to place and rotate the unit. Once they have successfully placed their unit, they can end their turn by clicking on the banner at the side of the table. The second step teaches players about the different ways they can manipulate their viewing position by attaching the camera to a unit or by clicking somewhere on the map. The description asks players to again place their unit in a highlighted position in front of the unit, but, before ending their turn, attaching the camera to their unit. Combat is the last area the tutorial teaches. After players have successfully done the first two steps, an enemy is placed in front of the players unit. The description now explains to players how to give their men the order to attack. Once the player ends their turn, the two units will fight. After the enemy unit has been defeated, the game will automatically return to the main menu.

33 5. Approach

5.7. Performance Improvements

As was already discussed in chapter 5.1.2, the initial implementation was suffering from some performance problems. Because the Vive headset runs at a refresh rate of 90Hz and because virtual reality applications profit immensely from a high refresh rate, the goal was to achieve a frame rate of more than 90 frames per second at all times [HTC]. Otherwise, the experience would have suffered greatly, which could hinder player’s understanding of the content presented to them. To try and achieve to reach a stable frame rate of 90 frames per second or more, two different measures were undertaken.

5.7.1. Reducing Clone Instancing The first measure was to reduce the number of clones that had to be instantiated in the game. Initially, every time a clone was placed in the game, a prefab instance had to be generated. As instantiating is very costly in Unity, instantiating a lot of objects at the same time lead to stutter and short freezes of the game [Uni]. The solution to this was to instantiate the clone of every unit only once at the start and save a reference to this object for the rest of the level. This way, when a clone had to be placed, only the existing object had to be enabled and set to the correct position and orientation. As these actions are significantly less resource intensive, this countered the freezes when a lot of clones were placed at the same time in the replay mode. Once the clone was no longer needed, it simply had to be disabled instead of destroyed. The drawback with this method is that the CPU intensive action of instantiating the clone prefabs now had to be done at the beginning of each level, which added to the load time before players could start playing when coming from the main menu. However, as the load was very short anyways and was only increased by a few seconds at the maximum, this drawback was considered unimportant.

5.7.2. Reducing the Amount of Detail in the Scene The second important aspect to improving the performance of the game was reducing the amount of detail in the scene. Because the models used in the game were too detailed for the amount of units visible at the same time, GPU performance suffered. Because the models still needed their current amount of details for the display in the main menu or when the camera was directly attached to the unit, it was first tried to integrate a Level of Detail (LOD) implementation into the game. Because not all models were in the same file format, with some not being easily editable, an implementation that could decimate meshes directly in Unity was considered ideal. After some research, such an implementation was found [Edl]. After initial success getting the solution to work, with meshes being correctly decimated and LODs being generated, issues arose with applying these changes to every unit in the game. Because the generated meshes could not be saved explicitly they could not be applied to prefabs, which meant that after generating the meshes every unit would have had to be re-set in the scene. This would have also meant that every replay step would have to have

34 5. Approach been remade. After unsuccessfully trying to find a way to work around this problem, the implementation was eventually discarded. Instead, a simpler - but not as elegant - solution was found. Instead of reducing the details of every model, the amount of models in the scene was reduced. Every unit now has only two soldiers as representation instead of the initial four (except for the war elephants, which only ever had one).

35 6. User Study

To assess how well the game performed in reaching its goals, a user study was done with 20 participants in total. Every participant was asked to play the tutorial of the game and the Battle of the Trebia - choosing their side freely - as well as the replay mode for the Battle of the Trebia. The order in which they chose to approach the game’s game modes also remained at their discretion, apart from starting with the tutorial. No introduction into the controls or mechanics of the game was given so that players started without any previous knowledge. This was done in order to better assess the effectiveness of the Tutorial. To quantify the results, a questionnaire was prepared that consisted of two parts. The first part was to be filled out before playing the game, the second afterwards. To measure the experience of the participants, the core module of the Game Experience Questionnaire (GEQ) was used to determine immersion, flow, competence, positive and negative affect, tension/annoyance, and challenge [IdP13]. The full questionnaire together with the complete answers of the participants can be found in the appendix (A).

6.1. Questions

The questionnaire participants had to fill out started with some demographic questions regarding their age and gender. After this, players were asked to state their player type according to Richard Bartle’s taxonomy of player types [Bar96]. To easily determine their player type, participants were asked to fill out an online form that, upon completion, instantly provided a result [Bar]. The next step was to find out about players previous experience regarding both video games as well as history. For this, they were asked to rate how often they play video games, how much they know about ancient battles and some free text questions about relevant historical topics. The last question before starting the play session then asked players to indicate how often they had used VR headsets before, if at all. The questions about the previous knowledge of the participants about the historic events was necessary in combination with the history related questions at the end of the questionnaire to evaluate whether they had learned anything new by playing the game or whether they had already acquired their knowledge previously. After the play session, players first filled out the questions for the core module of the Game Experience Questionnaire and were asked whether they experienced any motion sickness while playing the game. Afterwards, they were asked to rate the user interface, controls and tutorial of the game, with the option of adding remarks in free text. Before testing the

36 6. User Study retention of information presented in the game, participants were asked to indicate whether they preferred the virtual reality experience to a more typical monitor display and how well they were able to follow the events in the replay mode. The last part of the questionnaire aimed to test how well participants of the study were able to recall important information about the Second Punic Wars that was presented to them in the game. For the first question in this category, they had to correctly indicate which of the two sides had historically won the Battle of the Trebia - the Romans or the Carthaginians. Next, they were asked to state deciding factors for the outcome of the battle, as well as any Carthaginian units they might know. The last two questions asked the participants what they took away from the game and whether they thought the game was a good way to learn about history, with both being answered in free text.

Due to the nature of the game developed for this thesis, some of the statements of the core module for the GEQ did not fit very well. For example, to determine the challenge score, one of the statements is "I felt time pressure". Giving a low score for this would also lower the score calculated for the challenge level of the game. However, because the game is turn based, this statement would naturally get a low agreement rating, and because players were also supposed to learn, time pressure was not wanted and even deemed counterproductive. On the contrary, it was explicitly wanted to provide players with an experience were they were free to explore and learn about the different aspects presented in the game without any form of time pressure. Because of this, the scores calculated for the challenge score of the GEQ have been mostly discarded.

6.2. Results

Taking a look at the demographics of the participants of the user study, it stands out that all 20 users were between the age of 18 and 29, with most of them (85%) being male. Almost all of them also indicated to be experienced gamers, playing often or all the time, and to have at least minimal experience with virtual reality, with none of the participants using a VR headset for the first time. Because of this rather homogeneous demographic, the conclusions that can be drawn may be limited and may not or only in limited terms apply to the general public. Concerning participants Bartle type, a majority of players were classified as explorers, with killers following. Only one participant got the achiever classification, while none were considered socialisers. One participant scored the same amount of affiliation to both explorer as well as achiever player types, leaving their classification ambiguous. The previous historic knowledge of participants was mixed. Only one participant self- indicated no previous knowledge about ancient battles, with the majority rating themselves at a low-medium to medium level of knowledge. Subsequent free text questions, however, proved this self-assessment to be relatively unreliable. While a fair amount of participants knew some basic facts about the Carthaginian general Hannibal and his deeds, 13 participants - or 65% - could not name any important battles at all or only referred to the Trojan War. For this question in particular, participants also tended to give examples of ancient wars they

37 6. User Study could recall instead of actual battles. Only four participants, representing 20%, were able to name other ancient battles, with their answers still being quite limited. Asked to name any units of the Roman army, a significant amount of participants did also not give any answer at all. The remaining answers were also relatively imprecise, indicating only limited previous historic knowledge about the topic of the game, if at all. Asked about the Second Punic War, the conflict the game tries to educate about, 70% of participants indicated to know nothing at all about it. The remaining six participants also did not feel very confident in their knowledge, with four choosing a rating of one and only two giving themselves a rating of two, while none rated themselves at a three or above. The estimates given for the time period of the Second Punic War where very scattered, with only two (or three, depending on what was meant with "A.C.") estimates being very close to the actual time period of 218 B.C. until 201 B.C.. Three more answers were reasonably close, with the rest being off by a fair amount. This marked the end of the first part of the questionnaire, which was being answered before playing the game. In summary, participants were tech-savvy young gamers whose knowledge about ancient history in general and the time period of the Second Punic War in particular was limited. To measure the results of the Game Experience Questionnaire, individual results were calcu- lated for every participant and then an average as well as a median of all participants was calculated (table A.2). Most notably, players experienced relatively high levels of immersion, flow and positive affect, while their competence and the experienced challenge of the game was lacking. While scoring the lowest values for tension/annoyance and negative affect, play- ers nevertheless felt slightly annoyed and negatively affected. Taking a look at the differences between average and median for these values, it becomes apparent that these scores were influenced a lot by some testers that experienced very high levels of annoyance and negative affect. While playing the game, 80% of participants did not experience motion sickness, with the rest indicating issues to some degree. Asked about the user interface of the game, testers had some gripes with it, scoring an average of 2.65 on a scale from 0 to 4. Provided free text answers, however, did not establish a consensus. Players were also not fully satisfied with the control scheme of the game, giving it an average rating of 2.15. In the additional feedback provided, testers complained about not being able to manipulate multiple units at once. They also found selecting the correct unit difficult, as well as rotating units. Observations of the gameplay also showed a significant amount of players not utilizing the feature to rotate the units at all. One participant specifically said about the controls that they were "confusing and [provided] little feedback". Interestingly, the ambidextrous control scheme of the game got praise both in the additional remarks for the controls as well as for the UI. The tutorial of the game was rated at an average of 2.32, with the most often voiced issue with it being that players found it confusing and did not immediately know what they were supposed to do, suggesting to split the explanations into more substeps and explaining what

38 6. User Study each button does individually. Another expressed suggestion was to increase the number of units to be tried. As the experience with the controls and the tutorial are heavily intertwined, some of these gripes with both of them and their relatively low average values could also help explain the lacking competence and annoyance scores of the Game Experience Questionnaire. Asked whether they preferred the virtual reality experience to a more typical monitor experience, 55% of testers stated that they indeed preferred the virtual reality experience, while 25% said they did not. The remaining four participants stated that, for them, it would depend on the circumstances. With an average value of 3.35, participants were able to easily follow the events presented to them in the replay mode. This high score could also explain that 100% of participants were able to correctly state which side had historically won the Battle of the Trebia. These results indicate that the attempts to implement a working user guidance system were successful. When asked about the deciding factors that led to the outcome of the battle, the majority of participants were able to correctly recall some or all of them, with some even being able to do so in great detail. Only two answers given were lacking so severely that they could not be considered correct. The deciding factors that were considered correct were the impact of the cold river on the Romans, the fact that they did not have time to prepare for the cold and the battle, the fact that the Carthaginians did, the ambush, the fast encirclement and that the river blocked the retreat of the Romans. When asked about any Carthaginian units they might be able to remember, 13 testers could name the Carthaginian war elephants, making them the frontrunner. In total, only two participants gave no answer, with ten being able to remember two or more units correctly. When asked about what they would take away from the game, the answers given were very diverse. Some talked about some aspects of their newly gained knowledge, others simply stated that they realized they had to learn more about history. At last, when asked whether they thought it was a good way to learn about history, all the participants agreed that it was, some citing a great amount of immersion and engagement and being able to explore things, while others still had some concerns, especially regarding the level of detail such a game could provide compared to more classical education. The description texts were also described as being detailed but not overly long, keeping them educational while not being demotivating. The interactivity was also described as helping in remembering the newly gained knowledge better than when reading a book or watching a video. When analyzing the data further, some connections could be made between how often participants played video games and how they rated the UI, controls and the tutorial of the game (see A.1. The participants that indicated that they play video games "all the time" gave higher ratings in all three categories than those that stated they played video games "often" or less. The average ratings of those participants that played all the time were 3.17 for the UI, 2.83 for the controls and 2.67 for the tutorial. Participants that played less gave on average ratings of 2.43 for the UI, 1.86 for the controls and 2.15 for the tutorial.

39 6. User Study

A correlation between the rating of the controls and the tutorial in particular and GEQ core module scores could also be observed (figure A.2). Users that had rated either the controls or the tutorial of the game or even both with a 1 or less had lower competence, immersion, flow, challenge and positive effect scores and higher annoyance and negative affect scores than those who hadn’t. Regarding retention, the four participants that had given the best and most complete answers in the retention part of the questionnaire also scored on average higher in competence, immersion, flow and positive affect and lower for annoyance, challenge and negative affect than the rest (figure A.3). The four best participants with regards to retention were selected based on their answers to the questions "What were deciding factors in the Battle of the Trebia?" and "Name any units of the Carthaginian Army you might know". The answer to the question "Which side won the Battle of the Trebia?" was not relevant because every participant answered correctly. Additionally, the better users had done in the retention questions, the higher their rating for the UI, the controls and the tutorial had been (figure A.4). The results for the question how well participants had been able to follow the events of the replay, however, remained stable and showed no correlation to how well participants had retained the knowledge presented to them in the game. The best category was again comprised of the four best results for the questions "What were deciding factors in the Battle of the Trebia?" and "Name any units of the Carthaginian Army you might know". Additionally, five "good" answers were recorded.

Not properly reflected by the results of the questionnaire is that during the user study, a significant amount of testers verbally complained about neck strain.

40 7. Evaluation

This chapter will address the evaluation of the game, starting with the performance of the game. Following that, it will be analyzed whether the game was successful in achieving its goals. For this, the results of the user study will be analyzed.

7.1. Performance

Figure 7.1.: The frame times measured during different situations in the finished game.

As has been stated in the requirements for the game, to provide an optimal experience for players that doesn’t distract from the contents of the game players are supposed to learn about or cause discomfort, the game has to have a good performance. As the refresh rate of the head-mounted display of the Vive is 90Hz [HTC], the performance level aimed for is at least 90 frames per second which corresponds to a frame time of roughly 11ms. Less frame time means more frames per second. All benchmarks were done on a system with an i7-7700k at 4.2GHz, a RTX 2070 and 16GB of DDR4 RAM at 2133MHz. As this is a relatively capable system configuration, performance on weaker machines may be worse. As the built-in profiler and the stats window of Unity always provide unwanted over- head that can distort the reported values for frames per second or frame times respectively, they were deemed too inaccurate. As running the game directly from the Unity editor also

41 7. Evaluation produces different results than testing a built version of the game, this too was deemed too inaccurate and not representative enough of final performance. Finally, to measure the performance of the game, a script was used in the games code that saves the time it took to calculate every single frame in an interval of five seconds and at the end finds the longest, shortest and average values. To provide comparable results, it also sets the camera to a fixed position overlooking the scene. Finally, the calculated values are written to a file. The game was then built in Unity and the final stand-alone application was benchmarked using the script.

In the main menu of the game, the frame times of the benchmark are significantly below the aimed for 11ms. The average frame time is only one tenth of that at 1.11ms, meaning the game runs at an average of 900fps in the main menu. The significantly higher maximum frame times occur only in occasional spikes without any specific reason. However, with 6.79ms they are also well below the threshold of 11ms aimed for. In the tutorial of the game, the average frame times are even better than in the main menu, providing a smooth gameplay experience. Only when a step is being finished and the particle effect of the display is triggered, the frame times spike. But the maximum value still does not exceed the limit of 11ms. During normal gameplay, the goal of 90 frames per second is also comfortably reached with an average frame rate of around 130fps. However, spikes in frame times occur when ending a turn. At these points, the frame times can reach highs of more than 35ms for a few frames. These increases in frame times are extremely brief, though, and are hard to notice when playing the game. During normal interaction with the replay mode, average frame times reach and even slightly exceed the limit of 11ms, but only by a marginal amount. The highest frame times still occur when changing replay steps and the game subsequently has to "create" the clones for all of the units on the map. The act of enabling and correctly setting all the clones at the same time still takes a toll on CPU performance and additionally, a newly implemented particle effect is triggered, which already proved to have an impact on performance as can be seen by the results for the tutorial. At these occasions, frame rates drop to around 25 frames per second. This is well below the aimed for 90fps, but marks a significant improvement compared to the 211.4ms of frame time (or less than 5 frames per second) that could be observed (in figure 5.2) before implementing the new measures to improve performance as described in chapter 5.7. Because these high values of frame time also only exist for very few frames, they are hardly noticeable compared to before and their impact on the game experience has been greatly minimized. The only area in which the goal of 11ms frame time or 90fps is not met is when a replay step is observed with all clones on display. At these times, the average frame time measures 13.35ms, or roughly 75 frames per second. The way in which this was measured was that the camera was locked to a position overlooking the complete map and a replay step configuration was chosen which displayed every clone, together with highlighters from the implemented user guidance system. No input was given to the game to measure only the impact of the

42 7. Evaluation graphical complexity of the scene on the performance. Even though the goal of 90fps was not reached, the results mark an improvement in performance over the previously measured 16.64ms (see figure 5.2)before the game had been optimized for performance. This result is further amplified by the fact that the previously measured scene for the Battle of Cannae features less units in total than the scene for the Battle of the Trebia (Cannae: 96, Trebia: 119) that was measured after the optimizations.

Even though the goal of 90 frames per second could not be reached in all situations, most of the game was able to clear the hurdle of 11ms for frame times. The performance optimization measures implemented in chapter 5.7 were also proven to have contributed to a better performance of the game. This improved and smoothed out the experience for players of the game and enabled them to fully concentrate and immerse themselves in the historic contents of the game. To guarantee this is also the case on weaker system configurations, though, further performance optimizations should be pursued in the future. Another aspect in which the performance of the game struggled was the calculation of the UI element to show players all possible positions at which they could place a unit. Upon selection, it could take up to a few seconds until the range was displayed. In the future, it should also be aimed to be make this more instant.

7.2. Goals

To evaluate how well the game did with regards to reaching its goals, the results of the user study have to be analyzed and interpreted. For this purpose, the goals of this thesis will be shortly recapitulated. As is necessary for a serious game, first and foremost the transfer of knowledge to players of the game had to be successful. To achieve this, the game had to provide historically accurate content that educates about the actual events of the battles of the Second Punic War and it had to transport this content in an engaging and immersive way to motivate players and optimize knowledge retention by keeping them focused. To begin with, because of the demographics of the participants of the user study, it is hard to draw conclusion that are valid for the general population (see 6.2). All of the participants were young and tech-savvy, meaning results with an older and less tech-savvy demographic may differ. First, it shall be analyzed how well the game did for its main purpose, the transfer of knowledge. Considering the answers to the first part of the questionnaire about previous knowledge, it would be fair to assume that none of the participants started into the game with any previous knowledge about the Battle of the Trebia in particular, which was the battle that was presented to them in the game. This means that any knowledge about this particular battle they had afterwards must have come from playing the game. The most simple question to test how much users had actually learned when playing the game was to ask them who had won the Battle of the Trebia. This question was answered correctly by all of the participants, which is a very good sign for the capabilities of the game

43 7. Evaluation as a device for learning about ancient battles. One of the reasons for this can likely be found in the high average rating of the question "How well could you follow the battles in the replay mode?". On a scale from 0 to 4, users gave a 3.35 rating on average. As the historic winner of the battle is only disclosed in the replay mode (in the play mode, users can take command of either army and lead it to victory, with no predefined winner) a correlation between these two results is highly likely. The amount of detail about the battle which players were able to remember was also quite good. To measure this, players were asked to give two free text answers. Asked about the deciding factors that historically led to the outcome of the battle, almost all players were able to correctly recall at least some of them. Only two participants could not give an answer that could be considered correct. Additionally, twelve users were able to name two or more important factors for the battle. This is a very good amount of players that were able to recall additional information about the battle, and shows that the transfer of background knowledge about the battle, as intended, was successful. Going over to the next question, the results were a bit worse, but still reasonably good. Players were asked to recall any Carthaginian units they might remember. Only two partici- pants were unable to recall any at all. The most often remembered units were the new war elephants, being named 13 times. Ten participants were able to name two or more units. It has to be noted, however, that sometimes answers were unspecific and did not necessarily prove a very detailed knowledge of the units the Carthaginians used. This might have come from an issue with their naming in the game, where for example the Iberian and Celtic heavy infantry is simply called "heavy infantry" and a more detailed explanation is only given in a short description text. In the future, more descriptive names should be chosen.

Figure 7.2.: The average rating of UI, Controls, Tutorial and how well players could follow in the replay mode with regards to quality of Retention results. The average of all results is also displayed for comparison. (Best: 4 Participants, Good: 5 Participants, Rest: 11 Participants)

44 7. Evaluation

An interesting point to mention is that even though players rating of UI, controls and tutorial correlated with how well they performed in the retention part of the questionnaire (figure 7.2), this was not the case for the replay functionality, whose rating remained relatively stable. What this means is that how well players were able to remember the history contents of the game did not rely on how well they could follow the events in the replay mode. This is counter-intuitive at first, as the replay functionality provides most of the background informa- tion available to the players. Instead, this proves another key aspect: The importance of the game experience for the retention of knowledge and therefore the success of the educational purpose of the game.

To find out whether players were actually engaged and immersed in the game, the results of the Game Experience Questionnaire core module are the most relevant ones, together with the free text answers at the end of the questionnaire. While, according to the averaged GEQ results, players experienced only low levels of annoyance and negative affect, the average score in "competence" was also quite low with only 1.43. This is very likely to be a result of the control scheme of the game with which players had issues. This is evidenced by the also quite low average rating of 2.15 for the controls of the game. The tutorial of the game is likely to also be responsible for the low levels of competence players experienced while playing the game. As the controls and the tutorial are necessarily intertwined - the one teaches the other - the relatively low rating of 2.32 for the tutorial can partly explain the low scores for the controls and therefore also the low scores for competence.

Figure 7.3.: The average GEQ core module scores for the four players with the best retention results compared to the rest.

The highest scores on average were for positive affect (2.57), sensory and imaginative immersion (2.49) and flow (2.48), while negative affect (0.75) and tension/annoyance (1) scored the lowest. This is a good sign, as it proves that players actually felt immersed and engaged by the game. This is also evidenced by the free text answers that, answering the

45 7. Evaluation question whether participants think this is a good way to learn about history, state multiple times that players felt immersed and engaged by the game, which aided their learning process immensely. A likely reason for the high level of immersion and flow players experienced is the choice of VR for the game. The head-mounted display of the Vive VR system uses the full field of view of users and cuts off any direct visual contact to the outside world. It can not be completely ruled out, though, that use of VR alone - as a new and exciting technology for some - may have inflated these scores. For example, the high scores for "I found it impressive" could maybe be attributed to the VR technology used, and not the game itself. However, there were also other contributing factors to these scores, as for example the large quantity of high scores for "I was interested in the game’s story", that can not be explained by the use of virtual reality alone. Also, only 55% of people explicitly preferred the VR experience to a desktop experience when asked, which would contradict this argument as well. The fact that the average scores for tension/annoyance and negative affect are relatively low is very important to provide the best learning environment for users. This is evidenced by the fact that the four players that had an annoyance score of 2 or more also did very badly in the final knowledge retention questions, with three of them remembering almost nothing besides the basic information of who had won the battle. On the other hand, the four players that did the best in the retention part of the questionnaire also had lower average values for annoyance and tension than the rest of the participants (see figure 7.3). Therefore, it is absolutely paramount to keep the annoyance and tension levels of players as low as possible to effectively facilitate the learning process.

Figure 7.4.: Average GEQ core module scores for players that rated either the controls or the tutorial (or both) with a 1 or less, compared to those that didn’t and the average.

One of the factors that could have lead to the annoyance and tension some players expe- rienced could have been the low rating of the controls and the tutorial. As can be seen in figure 7.4, players that gave the controls or tutorial a low score of 1 or less also had on average

46 7. Evaluation higher levels of annoyance than those that didn’t. This indicates that the low rating for the controls of the game and the tutorial directly correlate the existing annoyance score. Even a direct link can be drawn between doing worse in the retention part of the questionnaire and rating the UI, controls and tutorial lower than average (see figure 7.2), which proves the point that, in order to provide a more effective learning experience, the controls have to be improved to lower annoyance and increase competence levels.

Figure 7.5.: The average rating for UI, Controls and Tutorial, divided into players that said they played video games "all the time" and those that played less often.

In general, it can be said that the players that did worse in remembering the serious contents of the game were also the ones that experienced higher levels of frustration and lower levels of competence, immersion and flow (figure 7.3). These factors can then again be attributed to problems with the controls and/or tutorial of the game, as players that rated these areas lower had had a worse game experience (figure 7.4). There is also a direct link between how well participants did in the retention part and how they rated the UI, controls and tutorial of the game (figure 7.2). A correlation can also be found between how often participants played video games, how they rated the UI, controls and tutorial (figure 7.5) and their retention, as all participants that indicated they played video games "all the time" were rated in the "good" or "best" categories for retention. What this indicates is that the game has to be made more accommodating to players that play less or possibly even no video games at all to achieve its goal of being suited for a wide variety of audiences. This will be necessary to prevent them from getting frustrated and providing them with the most optimal experience that facilitates learning. Despite some of the issues with the controls of the game, when participants were asked whether they thought that this was a good way to learn about history and why, they all thought that it was. For their reasons, they cited a level of immersion and engagement that other mediums could not provide, as well as the interactivity of the content and the

47 7. Evaluation visualization of the battles. This, together with the good results of the retention part of the questionnaire, proves that the goal of creating a serious game to effectively teach about the battles of the Second Punic War was achieved. Only a few testers were concerned about the limitations of the chosen medium game when compared to other forms of education. Even though in the approach for the game it was tried to improve the legibility of the game, some players still expressed trouble being able to identify which side a unit belonged to. As this could negatively influence players understanding of the events presented to them, this was highly undesirable and would have to be improved in the future. As 80% of users also did not perceive any kind of motion sickness, ultimately the perfor- mance goals for the game could also be considered to have been met, however, improvements are still possible and should be done in the future.

48 8. Conclusion

In this thesis, a serious game to teach about the battles of the Second Punic War was proposed. After defining important terms and concepts, related work has been analyzed. Requirements for this game have been formulated that aimed to make the game immersive and engaging to effectively facilitate the transfer or knowledge about the historic events of the battles of the Second Punic War. Afterwards, a prototype has been implemented and the approach that was chosen as well as the reasons for the decisions made during the implementation process have been explained. To test the game, a user study has been conducted, the evaluation of which has shown that the main purpose of the game - teaching about the historic battles of the Second Punic War in an immersive and engaging way - has been reached. However, some flaws, mainly with the control scheme and implemented tutorial, have been discovered. These issues proved to cause frustration and annoyance for some players, which in turn hindered their immersion. Because of this, these player’s learning experience was negatively impacted and their knowledge retention suffered as a result. All in all, this thesis established that it is possible to effectively teach about the battles of the Second Punic War with a serious game in VR, even though the game experience still had some issues remaining.

49 9. Future Work

As shown by the user study, even though the game was quite successful at reaching its main goal of teaching about the circumstances and results of the battles of the Second Punic War, there is still room for improvement. These improvements should focus on the following areas of the game.

9.1. User Experience

9.1.1. Controls It became clear during the user study that the control scheme of the game is not intuitive enough to not be a distraction to players and in the worst cases became the reason for annoyance and serious frustration. This was a major issue as it posed a serious hindrance to the immersion of testers in the game and their ability to learn. Only a small number of players quickly understood the actions necessary to place and rotate a unit, with a not insignificant amount of testers not understanding how to correctly rotate units until the end of their play session. The concept of selecting a unit, placing it, rotating it and then confirming the rotation with another press of the trigger again (as illustrated in figure 9.1) was too unnecessarily complex. That is why any future work on this project should first and foremost focus on revamping the control scheme of the game. A possible solution would be to simplify and combine the process of placement and rotation of the unit. The new control scheme would be to select a unit with the trigger and to show a ghost at the location at which the player is currently pointing at on the battle map with his controller. Touching the touch pad will now rotate the ghost. Depending on whether the current position and orientation would be possible to reach, the ghost could be colored green or red accordingly, like it is being utilized in the game in its current form to distinguish between valid or invalid orientations already. When the player then presses the trigger while having both selected a valid location and orientation, the ghost would be set for the selected unit. This would simplify the mechanic of placing and rotating units to only need one confirmation in total instead of one for each. Another aspect that came up multiple times during the user studies and should be explored was the wish to be able to move multiple units at once instead of having to place every unit individually.

50 9. Future Work

Figure 9.1.: The control scheme of the game. [Mül17]

51 9. Future Work

9.1.2. Tutorial Another important aspect of improving players competence in using the games controls is to improve the tutorial of the game. To achieve this, the existing tutorial should be extended as well as broken down into more steps. Instead of explaining the process of selecting a unit, placing it and rotating it all at once, every step along the way should get its own explanation to make sure players understand every action correctly. Additionally, the tutorial should be extended to cover not only the basic controls but also feature different unit types to explain differences between ranged and melee units as well as the mechanics of the game, e.g. that units can be hidden.

9.1.3. Ergonomics Because most of the game takes place on the surface of the virtual table, players have to look down for prolonged periods of time. This, combined with the weight of the head-mounted display, caused some of the participants of the user study to complain about strain on their neck. This could be tackled by simply making the table a bit taller.

9.1.4. User Guidance As the results of the retention part of the questionnaire for the user study show, the user guidance for the replay system seems to work quite well. However, improvements could still be made by using voice over instead of or in addition to the text display at the side of the table. This way, players wouldn’t have to shift their view at all and could stay completely focused on what is happening on the battle map. Another way to use sound to improve the user guidance would be to play an alert sound effect at the position of every important event. This would give players a spatial queue to know where to look even if the event should happen outside of their field of view.

9.1.5. Legibility Even though this thesis tried to improve the legibility of the game as described in chapter 5.2, participants of the user study still complained about not being able to tell the side to which units belonged to easily. Possible solutions to this problem could be to either increase the share of the color that designates faction affiliation to a 1:1 ratio with the color that designates unit type or flip the color scheme to show unit type on the outer rim and unit side on the inside.

9.2. Performance

Improving the game’s performance should also be a main focus of working on the game in the future. In its current form, when having to display every unit and its corresponding ghost (as is the case when displaying a replay step), frame rates take a significant hit and the game

52 9. Future Work

Figure 9.2.: The color coding used to distinguish units. The outer rim shows which side the unit belongs to, the inner color corresponds with the unit type. is not able to sustain the aimed for 90 frames per second. A working LOD implementation could help take some of the strain off of the graphics processing of the system to provide better performance. The calculation of valid placement spots for units - both for the AI as well as for the display of the blue movement range indicator when the player selects a unit - could also be sped up by using a grid based approach or a k-d tree.

9.3. Main Menu

One of the participants of the user study remarked that they felt the main menu was overloaded and made them feel overwhelmed in the beginning. A simple solution for this would be to move the models of the Roman and Carthaginian fighters as well as the selectable weapons on the wall to a separate scene and make it accessible by, for example, clicking on a door with the label "Armory" on it.

9.4. Content

Maybe the most obvious of the sections, any future work should add more content to the game. To provide a complete picture of the Second Punic War, more battles have to be added, the most important ones being the Battle of Lake Trasimene, which occurred in 217 B.C. in-between the two already implemented battles at the Trebia and at Cannae, and the Battle of Zama in 202 B.C., which marked the end of the Second Punic War.

53 9. Future Work

The Battle of Lake Trasimene is important to understand the tactic of the Romans going into Cannae and Zama is significant as it not only marks the first major defeat of Hannibal against the Romans in an open field, but also triggered the complete surrender of Carthage and with that the end of the war [Man17]. Another aspect of the Second Punic War, Hannibal’s famous crossing of the Alps, would also be worth going into more detail about, giving players more information about the possibly most well known (as evidenced by the user study) feat of the Carthaginian general.

54 A. User Study Questionnaire and Results

A.1. Demographic Questions

Please state your gender:

• Male (17, 85%)

• Female (3, 15%)

• Other

Please state your age:

• <18

• 18-29 (20, 100%)

• 30-44

• 45-64

• >64

A.2. Previous Knowledge

Player type (Bartle):

• Explorer (12, 60%)

• Killer (6, 30%)

• Achiever (1, 5%)

• Explorer/Achiever (1, 5%)

How often do you play video games?

• Never

• Seldom (1, 5%)

• Occasionally (2, 10%)

55 A. User Study Questionnaire and Results

• Often (11, 55%)

• All the time (6, 30%)

How much do you know about ancient battles?

• 0: Nothing at all (1, 5%)

• 1 (9, 45%)

• 2 (8, 40%)

• 3 (2, 10%)

• 4: I’m an expert

What do you know about Hannibal?

• blank (5)

• Only know the name (3)

• "Das ist schon länger her"

• "Er überquerte die Alpen mit Elephanten"

• "He crossed the Alps and battled Rome"

• "Was leader from Carthage that invaded the Roman empire by climbing the alps with his army and elephants, he died when captured by enemies by drinking poison"

• "Ein Feldherr, der mit Elefanten über die Alpen ist und (fast) Rom erobert hat"

• "serial killer"

• "a serial murderer who cooks and kills humans"

• "Er führte die Karthager in die Schlacht gegen das römische Reich, wobei er aber unterlag. Im Zuge dessen überquerte er mit seinem Heer und unter anderem Elefanten die Alpen"

• "Kam aus Karthago, kämpfte gegen die Römer, dafür führte er eine Armee mit Elefanten über die Alpen"

• "Warlord who crossed the Alps with Elefants"

• "Karthago, Elefanten über Alpen"

• "Karthagischer Feldherr; führte relativ erfolgreichen gegen Rom; Alpenüberquerung"

• "Irgendein Kriegsherr; ich verbinde ihn mit Elefanten also war er vermutlich im Osten"

56 A. User Study Questionnaire and Results

Name any important battles you might know:

• blank (8)

• "Battle of Troy" (5)

• "the one, where the whole fighter lost in the germanion forest"

• "trojan war, gallic war, first punic war, second punic war, roman civil war"

• "Troy, Adventures on the Odyssey, the fall of the roman empire, ghengis khans monopoly"

• "Trojanischer Krieg, Varusschlacht"

• "Teutoburger Wald"

• "Schlacht im teutonischen Wald + ein paar andere wo ich den Namen vergessen habe"

• "Battle of the 300"

Name any units of the Roman Army you might know:

• blank (8)

• "Centurio" (3)

• "phalange (I think it’s a french naming, forgot specific names)"

• "Legionär, Centurion" (3)

• "Kaiser, Generals, foot soldiers, commanders"

• "Legionen" (2)

• "spearmen, archers, shield carriers"

• "Legionär? Ich denke Fußsoldaten & Bogenschützen gab es auch"

How much do you know about the Punic Wars?

• 0: Nothing at all (14, 70%)

• 1 (4, 20%)

• 2 (2, 10%)

• 3

• 4: I’m an expert

When would you estimate the Second Punic War took place?

57 A. User Study Questionnaire and Results

• blank (5)

• "200 A.C."

• "500 - 1000 A.C."

• "200 B.C." (2)

• "40 B.C."

• "400 n. Chr."

• "1400"

• "1000"

• "150 v. Chr."

• "900 v.C."

• "50 v. Chr."

• "500 A.C."

• "150 - 100 B.C."

• "100 B.C."

• "1700"

Did you ever use a Virtual Reality headset before?

• No, never.

• Yes, once or twice. (13, 65%)

• Yes, occasionally. (7, 35%)

• Yes, often.

• "Once, but couldn’t appreciate it because of my eyesight"

58 A. User Study Questionnaire and Results

A.3. Game Experience

Please indicate how you felt for each of the items in table A.1:

Table A.1.: The items for the core module of the Game Experience Questionnaire not at all (0) slightly (1) moderately (2) fairly (3) extremely (4) I felt content 0 5 6 6 2 I felt skillful 3 9 3 4 1 I was interested in the game’s story 0 0 4 12 4 I thought it was fun 1 1 6 9 3 I was fully occupied with the game 0 1 7 9 3 I felt happy 0 3 9 5 3 It gave me a bad mood 17 2 0 1 0 I thought about other things 11 8 0 1 0 I found it tiresome 4 8 5 2 1 I felt competent 3 5 8 4 0 I thought it was hard 1 12 4 2 1 It was aesthetically pleasing 0 4 10 5 1 I forgot everything around me 0 5 5 9 1 I felt good 0 2 5 10 3 I was good at it 6 7 4 3 0 I felt bored 10 7 2 1 0 I felt successful 3 3 7 6 1 I felt imaginative 3 2 9 3 3 I felt that I could explore things 2 3 6 5 4 I enjoyed it 0 2 4 9 5 I was fast at reaching the games target 10 6 2 2 0 I felt annoyed 9 7 2 2 0 I felt pressured 11 6 3 0 0 I felt irritable 6 7 7 0 0 I lost track of time 0 4 6 4 6 I felt challenged 1 2 8 7 2 I found it impressive 0 0 6 11 3 I was deeply concentrated in the game 0 1 8 8 3 I felt frustrated 9 5 3 2 1 It felt like a rich experience 0 3 5 9 3 I lost connection to the outside world 0 9 2 6 3 I felt time pressure 14 3 3 0 0 I had to put a lot of effort into it 1 8 8 3 0

59 A. User Study Questionnaire and Results

Table A.2.: The module has been evaluated for every participant individually and then the average and median for the results have been calculated Average Median Competence 1.43 1.4 Sensory and Imaginative Immersion 2.49 2.5 Flow 2.48 2.5 Tension/Annoyance 1 0.67 Challenge 1.33 1.3 negative Affect 0.75 0.63 positive Affect 2.57 2.55

Did you experience motion sickness while playing the game?

• Yes (2, 10%)

• No (16, 80%)

• "a bit, low resolution"

• "Nein, aber stattdessen Kreislaufbeschwerden"

Rate the user interface of the game:

• 0: Badly designed

• 1 (4, 20%)

• 2 (4, 20%)

• 3 (7, 35%)

• 4: Well designed (5, 25%)

Additional remarks for the user interface:

• "Maybe some more texts while playing with controls"

• "Sinnvoll, aber schwer für VR-Anfänger zu verstehen wo man anfangen soll, da gerade am Anfang alles sehr überladen ist in den 360 Grad"

• "Die Einheiten waren zu klein, um zu erkennen zu können auf welcher Seite sie stehen"

• "good thing with the interchangeable left and right hand"

• "I didn’t find where to adress this but i think enemy units should be highlighted in a different color when pointed at. I found the UI very intuitive"

• "Es wäre gut, wenn man nicht die ganze Zeit stehen müsste. Außderdem war die Schrift etwas klein."

60 A. User Study Questionnaire and Results

• "Anleitung, welcher Button was macht. Vorallem im Tutorial. Farbenblinden Farben benutzen"

• "Man hat oft den Überblick verloren welche Truppen man schon bewegt hat und welche nicht"

• "It needs an indicator for where to look"

• "Recht viel Text"

• "Am Anfang etwas verwirrend, aber man gewöhnt sich relativ schnell dran"

Rate the controls of the game:

• 0: Doesn’t work well (1, 5%)

• 1 (4, 20%)

• 2 (7, 35%)

• 3 (7, 35%)

• 4: Works very well (1, 5%)

Additional remarks for the controls:

• "rotating units didn’t feel so well"

• "Ziemlich schwer die Richtige Einheit auszuwählen"

• "Constantly pointing down with the controller doesn’t feel that healthy for the hand"

• "No multi unit controls, felt kinda hard to notice the current state of the unit, moving the units felt pretty limited"

• "The units rotation is hard since a lot of time many directions are blocked/rotation is inversed"

• "Manchmal schwer, die Einheiten zu bewegen ohne aus Versehen eine andere zu treffen"

• "Hard to select units, should be able to select multiple units at once"

• "confusing and little feedback"

• "Bisschen schwer manchmal die richtige Truppe auszuwählen, man müsste dafür ein wenig seine Position wechseln. Evtl. die Truppen ein wenig vergrößern. Liegt vielleicht auch am begrenzten Spielfeld"

• "would be nice if you could have moved multiple units at the same time"

• "Einige Bugs, nicht sonderlich intuitiv, Anschaulicheres Tutorial würde helfen."

61 A. User Study Questionnaire and Results

• "v. a. die Idee, das man das Spiel mit einer beliebigen Hand steuern (& diese fließend wechseln ) kann finde ich gut"

• "Unit selection is hard when they are clustered together. no attack option without moving"

• "Works well when tracking works"

Rate the tutorial of the game:

• 0: Didn’t help at all (1, 5%)

• 1 (3, 15%)

• 2 (6, 30%)

• 3 (7, 35%)

• 4: Very good (2, 10%)

• (no option selected) (1, 5%)

Additional remarks for the tutorial:

• "Not enough explanation (what is rotation for, which units)"

• "Nicht leicht sich zurechtzufinden"

• "Maybe add the ’what button does what’ (e.g. I managed to zoom in the map before knowing I even can do that)"

• "Maybe let more units be tried"

• "Maybe instead of a long text short paragraphs after each step"

• "vielleicht hab ich es nur nicht mitbekommen, aber vielleicht nützlich zu sagen, dass die Truppen automatisch angreifen, sofern in Range"

• "Ich wusste nicht was ich machen sollte"

• "Ich habe leider nicht sofort gewusst was ich machen soll"

• "Raycasting von Einheiten blockiert"

• "Ein bisschen "mehr" wäre schön gewesen (evtl. eine kleine Bsp.-Schlacht 4v4/5v5 Einheiten mit den verschiedenen Einheitentypen)

Do you prefer the Virtual Reality experience compared to the typical monitor experience in this game context?

• Yes (11, 55%)

62 A. User Study Questionnaire and Results

• No (5, 25%)

• "Gelegentlich macht es Spaß, aber nicht auf Dauer"

• "Yes, more immersive & explorative experience"

• "both are very enjoyable"

• "No, because it hurt my eyes"

• "Sometimes"

• "Standing over the battlefield is great, however I can’t imagine myself standing for hours"

• "It’s nice to actually stand in the battlefield"

How well could you follow the battles in the replay mode?

• 0: I couldn’t follow at all

• 1

• 2 (3, 15%)

• 3 (7, 35%)

• 4: I could follow easily (10, 50%)

• "it gets a bit blurry and don’t know exactly which units died"

A.4. Retention

Which side won the Battle of the Trebia?

• Rome

• Carthage (20, 100%)

What were deciding factors in the Battle of the Trebia?

• "Roman soldiers were exhausted and cold. The others were rested"

• "Hannibals überragende Taktik"

• "Das Flankieren"

• "Preparation for the cold conditions, use of stronger units (elephants against infantry), to break through and surround enemies"

• "the cold and the number advantage of carthage"

63 A. User Study Questionnaire and Results

• "the tiredness of the roman soldiers that had to cross the river, the river blocking the soldiers from retreating, the ambush that made the romans totally encircled"

• "Die Flanke hat gegen die Römische Kavallerie gewonnen, und dadurch konnte Hannibal die Römer einkesseln + der Hinterhalt"

• "Cold river behind the romans, encirclements of the army, war elephants attacking infantry"

• "strategy and positioning"

• "Kavallerie musste auf den Flanken schnell gewinnen, um die restlichen ’einzukesseln’"

• "Die Strategie, die die Karthager eingesetzt haben, um die römische Armee zu um- stellen."

• "Fluss, Kälte, die anderen waren nicht ausgeruht bzw. gestärkt"

• "Der Fluss und dass die Carthage die Römer eingekesselt haben"

• "Ein Hinterhalt und die Tatsache, dass die römischen Einheiten vom Durchqueren des Flusses geschwächt waren. Außerdem haben sie kein Mittagessen gegessen."

• "Romans got surrounded"

• "The romans had no time to eat and were cold because they went through the river"

• "Der Hinterhalt der Karthager Kavallerie und der Fluss"

• "Römer wurden umzingelt und ca. 10k flohen"

• "Romans were outnumbered, tired and were freezing. Carthagians were well prepared"

• "Romans were tired, while the other were well rested"

Name any units of the Carthaginian Army you might know:

• blank (2)

• "war elephants" (3)

• "elephants, light infantry"

• "war elephants, heavy infantry, light infantry" (2)

• "elephants, riders, infantry"

• "elephants, cavalry, light front line soldiers"

• "War elephants, light infantry, heavy infantry, cavalry" (2)

64 A. User Study Questionnaire and Results

• "I didn’t focus on their units since I played the romans; but soldiers, horseback riders/- calvary"

• "Heavy infantry"

• "Kavallerie, Kriegselefanten"

• "Reiter, Fußsoldaten, Bogenschützen"

• "Infanterie"

• "Cavalry, infantry"

• "Kriegselefanten, heavy infantry, Kavallerie"

• "war elephants, heavy armored warriors"

What did you take away from the game?

• blank (2)

• "Partly story, much fun, the will to play more fights and learn about how they were in reality"

• "Ich muss mehr Geschichte nachlernen"

• "battle tactics and knowing your Armys capabilities can decide on win or loose, more experienced units are behind the less experienced ones"

• "the romans played a game against time"

• "Hannibal has a better strategic mind than me. I learned about the composition of the roman and carthaginian armies"

• "Dass Hannibal sehr vorbereitet war und wie einfach der Kampf für ihn entschieden wurde"

• "The battle was important and a big victory for the carthargians"

• "the battle of trebia (history)"

• "Mir sind wieder viele Namen von Soldaten eingefallen (viel Rome Total War gespielt früher...)"

• "Schlachtstrategie"

• "Ein bisschen Wissen über die Schlach und erste richtige Erfahrungen mit der VR-Brille"

• "Rahmenbedingungen (Ort, Zeit) können eine Schlacht entscheiden. Der 2. Punische Krieg war etwa 220 v. Chr."

65 A. User Study Questionnaire and Results

• "Hannibal won against the Romans"

• "Hannibal was a clever dude"

• "Schlachten sind ziemlich unübersichtlich. Hannibal war ein guter und geschickter Feldherr"

• "Carthagos Strategie; Einige Details über Einheiten(strukturierungen) beider Seiten"

• "The armies were much more organized than they are shown in other games"

• "I realised, I know nothing about history"

Do you think this is a good way to learn about history? Please give reasons for your answer.

• "yes it is, if you watch the fights and you read the story behind it, you need way less effort to remember things of punic war."

• "Ja, sehr sogar! Man lernt es auf einem selber erforschenden Weg mit sehr viel mehr Spaßfaktor"

• "Ja, nur die Steuerung sollte verbessert werden, damit man nicht so leicht frustriert wird und der Tisch sollte v[ie]ll[eich]t aufrecht stehen, sodass man nicht immer runter schauen muss"

• "Yes, the spectate battle mode shows step by step what lead to the outcome. Detailed but still not endlessly long texts give a good understanding of the situation. Interactive learning is better than video/books (in that example)"

• "yes, because it felt very immersive and engaging"

• "It was a really enjoyable way to learn about how the battle took place and the spectate mode explained the events very well, but I found remembering unit names difficult since focusing on text in VR mode is a bit tiresome. I feel like this is a good way to get a general idea on events/wars and strategies"

• "Ja, weil der eigene Kampf am Anfang einen Vergleich gibt und den Spieler einbindet. Außerdem sieht es besser aus als der Text in einem Geschichtsbuch und ist einfacher zu folgen als eine Doku irgendwo im Fernsehen"

• "Yes, because it is interactive and you feel more immersed in the battle"

• "Yes, games are one of the best ways to learn history in my opinion since you take part in it (immerse yourself)"

• "Klassischen Unterricht über Geschichte finde ich besser, da man auch prinzipiell offener gegenüber Fragen ist, aber als ’Nebenliteratur’ ist das schon cool"

66 A. User Study Questionnaire and Results

• "Ich denke es ist gut, um gelerntes Wissen zu wiederholen, weil man auf diese Weise Geschichte nur von einem Standpunkt aus betrachtet. Außerdem denke ich, dass man nicht den gesamten Geschichtsstoff gut in Spiele umsetzen kann, sondern eher nur einzelne Ereignisse."

• "Ja, weil man spielerisch die Welt erforscht. Und sich das gelernte besser merken kann."

• "Ja, ich denke dass das ein guter Weg ist, da es relativ anschaulich ist und man sich somit die Schlachten besser vorstellen kann"

• "Ja, denn es weckt Interesse, statt nur mit Fakten zu langweilen"

• "It is better than just reading about it, because you also get a visual experience"

• "Yes, the biggest problem for me when it comes to learning history is always the fact that there not many images I can associate it with."

• "Ja, viel anschaulicher, als nur darüber zu lesen"

• "Ja, ich denke das dies eine sehr immersive & interessante Methode ist historische Schlachten darzustellen"

• "Yes, you can experience the history yourself and see how things happened. educational movies may be transferred into this kind of games without losing educational value"

• "For someone who likes playing games like this, I think it is. Me personally, I prefer single player RPGs like Assassin’s Creed, where you can learn something"

Figure A.1.: The average rating for UI, Controls and Tutorial, divided into players that said they played video games "all the time" and those that played less often.

67 A. User Study Questionnaire and Results

Figure A.2.: Average GEQ core module scores for players that rated either the controls or the tutorial (or both) with a 1 or less, compared to those that didn’t and the average.

Figure A.3.: The average GEQ core module scores for the four players with the best retention results compared to the rest.

68 A. User Study Questionnaire and Results

Figure A.4.: The average rating of UI, Controls, Tutorial and how well players could follow in the replay mode with regards to quality of Retention results. The average of all results is also displayed for comparison. (Best: 4 Participants, Good: 5 Participants, Rest: 11 Participants)

69 List of Figures

2.1. The Vive head-mounted display with controllers. [Pes16] ...... 3

3.1. The historical Battle of the Trebia as depicted in Rome: Total War...... 6 3.2. The serious game to explore the solar system by Detlefsen. [Det] ...... 8 3.3. Inside view of one of the buildings in Virtual Rome. [Nic] ...... 9 3.4. The VR battle simulation game developed by Steen Müller. [Mül17] ...... 10

5.1. The old color scheme of the game. No distinction in color was made between different unit types...... 19 5.2. The frame times measured during different situations in the game...... 20 5.3. The new color scheme of the game. The outer rim shows the side the unit belongs to, while the inner color signifies the type of the unit...... 25 5.4. The new color scheme of the game when viewed from further away. The different types of units and the side they belong to are clearly distinguishable. 25 5.5. The highlighted cavalry fight on the wings...... 26 5.6. The new war elephant unit as shown in the game...... 29 5.7. Three hidden units in an ambush...... 30 5.8. The main menu of the game. Behind the table, the weapons that can be wielded in first person can be seen. At the left and right side of the table are the models that can be inspected...... 32

7.1. The frame times measured during different situations in the finished game. . . 41 7.2. Average Rating with Regards to Retention ...... 44 7.3. Average GEQ Scores with Regards to Retention ...... 45 7.4. Average GEQ Scores with Regards to Controls/Tutorial Rating ...... 46 7.5. Average Rating with Regards to Video Game Experience ...... 47

9.1. The control scheme of the game. [Mül17] ...... 51 9.2. The color coding used to distinguish units. The outer rim shows which side the unit belongs to, the inner color corresponds with the unit type...... 53

A.1. Average Rating with Regards to Video Game Experience ...... 67 A.2. Average GEQ Scores with Regards to Controls/Tutorial Rating ...... 68 A.3. Average GEQ Scores with Regards to Retention ...... 68 A.4. Average Rating with Regards to Retention ...... 69

70 List of Tables

A.1. GEQ Core Module ...... 59 A.2. GEQ Core Module Results ...... 60

71 Bibliography

[Pes16] M. Pesce. HTC Vive Pre VR Consumer headset. 2016. url: https://www.flickr.com/ photos/pestoverde/26087568804/ (visited on 02/11/2019). [Jer15] J. Jerald. The VR book: Human-centered design for virtual reality. Morgan & Claypool, 2015. [Ste92] J. Steuer. “Defining virtual reality: Dimensions determining telepresence”. In: Journal of communication 42.4 (1992), pp. 73–93. [HTC] HTC. VIVETM | VIVE Virtual Reality System. url: https://www.vive.com/us/ product/vive-virtual-reality-system/ (visited on 02/08/2019). [DAJ11] D. Djaouti, J. Alvarez, and J.-P. Jessel. “Classifying Serious Games: The G/P/S Model”. In: Handbook of Research on Improving Learning and Motivation through Educational Games: Multidisciplinary Approaches. Ed. by P. Felicia. 2011. [MC05] D. R. Michael and S. L. Chen. Serious games: Games that educate, train, and inform. Muska & Lipman/Premier-Trade, 2005. [SJB07] T. Susi, M. Johannesson, and P. Backlund. Serious games: An overview. 2007. [Det+11] S. Deterding, D. Dixon, R. Khaled, and L. Nacke. “From game design elements to gamefulness: defining gamification”. In: Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments. ACM. 2011, pp. 9–15. [Juu11] J. Juul. Half-real: Video games between real rules and fictional worlds. MIT press, 2011. [Cre04] Creative Assembly. Rome: Total War. CD-ROM. 2004. [Cre13] Creative Assembly. Total War: Rome II. Download. 2013. [Man17] G. Manz. Roms Aufstieg zur Weltmacht: Das Zeitalter der Punischen Kriege. Springer Fachmedien Wiesbaden, 2017. isbn: 9783658121440. [Det] J. Detlefsen. The Cosmic Perspective: Teaching Middle-School Children Astronomy Using Ego-Centric Virtual Reality on Vimeo. url: https://vimeo.com/100696247 (visited on 12/21/2018). [Det14] J. Detlefsen. “The cosmic perspective: Teaching middle-school children astronomy using ego-centric virtual reality”. Master Thesis, Aalborg University, 2014. [Nic] M. Nicholls. Virtual Rome. url: https://research.reading.ac.uk/virtualrome/ (visited on 12/20/2018). [Fri] B. Frischer. Rome Reborn. url: https://www.romereborn.org/ (visited on 12/20/2018).

72 Bibliography

[Mül17] S. Müller. “Serious Game: Ancient Battles in VR”. Master Thesis, Technical Univer- sity Munich, 2017. [Bar98] P. A. Barceló. Hannibal. CH Beck, 1998. [Pol89] Polybius. Histories. English. Trans. Greek by E. S. Shuckburgh. London: Macmillan, 1889. url: http://www.perseus.tufts.edu/hopper/text?doc=Perseus:text: 1999.01.0234 (visited on 02/11/2019). [Hun17] P. Hunt. Hannibal. Simon and Schuster, 2017. [Sei93] J. Seibert. Hannibal. Wissenschaftliche Buchgesellschaft, 1993. [Bar12] P. A. Barceló. Keine Elefanten über die Alpen - Historiker widerspricht Hannibal- Geschichtsschreibung | Gruner + Jahr, P.M. History | Presseportal. 2012. url: https: //web.archive.org/web/20120113202145/http://www.presseportal.de/pm/ 55502/2180329/keine-elefanten-ueber-die-alpen-historiker-widerspricht- hannibal-geschichtsschreibung (visited on 02/12/2019). [BG16] D. Beeler and A. Gosalia. Asynchronous Timewarp on Oculus Rift | Oculus. 2016. url: https://developer.oculus.com/blog/asynchronous- timewarp- on- oculus- rift/ (visited on 02/12/2019). [Uni] Unity Technologies. Optimizing scripts in Unity games - Unity. url: https : / / unity3d.com/de/learn/tutorials/topics/performance-optimization/optimizing- scripts-unity-games (visited on 02/13/2019). [Mer] D. Merfield. Public-Icons - Free icons in the public domain. url: https://publicicons. lllllllllllllllll.com/ (visited on 02/07/2019). [Mys] Mysteryem. 29 grounds and walls (and water) (1024x1024) | OpenGameArt.org. url: https : / / opengameart . org / content / 29 - grounds - and - walls - and - water - 1024x1024 (visited on 02/07/2019). [Blu] BlueRatDesigns. Wall textures, tileable - 20.jpg | OpenGameArt.org. url: https : / / opengameart . org / content / wall - textures - tileable - 20jpg (visited on 02/07/2019). [Edl] M. Edlund. Whinarn/MeshDecimator: A mesh decimation library for .NET and Unity. url: https://github.com/Whinarn/MeshDecimator (visited on 02/08/2019). [IdP13] W. IJsselsteijn, Y. de Kort, and K. Poels. The Game Experience Questionnaire. English. Technische Universiteit Eindhoven, 2013. [Bar96] R. Bartle. Hearts, Clubs, Diamonds, Spades: Players Who Suit MUDs. 1996. url: http://mud.co.uk/richard/hcds.htm (visited on 02/09/2019). [Bar] M. Barr. Take the Bartle Test. url: http://matthewbarr.co.uk/bartle/ (visited on 02/09/2019).

73