Meta Matters in Interactive Storytelling and Serious Gaming
Total Page:16
File Type:pdf, Size:1020Kb
METAMATTERS ININTERACTIVESTORYTELLING ANDSERIOUSGAMING APLAYONWORLDS BY JEROENLINSSEN Graduation Committee Paranymphs Chairman and Secretary: Dr. M. Bruijnes Prof.dr. P. M. G. Apers Ing. T. F. de Groot Promotor: Prof.dr. D. K. J. Heylen Assistant-promotor: Dr. M. Theune Members: Prof.dr. R. Aylett Dr. R. Bidarra Prof.dr. J. van Hilligersberg Prof.dr. T. W. C. Huibers Prof.dr. J-J. Ch. Meyer Dr. N. Szilas The research reported in this dissertation was carried out at the Human Media Interaction group of the University of Twente. Human Media Interaction The research reported in this dissertation was suppor- ted by the Dutch national program COMMIT. CTIT Ph.D. Thesis Series No. 17-432 Centre for Telematics and Information Technology P.O. Box 217, 7500 AE Enschede, The Netherlands SIKS Dissertation Series No. 2017-21 The research reported in this thesis has been carried out under the auspices of SIKS, the Dutch Research School for Information and Knowledge Systems. isbn: 978-90-365-4343-9 issn: 1381-3617 (CTIT Ph.D. Thesis Series No. 17-432) doi: 10.3990/1.9789036543439 (https://doi.org/10.3990/1.9789036543439) ⃝c 2017 Jeroen Linssen, Enschede, the Netherlands METAMATTERSININTERACTIVESTORYTELLINGANDSERIOUSGAMING ( A P L A Y O N W O R L D S) DISSERTATION to obtain the degree of doctor at the University of Twente, on the authority of the rector magnificus, prof.dr. T. T. M. Palstra, on account of the decision of the graduation committee to be publicly defended on Thursday, May 4, 2017 at 16:45 by Johannes Maria Linssen born on June 11, 1987 in Venray, the Netherlands This dissertation has been approved by: Prof.dr. D. K. J. Heylen, University of Twente, NL (supervisor) Dr. M. Theune, University of Twente, NL (co-supervisor) SUMMARY(ENGLISH) Stories, in games, novels, films, and all media alike, help us understand our world. Stories provide an escape from the real world, but they also reflect reality. They provide situations that may previously have been unimaginable, leading us to think about concepts in different ways. We learn through stories, we construct our mental model of the world through them. In this dissertation, I investigate the use of storytelling techniques in the domains of interactive storytelling and serious gaming. The field of interactive storytelling attempts to create and analyse systems that provide an interactive experience to users: the latter can influence the plot of story as it is being told. Serious gaming is concerned with the use of games for purposes other than only entertainment; in this dissertation, I investigate serious games with educational purposes. In both domains, I inspect how both human users and virtual agents (artificial, autonomous entities) can perceive stories. In Part i of this dissertation, I present my main thesis: the meta matters when interacting with interactive storytelling systems and serious games. All stories and games take place in a context: they take place in virtual worlds embedded within the real world. This concept is captured by the term magic circle. Virtual worlds are not separated from the real world; they are interconnected. Moreover, events and perceptions in the real world influence people’s behaviour in the virtual world and vice versa. My research revolves around the use of such meta-information to improve experiences with interactive storytelling systems and serious games. In Part ii of this dissertation, I delve into the field of interactive storytelling, specifically focusing on the Virtual Storyteller (vst) and its successor, the Interactive Storyteller (ist). The vst provides users with an emergent narrative: stories are generated based on the interactions between virtual agents which play characters in the story. In the ist, users can take control of one of the characters, substituting a virtual agent, and influence how the story plays out. I first investigated how the meta matters to these systems by analysing interactions between pairs of children and the ist. This analysis showed that children, apart from only enacting in-character (ic) behaviour, actively go meta to reason about their plans and discuss their experience amongst each other (out-of-character behaviour, ooc). We created a scheme that allows for the annotation of this communication to discern the types of ic and ooc behaviour. With this scheme, we could see how the children’s perspectives toward their interaction changed over time. Using the vst, we investigated how the meta can matter to its virtual agents. The generation of stories in the vst relies on the characters in the story world performing actions that advance the story. However, this does not necessarily entail that characters will take those actions. For example, if they can go about their way and never meet each other, possible conflicts can be avoided. To circumvent this problem, we outfitted the virtual agents with ooc techniques that allow them v to go meta to plan actions that result in conflicts. For example, a character can go meta to assume (ooc) that an object it is looking for is at the location of its antagonist, leading to a confrontation between the two. This way,generated stories can become more interesting. In Part iii of this dissertation, I investigate serious games and take an approach which is similarly structured to that of Part ii. We created a digital serious game called loiter which features a police officer (the player character) who has to convince a small group of loitering juveniles to cease their loitering. The overarching learning objective of this game is that players should improve their social awareness, being able to interpret how their own behaviour influences that of others and vice versa. To bring about this understanding in players, I designed feedback mechanics that are shown during the game and give insight into the thoughts of the virtual juveniles: thought bubbles and flashbacks. These mechanics allow players to briefly go meta while they are playing the game, giving them the opportunity to reflect and learn from their interaction. Secondly, I investigated how the virtual agents can further support the learning objectives of loiter. I designed mechanics that allow the virtual agents to make use of meta-information to adapt the game’s difficulty to players. Based on a player’s competence, the virtual agent can determine whether that player has reached a certain level of social awareness. If so, the virtual agent adapts its behaviour to be less cooperative so that the player will have a harder challenge to convince the virtual juvenile to stop loitering. As this change in behaviour may come across as incoherent, the game presents motivations for such a change in the form of either a thought bubble or flashback. This way, loiter can provide a difficulty level that is tailored to individual players. This dissertation concludes with Part iv, in which I summarize the above points and provide an outlook on future work. One such research direction concerns envisioned improvements to the Interactive Storyteller, most notable how such an interactive storytelling system can receive player input. Future work should also expand on the possibilities of virtual reality (vr) for serious gaming. We already took a small step in this direction by implementing loiter in a multi- modal vr environment. Here, again, the challenge is to correctly interpret players’ intentions through the available channels for input. In the end, this dissertation shows that, by enabling and stimulating both virtual agents and human users to use meta-information, interactive storytelling systems and serious games can be improved. vi SAMENVATTING(NEDERLANDS) Verhalen, in games, romans, films, en alle vergelijkbare media, helpen ons om de wereld te begrijpen. Verhalen bieden de mogelijkheid om te ontsnappen aan de echte wereld, maar reflecteren ook de realiteit. Ze verschaffen situaties die we voorheen niet hadden kunnen voorstellen, zodat we na kunnen denken over concepten op verschillende manieren. We leren dankzij verhalen, dankzij hun creëren we een mentaal model van de wereld. In dit proefschrift onderzoek ik het gebruik van verhaaltechnieken in de domeinen interactive storytelling (interactieve verhalen) en serious games (edu- catieve spellen). Het veld van interactieve verhalen poogt systemen te maken en analyseren om een interactieve beleving aan te bieden aan gebruikers: die laat- sten kunnen het plot van het verhaal beïnvloeden terwijl het verteld wordt. Seri- ous games zijn spellen met een ander doel dan louter vermaak; in dit proefschrift onderzoek ik serious games met educatieve doeleinden. In beide domeinen ana- lyseer ik hoe menselijke gebruikers en virtuele agenten (kunstmatige, autonome entiteiten) verhalen waarnemen. In Deel i van dit proefschrift presenteer ik mijn hoofdthese: de meta maakt uit tijdens het interageren met interactieve verhaalsystemen en educatieve spellen. Alle verhalen en spellen spelen zich af in een context: ze spelen zich af in virtuele werelden die ingebed zijn in de echte wereld. Dit concept is omvat in de term de magic circle. Virtuele werelden zijn niet gescheiden van de echte wereld; zij zijn onderling verbonden. Gebeurtenissen en percepties in de echte wereld beïnvloeden bovendien het gedrag van mensen in de virtuele wereld en vice versa. Mijn onderzoek draait om het gebruiken van meta-informatie met als doel belevingen met interactieve verhaalsystemen en educatieve spellen te verbeteren. In Deel ii van dit proefschrift verdiep ik me in het veld van interactieve verhalen, waarbij ik me concentreer op de Virtual Storyteller (vst) en haar opvolger, de Interactive Storyteller (ist). De vst biedt gebruikers een emergente vertelling aan: verhalen worden gegenereerd op basis van de interacties van virtuele agenten die karakters spelen in het verhaal. In de ist kunnen gebruikers de rol van een virtuele agent overnemen en een karakter spelen om op die manier te beïnvloeden hoe dat verhaal verloopt. Ik heb onderzocht wat de meta uitmaakt voor deze systemen door interacties tussen kinderparen en de ist te analyseren.