Towards a Crowdsourced Solution for the Authoring Bottleneck in Interactive Narratives
Total Page:16
File Type:pdf, Size:1020Kb
Heriot-Watt University Towards A Crowdsourced Solution For The Authoring Bottleneck In Interactive Narratives Michael Kriegel April 2015 Submitted for the degree of Doctor of Philosophy in Computer Science on completion of research in the Department of Mathematics, School of Mathematical and Computer Sciences. The copyright in this thesis is owned by the author. Any quotation from the thesis or use of any of the information contained in it must acknowledge this thesis as the source of the quotation or information. Abstract Interactive Storytelling research has produced a wealth of technologies that can be employed to create personalised narrative experiences, in which the audience takes a participating rather than observing role. But so far this technology has not led to the production of large scale playable interactive story experiences that realise the ambitions of the field. One main reason for this state of affairs is the difficulty of authoring interactive stories, a task that requires describing a huge amount of story building blocks in a machine friendly fashion. This is not only technically and conceptually more challenging than traditional narrative authoring but also a scalability problem. This thesis examines the authoring bottleneck through a case study and a literature survey and advocates a solution based on crowdsourcing. Prior work has already shown that combining a large number of example stories collected from crowd workers with a system that merges these contributions into a single interactive story can be an effective way to reduce the authorial burden. As a refinement of such an approach, this thesis introduces the novel concept of Crowd Task Adaptation. It argues that in order to maximise the usefulness of the collected stories, a system should dynamically and intelligently analyse the corpus of collected stories and based on this analysis modify the tasks handed out to crowd workers. Two authoring systems, ENIGMA and CROSCAT, which show two radically different approaches of using the Crowd Task Adaptation paradigm have been implemented and are described in this thesis. While ENIGMA adapts tasks through a realtime dialog between crowd workers and the system that is based on what has been learned from previously collected stories, CROSCAT modifies the backstory given to crowd workers in order to optimise the distribution of branching points in the tree structure that combines all collected stories. Two experimental studies of crowdsourced authoring are also presented. They lead to guidelines on how to employ crowdsourced authoring effectively, but more importantly the results of one of the studies demonstrate the effectiveness of the Crowd Task Adaptation approach. Acknowledgements I am deeply grateful to my supervisors Ruth Aylett and Sandy Louchart for bearing with me for all this time, teaching me so much about this topic, always having an open door to discuss problems and provide advice and for giving me many opportunities to connect to the Interactive Storytelling research community through conference visits. A big thank you to my examiners Dr. Mari¨et Theune and Dr. Yun-Heh (Jessica) Chen-Burger for showing great interest in my work and providing much appreciated feedback and a very constructive and inspiring discussion during my viva. And of course also for awarding me the PhD title. I would like to thank my colleagues and fellow students from the Computer Science Department at Heriot-Watt University for invaluable discussions and feedback. Par- ticularly Asad, Amol, Christopher, Iain, Rob, Irene, Matthias, Shahira and Patricia, all of whom I shared an office with at some time and the Storylab guys Allan, Neil and Andy all gave me many opportunities to bounce off ideas. During the e-Circus project I had the chance to be part of a multidisciplinary and multinational team doing groundbreaking work in educational IS applications that also motivated and spawned this PhD. Being part of this team was a fantastic ex- perience and I want to take this opportunity to acknowledge the many talented and genuinely nice people I had the pleasure to work with. I worked especially close with Joao, Rui and Marco of the GAIPS research group at INESC-ID, Lisbon, who taught me much about the ins and outs of FAtiMA and who I was lucky enough to be able to visit several times for collaborative work. Ivo Swartjes, formerly a PhD student and researcher at University of Twente, on his research visit to our department provided me with a wonderfully enlightening collab- oration opportunity and helped in shaping my views on Interactive Storytelling that inform this work. Alasdair Clarke and Scott Watson are both much more well versed in statistics than me and deserve my thanks for having kindly answered my questions regarding experimental design and statistical analysis. I would also like to thank all the volunteers who participated in one of the several authoring experiments that this thesis describes. Without their help this research would have been impossible to com- plete. The work on this PhD has accompanied me for the better part of a decade. This is 1 a long time and along the way life has produced many distractions, while the PhD work itself has also not been without its diversions, dead ends and desperate moments. Most importantly, I would therefore like to express my eternal gratitude to my family. This includes my parents, my sister, my two brothers-in-law and my parents-in-law who were all fantastic in encouraging me along the way. I am especially indebted to my dad who took it upon himself to proof-read this thesis and gave me much appre- ciated feedback. But my most heartfelt thanks have to go to my wonderful wife Mei Yii and daughter Alicia, whose support and understanding allowed me to find enough time to focus on writing this thesis and who patiently suffered with me through the final stretches of this work, when many evenings, weekends and holidays had to be sacrificed for this endeavour. Now, as I'm writing these words, that is almost done and I am looking forward to give them and our newest family member Sebastian the time as husband and father that they deserve. ACADEMIC REGISTRY Research Thesis Submission Name: School/PGI: Version: (i.e. First, Degree Sought Resubmission, Final) (Award and Subject area) Declaration In accordance with the appropriate regulations I hereby submit my thesis and I declare that: 1) the thesis embodies the results of my own work and has been composed by myself 2) where appropriate, I have made acknowledgement of the work of others and have made reference to work carried out in collaboration with other persons 3) the thesis is the correct version of the thesis for submission and is the same version as any electronic versions submitted*. 4) my thesis for the award referred to, deposited in the Heriot-Watt University Library, should be made available for loan or photocopying and be available via the Institutional Repository, subject to such conditions as the Librarian may require 5) I understand that as a student of the University I am required to abide by the Regulations of the University and to conform to its discipline. * Please note that it is the responsibility of the candidate to ensure that the correct version of the thesis is submitted. Signature of Date: Candidate: Submission Submitted By (name in capitals): Signature of Individual Submitting: Date Submitted: For Completion in the Student Service Centre (SSC) Received in the SSC by (name in capitals): Method of Submission (Handed in to SSC; posted through internal/external mail): E-thesis Submitted (mandatory for final theses) Signature: Date: Please note this form should bound into the submitted thesis. Updated February 2008, November 2008, February 2009, January 2011 Contents List of Figures vii 1 Introduction 1 1.1 Motivation . .1 1.1.1 Interactive Digital Storytelling . .1 1.1.2 Story Worlds and Runtime Engines . .2 1.1.3 Fully Realized IS Artefacts . .3 1.1.4 The Authoring Bottleneck . .4 1.2 Crowdsourced Authoring . .5 1.2.1 Collecting Linear Example Stories . .6 1.2.2 The Crowd Task Adaptation Hypothesis . .6 1.3 Summary Of Contributions . .7 1.4 Thesis structure . .8 I The Authoring Bottleneck 10 2 Creating FearNot! - An Authoring Case Study 11 2.1 FearNot! Overview . 12 2.1.1 FearNot Creation Stages . 13 2.2 An Overview of the FAtiMA Architecture . 15 2.2.1 Planning . 15 2.2.2 Emotions . 17 2.2.3 Reactive Behaviour . 19 2.2.4 Agents and their environment . 20 2.2.5 Personality . 21 2.3 Integrating FAtiMA into an IS Runtime Engine . 22 2.3.1 The Story Facilitator . 24 2.4 Story World Contents . 27 2.4.1 Character Configuration Content . 27 2.4.2 Dialogue Content . 28 2.4.3 Interaction Rules . 29 i CONTENTS 2.4.4 Presentational Content . 29 2.4.5 Story Structure Content . 30 2.5 FearNot! Authoring Process . 30 2.5.1 Resources . 30 2.5.2 Workflow . 31 2.5.3 Knowledge Representation Encoding . 32 2.6 Authoring Observations . 36 2.6.1 Actual Use of Planning . 36 2.6.2 Decision Making . 37 2.6.3 Use of Emotion Model . 39 2.6.4 Interactivity . 41 2.6.5 Content Reuse and Abstraction . 42 2.7 Conclusion . 43 3 Data Structures for Story Representation 44 3.1 Explicit Specification of Branches . 45 3.2 Plot-centric Story Representation . 47 3.2.1 Planning . 47 3.2.2 Case-Based Reasoning . 49 3.2.3 Narrative Theories . 50 3.2.4 User Models . 52 3.3 Character-centric Story Representation . 53 3.3.1 Agents Using STRIPS-like Planning . 53 3.3.2 Agents Using Hierarchical Task Networks .