Exploring the Imitation Game Using Machine Intelligence in Improvised Theatre
Total Page:16
File Type:pdf, Size:1020Kb
Proceedings of the Fourteenth Artificial Intelligence and Interactive Digital Entertainment Conference (AIIDE 2018) Improbotics: Exploring the Imitation Game Using Machine Intelligence in Improvised Theatre Kory W. Mathewson,1,2 Piotr Mirowski2 1University of Alberta Edmonton, Alberta, Canada 2HumanMachine, London, UK [email protected], [email protected] Abstract Theatrical improvisation (impro or improv) is a demanding form of live, collaborative performance. Improv is a humor- ous and playful artform built on an open-ended narrative structure which simultaneously celebrates effort and failure. It is thus an ideal test bed for the development and deploy- ment of interactive artificial intelligence (AI)-based conversa- tional agents, or artificial improvisors. This case study intro- duces an improv show experiment featuring human actors and Figure 1: Illustration of two Improbotics rehearsals. artificial improvisors. We have previously developed a deep- learning-based artificial improvisor, trained on movie subti- tles, that can generate plausible, context-based, lines of dia- logue suitable for theatre (Mathewson and Mirowski 2017b). and the audience, short- and long-term memory of narrative In this work, we have employed it to control what a subset elements, and practiced storytelling skills (Johnstone 1979). of human actors say during an improv performance. We also From an audience point of view, improvisors must express give human-generated lines to a different subset of perform- convincing raw emotions and act physically. ers. All lines are provided to actors with headphones and all We agree that improvisational computational storytelling performers are wearing headphones. This paper describes a is a grand challenge in artificial intelligence (AI) as proposed Turing test, or imitation game, taking place in a theatre, with by (Martin and others 2016). While success on the grand both the audience members and the performers left to guess challenge might be contingent on solving open-domain con- who is a human and who is a machine. In order to test scien- versational general artificial intelligence, there have been in- tific hypotheses about the perception of humans versus ma- chines we collect anonymous feedback from volunteer per- cremental scientific steps made progressing toward a unified formers and audience members. Our results suggest that re- system which can engage in improvised theatre in an open hearsal increases proficiency and possibility to control events world (Zhang and others 2018; Mathewson and Mirowski in the performance. That said, consistency with real world ex- 2017b; Guo 2018; Cappo and others 2018). While these sys- perience is limited by the interface and the mechanisms used tems do not fully understand the interaction, they can, in to perform the show. We also show that human-generated spite of (or perhaps, as an improvisor would think, thanks lines are shorter, more positive, and have less difficult words to) their imperfections, fuel the creativity of the performers. with more grammar and spelling mistakes than the artificial improvisor generated lines. 1.1 Related Work Research on computational improvisation often focuses on 1 Introduction music and dance, and on how humans interact and co- create with artificial systems (Fiebrink 2011; Hoffman and Improvisation (impro or improv) is a complex theatrical art Weinberg 2011; Thomaz and others 2016). Improvised the- form modelled on natural human interaction and demand- atre has also been a platform for digital storytelling and ing constant adaptation to an evolving context. It has been video game research for more than two decades (Perlin and defined as “real-time dynamic problem solving” (Johnson- Goldberg 1996; Hayes-Roth and Van Gent 1996). Theo- Laird 2002; Magerko and others 2009). Improv requires per- reticians and practitioners have experimented with several formers to exhibit acute listening to both verbal and non- rule- or knowledge-based methods for collaborative story- verbal suggestions coming from the other improvisors, split- telling and digital improvisation (O’Neill and others 2011; second reaction, rapid empathy towards the other performers Si and others 2005; Zhang and others 2007; Magerko and Copyright c 2018, Association for the Advancement of Artificial others 2011), and computer-aided interactive storytelling has Intelligence (www.aaai.org). All rights reserved. been explored in video game development, aiming to cre- 59 ate near-infinite narrative possibilities to drive longer-term natural language processing. The show is inspired by im- player engagement (Riedl and Stern 2006). To the best of provisation game Actor’s Nightmare (Durang 1980)–where our knowledge, our case study describes the first application one of the performers reads lines from a play and the other of deep learning-based conversational agents (Vinyals and performers seamlessly justify these otherwise incongruous Le 2015) to control and guide the improvised theatre perfor- lines while progressing a narrative. This game is modified mance of human actors. to incorporate previous work on improvised theatre along- Robotic performances have been explored previously side artificial intelligence. Specifically, this work builds on (Breazeal and others 2003). In 2000, Tom Sgorous per- the performances of (Mathewson and Mirowski 2017b), Hu- formed Judy, or What is it Like to Be A Robot? In 2010, manMachine: Artificial Intelligence Improvisation, and Etan the realistic humanoid robot Gemenoid F performed Say- Muskat’s Yes, Android3. onara, which was later turned into a movie. Incorporating This work explores wizard-of-oz style experimental meth- audience feedback into a robotic performance was reported ods that have been used extensively in previous human-robot by (Knight and others 2011). In their work, the authors used interaction studies and dialogue system research (Riek 2012; visual sensors to track audience sentiment following a line Edlund and others 2008; Fong and others 2003; Mateas delivered by the robotic performer, and used this informa- 1999). Wizard-of-Oz style interactions with artificial intel- tion to modify the next line selection based on the feed- ligence controllers have been used to provide suggestions to back received. In a similar way, and as we describe in the actors into previous artistic works 4. In these studies, hu- Methods section, a human is involved in the selection of the mans receive inputs from an external source. The source next line produced by our conversational system. In 2014, may be another human, or the machine learning system. Im- Carnegie Mellon University’s Personal Robotics Lab collab- portantly, the source is unknown to the human. This allows orated with their School of Drama to produce Sure Thing for separation between the human subjects’ outputs, and the (Zeglin and others 2014). In these performances, robots corresponding inputs. Similar to Actor’s Nightmare, the con- were precisely choreographed, deterministic, or piloted on trolled humans in Improbotics will say and justify the lines stage (Hoffman and others 2008). These shows required the they are prescribed through emotion, intonation, and physi- audience to suspend disbelief and embrace the mirage of au- cality. What sets this format apart from previous work is that tonomy. Those robot-based performances had to challenge in Improbotics the lines depend on the context of the impro- the uncanny valley—the idea that as the appearance of a vised scene. Improvisors not fed lines work to justify as the human-like robot approaches a human likeness, human re- lines are not completely congruous. These justifications aim sponses shift from empathy toward revulsion (Mori and oth- to make the scene look and feel more natural. ers 2012). Recently, toy-like humanoid robots have been in- In a way, Improbotics can be seen as a theatrical Turing volved in improvised theatre performances (Magerko and Test (Turing 1950; Mathewson and Mirowski 2017a). Can others 2009), for instance Arthur Simone’s Bot Party: Im- the performers and audience discern who is delivering lines prov Comedy with Robots1 and HumanMachine: Artificial generated by a human from those delivering lines from a Intelligence Improvisation2. Unlike those shows, our perfor- machine? We now cover methods to test this question. mance does not employ robotic avatars but sends the AI- generated dialogue to a human embodiment. 2 Methods 1.2 Motivation Improbotics is a show structure created to explore the Recent cinematic releases including Her (Jonze 2013) and grand challenge of artificial improvisation (Martin and oth- Robot & Frank (Schreier and Ford 2012) explored robots ers 2016). The show is composed of a cast of trained hu- interacting with humans naturally in day-to-day life; we in- man performers (semi-professional improvisors with at least vite live audiences to consider such interactions in a theatri- 2 years of experience). cal setting. We believe that theatre practitioners can embrace The cast is broken down into four roles: Cyborgs, Pup- AI as a new tool to explore dramatic interactions and to ex- pets, Free-will Humans, and Controllers. pand the realm of stories that artists can create. This aligns • Cyborgs are humans who take lines via headphones from with our research goal of augmenting creative abilities of hu- an AI-powered chatbot overseen by a CEO Controller; mans. To test the quality of this creative augmentation, we have developed a test-bed for theatrical co-creation which • Puppets take their lines via headphone