• Abstract and Figures
• Public Full-text

PLOTMACHINES: Outline-Conditioned Generation with Dynamic Plot State Tracking Hannah Rashkin1, Asli Celikyilmaz2, Yejin Choi1;3, Jianfeng Gao2 1 Paul G. Allen School of Computer Science & Engineering, University of Washington 2 Microsoft Research, Redmond, WA, USA 3 Allen Institute for Artificial Intelligence, Seattle, WA, USA fhrashkin,[email protected], faslicel,[email protected] Abstract • big bird's birthday celebration Story • cookie monster eats We propose the task of outline-conditioned Outline • roller skating rink • big birthday cake story generation: given an outline as a set of Plot dynamics i phrases that describe key characters and events ℙ = paragraph i to appear in a story, the task is to generate a Outline-conditioned Story Generation coherent narrative that is consistent with the It is Big Bird's birthday, and he goes to the roller skating rink with his friends. provided outline. This task is challenging as Back at Sesame Street, Maria and Susan take out the big 1 birthday cake and leave it on a table. the input only provides a rough sketch of the ℙ Cookie Monster sees the cake, but instead of eating it and spoiling the party, he eats a chair and other things all plot, and thus, models need to generate a story over Sesame Street. by interweaving the key points provided in the Big Bird and the other skaters return to Sesame Street outline. This requires the model to keep track and are shocked at what Cookie Monster ate, though the of the dynamic states of the latent plot, condi- cake is safe. 2 Gina and Count Von Count presents the cake to Big Bird. tioning on the input outline while generating ℙ It has 548 candles even though Big Bird is 6 years old. At the end, when Gina announces the sponsors, Cookie the full story. We present PLOTMACHINES, Monster eats them along with his cake. a neural narrative model that learns to trans- form an outline into a coherent story by track- Figure 1: An outline (input) paired with a story (output) ing the dynamic plot states. In addition, we from the Wikiplots training set. Plot elements from the enrich PLOTMACHINES with high-level dis- outline can appear and reappear non-linearly through- course structure so that the model can learn out the plot, as shown in plot dynamics graph. Com- different writing styles corresponding to dif- posing stories from an outline requires keeping track ferent parts of the narrative. Comprehensive of how outline phrases have been used while writing. experiments over three fiction and non-fiction datasets demonstrate that large-scale language models, such as GPT-2 and GROVER, despite to generate a coherent narrative that is consistent their impressive generation performance, are with the provided outline. This task is challenging not sufficient in generating coherent narratives for the given outline, and dynamic plot state as the input provides only the rough elements of 1 tracking is important for composing narratives the plot . Thus, the model needs to flesh out how with tighter, more consistent plots. these plot elements will intertwine with each other across different parts of the story. The flowchart in 1 Introduction Figure1 demonstrates an example of a latent plot arXiv:2004.14967v2 [cs.CL] 9 Oct 2020 Composing a story requires a complex planning structure: different key phrases from the outline process. First, the writer starts with a rough sketch appear and re-appear jointly throughout different of what key characters and events the story will sentences and paragraphs. Notably, the way that contain. Then, as they unfold the story, the writer outline points are interwoven needs to be deter- must keep track of the elaborate plot that weaves mined dynamically based on what’s already been together the characters and events in a coherent and composed while also staying true to the original consistent narrative. outline and overall narrative structure. We study this complex storytelling process by We present PLOTMACHINES, a novel narrative formulating it as the task of outline-conditioned transformer that simulates the outline-conditioned story generation, illustrated in Figure1. Given an outline, a set of phrases describing key char- 1Here, we define plot as the main sequence of events in acters and events to appear in a story, the task is the story. generation process described above.2 Our model 2 Outline-Conditioned Generation learns to transform an outline into a multi- paragraph story using dynamic memory blocks that keep track of the implicit plot states computed us- The Task: Our primary goal is to design a task ing the outline and the story generated thus far. We for investigating how story generation models can draw inspiration from prior work in dialogue state plan long narrative according to controllable story tracking (Thomson and Young, 2010; Lee, 2013; elements. To that end, we introduce the outline- Chao and Lane, 2019), entity tracking (Henaff et al., conditioned story generation task, which takes a 2017; Bosselut et al., 2018), and memory networks plot outline as input and produces a long, multi- (Sukhbaatar et al., 2015) for keeping track of plot paragraph story. states. We also inform our model with high-level narrative structure using discourse labels so that it In order to be flexible to multiple forms of con- can learn different styles of writing corresponding trol that might be required for different down- to different parts of the narrative (i.e. beginning, stream tasks, we envision plot outlines to be de- fined loosely as lists of an arbitrary number of middle, and end). PLOTMACHINES is, to the best of our knowledge, the first model designed to gener- un-ordered plot points that should guide a story ate multi-paragraph stories conditioned on outlines being generated. Plot points could consist of high- and can be trained end-to-end to learn the latent level concepts, low-level events, or even detailed plot patterns without explicit plot annotations for sentences. For practical reasons, in this work, we supervision. limit the scope of plot points to events and phrases since these can be automatically extracted. Future To support research on outline-conditioned gen- work could explore alternate methods of defining eration, we present three datasets, including both plot outlines, perhaps using an event-based plan- fiction and non-fiction domains, where multi- ning systems (Porteous and Cavazza, 2009; Riedl, paragraph narratives from existing datasets are 2009; Riedl and Young, 2010; Fan et al., 2019) for paired with automatically constructed outlines us- generating key points. ing state-of-the-art key phrase extraction. Impor- tantly, our task formulation of outline-conditioned More concretely, in this paper, we formulate the generation is general and can be applied to various outline as a list of un-ordered bullet points which forms of grounded language generation. Compre- reflect key phrases to be loosely integrated in the hensive experiments on these datasets demonstrate output narrative. These plot outlines are inspired, that recently introduced state-of-the-art large-scale in part, by previous work in short-form story gen- language models such as GPT-2 and GROVER eration tasks that conditioned on storylines (Peng (Radford et al., 2019; Zellers et al., 2019), despite et al., 2018; Yao et al., 2019), which were defined their impressive generation performance, still strug- as an ordered list of exactly five single-word points. gle to generate coherent narratives that are consis- We extend this concept to long-form story gener- tent with input outlines. Our experiments indicate ation by defining a plot outline more flexibly as: that dynamic plot state tracking is important for an un-ordered list of an arbitrary number of multi- constructing narratives with tighter and more con- word plot elements. An outline also differs from a sistent plots compared to competitive baselines. writing prompt, such as those found in other con- Our main contributions are: (1) a new task for- trollable writing tasks (Fan et al., 2018), which are mulation of outline-conditioned story generation, more abstract and often just a starting point for (2) the presentation of three new datasets for this a story. Unlike a prompt, an outline is a list of task, (3) PLOTMACHINES, a novel narrative trans- concrete points that must appear somewhere in the former that learns to transform outlines to full narrative. stories with dynamic plot state tracking, and (4) empirical results demonstrating the limitations of One challenge of this task is to create stories state-of-the-art large-scale language models and that have appropriate discourse and narrative flow. A second challenge is for stories to include the the advantage of PLOTMACHINES compared to competitive baselines. outline in a natural way. For example, it may be appropriate for certain outline points to be used 2code available at https://github.com/ only later on in the story (e.g. the protagonist dying hrashkin/plotmachines may be more typically used at the end). Wikiplots Outline: • the rocky horror picture show • convention attendees includes servants (...) # stories : 130k Story: A criminologist narrates the tale of the newly engaged couple, Brad Majors and Janet avg # pars : 3.1 Weiss, who find themselves lost and with a flat tire on a cold and rainy late November evening, data-split : 90/5/5 somewhere near Denton in 1974 (...) WritingPrompts Outline: • found something protruding • geometric shapes glowing • sister kneel- # stories : 300k ing beside • dead bodies everywhere • darkness overwhelmed • firelight flickering (...) avg # pars : 5.9 Story: It was dark and Levi was pretty sure he was lying on his back . There was firelight flickering data-split : 90/5/5 off of what was left of a ceiling . He could hear something but it was muffled .

Load more

  22

Copy Link