sign in sign up
<<
Home , Indirect speech, Past tense

Tense and Aspect Assignment in Discourse

David K. Elson and Kathleen R. McKeown Department of Computer Science Columbia University {delson,kathy}@cs.columbia.edu

Abstract tion to create semantic encodings of short stories. To do this, they construct (- argument structures) through a graphical, menu- We describe a method for assigning English based interface, and assign them to intervals on a tense and aspect in a system that realizes sur- timeline. Figure 1 shows a session in which the face text for symbolically encoded . Our user is encoding a of Aesop. The top-right testbed is an encoding interface in which proposi- panel shows the original fable, and the left-hand tions that are attached to a timeline must be real- panel shows a graphical timeline with buttons for ized from several temporal viewpoints. This in- constructing new propositions at certain intervals. volves a mapping from a semantic encoding of The left-hand and bottom-right panels contain au- time to a set of tense/aspect permutations. The tomatically generated text of the encoded story, as encoding tool realizes each permutation to give the system understands it, from different points of a readable, precise description of the narrative so view. Users rely on these realizations to check that that users can check whether they have correctly they have assigned the formal connections cor- encoded actions and statives in the formal repre- rectly. The tenses and aspects of these sentences sentation. Our method selects tenses and aspects are a key component of this feedback. We describe for individual event intervals as well as subinter- the general purpose of the system, its data model, vals (with multiple points), quoted and and the encoding methodology in a separate paper unquoted speech (which reassign the temporal fo- (Elson and McKeown, 2010). cus), and modal events such as conditionals. The paper is organized as follows: After dis- cussing related work in Section 2, we describe our 1 Introduction method for selecting tense and aspect for single Generation systems that communicate knowledge events in Section 3. Section 4 follows with more about time must select tense and aspect carefully complex cases involving multiple events and shifts in their surface realizations. An incorrect assign- in temporal . We then discuss the results. ment can give the erroneous impression that a con- 2 Related Work tinuous has ended, or that a previous state is the current reality. In this paper, we consider There has been intense interest in the interpre- English tense and aspect in the generation of nar- tation of tense and aspect into a formal under- rative discourse, where statives and actions occur standing of the ordering and duration of events. over connected intervals. This work has been in both (Dowty, We describe two contributions: first, a general 1979; Nerbonne, 1986; Vlach, 1993) and natu- application of theories of tense, aspect and inter- ral language understanding. Early systems inves- val logic to a generation context in which we map tigated rule-based approaches to parsing the du- temporal relationships to specific tense/aspect se- rations and orderings of events from the tenses lections. Second, we describe an implementation and aspects of their (Hinrichs, 1987; Web- of this approach in an interactive environment with ber, 1987; Song and Cohen, 1988; Passonneau, a basic sentence planner and realizer. The first re- 1988). Allen (1984) and Steedman (1995) focus sult does not depend on the second. on distinguishing between achievements (when an The purpose of the system is to allow users who event culminates in a result, such as John builds are na¨ıve to linguistics and knowledge representa- a house) and processes (such as walking). More Figure 1: Screenshot of our story encoding interface. recent work has centered on markup languages teractive narrative encoding project. Callaway for complex temporal information (Mani, 2004) and Lester’s STORYBOOK (2002) aims to im- and corpus-based (statistical) models for predict- prove fluency and discourse cohesion in realiz- ing temporal relationships on unseen text (Mani et ing formally encoded narratives; Ligozat and Zock al., 2006; Lapata and Lascarides, 2006). (1992) allow users to interactively construct sen- Our annotation interface requires a fast realizer tences in various temporal scenarios through a that can be easily integrated into an interactive, on- graphical interface. line encoding tool. We found that developing a custom realizer as a module to our Java-based sys- 3 Expressing single events tem was preferable to integrating a large, general 3.1 Temporal knowledge purpose system such as KPML/Nigel (Matthiessen and Bateman, 1991) or FUF/SURGE (Elhadad The propositions that we aim to realize take the and Robin, 1996). These realizers, along with Re- form of a predicate, one or more arguments, zero alPro (Lavoie and Rambow, 1997), accept tense as or more attached modifiers (either a oper- a parameter, but do not calculate it from a semantic ator or an adverbial, which is itself a ), representation of overlapping time intervals such and an assignment in time. Each argument is asso- as ours (though the Nigel grammar can calculate ciated with a semantic role (such as Agent or Ex- tense from speech, event, and reference time or- periencer), and may include nouns (such as char- derings, discussed below). The statistically trained acters) or other propositions. In our implemented FERGUS (Chen et al., 2002) contrasts with our system, the set of predicates available to the an- rule-based approach. notator is adapted from the VerbNet (Kingsbury and Palmer, 2002) and WordNet (Fellbaum, 1998) Dorr and Gaasterland (1995) and Grote (1998) linguistic databanks. These provide both durative focus on generating temporal connectives, such as actions and statives (Dowty, 1979); we will refer before, based on the relative times and durations of to both as events as they occur over intervals. For two events; Gagnon and Lapalme (1996) focus on example, here are an action and a stative: temporal adverbials (e.g., when to insert a known time of day for an event). By comparison, we ex- walk(Mary, store, 2, 6) (1) tend our approach to cover direct/ hungry(Julia, 1,∞) (2) and the subjunctive/conditional forms, which they do not report implementing. While our work fo- The latter two arguments in (1) refer to time cuses on English, Yang and Bateman (2009) de- states in a totally ordered sequence; Mary starts scribe a recent system for generating Chinese as- walking to the store at state 2 and finishes walking pect expressions based on a time interval represen- at state 6. (2) begins at state 1, but is unbounded tation, using KPML as their surface realizer. (Julia never ceases being hungry). While this pa- Several other projects run tangential to our in- per does not address the use of reference times (such as equating a state to 6:00 or yesterday), this Diagram Relations Perspective R < E1 Before is an area of ongoing work. E 1 E 2 (1) and (2), depending on the situation, can be R realized in several aspects and tenses. We adapt R = E1 Begin E 1 E 2 R < E2 and extend Reichenbach’s (1947) famous system R of symbols for distinguishing between simple and E1 < R During E 1 E 2 progressive aspect. Reichenbach identifies three R < E2 points that define the temporal position of the R R = E2 Finish event: the event time E, the speech time S, and E 1 E 2 R > E1 a reference time R which may or may not be in- R dicated by a temporal adverbial. The total order- R > E2 After E 1 E 2 ing between these times dictates the appropriate R aspect. For example, the simple John laughed has the relation E < S. R = E because there is Table 1: Perspective assignment for viewing an no separate reference time involved. The past per- event from a reference state. fect John had laughed [by the end of the ] has the relation E < R < S, in that it describe “the formally encoded the story. In the remainder of past of the past”, with the nearer “past” being R this section, we describe our method for assigning (the end of the play). R can be seen as the tempo- tenses and aspects to propositions such as these. ral focus of the sentence. As Reichenbach does not address events with 3.2 Reference state intervals, we redefine E as the transition (E1..E2) In both snapshot and modes, we often attached to the proposition (for example, (2,6) need to render the events that occur at some ref- for Mary’s walk). This definition deliberately as- erence state R. We would like to know, for in- sumes that no event ever occurs over a single “in- stance, what is happening now, or what happened stant” of time. The perception of an instantaneous at 6:00 yesterday evening. The tense and aspect event, when it is needed, is instead created by di- depend on the perspective of the reference state lating R into an interval large enough to contain on the event, which can be bounded or unbounded. the entire event, as in Dowty (1979). The two-step process for this scenario is to deter- We also distinguish between two generation mine the correct perspective, then pick the tense modes: realizing the story as a complete discourse and aspect class that best communicates it. (narration ) and describing the content of a We define the set of possible perspec- single state or interval (snapshot mode). Our sys- tives to follow Allen (1983), who describes tem supports both modes differently. In narration seven relationships between two intervals: be- mode, we realize the story as if all events occur be- fore/after, meets/met by, overlaps/overlapped by, fore the speech time S, which is the style of most starts/started by, during/contains, finishes/finished literary fiction. (We shall see that this does not by, and equals. Not all of these map to a relation- preclude the use of the .) In snapshot ship between a single reference point and an event mode, speech time is concurrent with reference interval. Table 1 maps each possible interaction time so that the same events are realized as though between E and R to a perspective, for both they are happening “now.” The system uses this bounded and unbounded events, including the mode to allow annotators to inspect and edit what defining relationships for each interaction. A dia- occurs at any point in the story. In Figure 1, for in- mond for E1 indicates at or before, i.e., the event stance, the lion’s watching of the bull is realized as is either anteriorly unbounded (E1 = −∞) or both a present, continuing event in snapshot mode beginning at a state prior to R and E2. Similarly, (the lion continues to watch the bull) and narrated a diamond for E2 indicates at or after. as a past, continuing event (the lion was watching Once the perspective is determined, covering the bull). In both cases, we aim to precisely trans- Reichenbach’s E and R, speech time S is deter- late the propositions and their temporal relation- mined by the generation mode. Following the ships into text, even if the results are not elegant guidelines of Reichenbach and Dowty, we then as- , so that annotators can see how they have sign a tense for each perspective/speech time per- Perspective Generation mode English tense System’s construction Example After Future Speech Past had {PAST } She had walked. Present Speech has/have {PAST PARTICIPLE} She has walked. Past Speech will have {PAST PARTICIPLE} She will have walked. Modal Infinitive to have {PAST PARTICIPLE} To have walked. Finish Future Speech “Finished” stopped {PROGRESSIVE} She stopped walking. Present Speech “Finishes” stops {PROGRESSIVE} She stops walking. Past Speech “Will finish” will stop {PROGRESSIVE} She will stop walking. Modal Infinitive to stop {PROGRESSIVE} To stop walking. During Future Speech Past progressive was/were {PROGRESSIVE} She was walking. Present Speech Present pro- am/is/are {PROGRESSIVE} She is walking. gressive Past Speech Future progres- will be {PROGRESSIVE} She will be walking. sive Modal Infinitive to be {PROGRESSIVE} To be walking. During- Future Speech Past perfect had been {PROGRESSIVE} She had been walking. After progressive Present Speech Present perfect has/have been {PROGRESSIVE} She has been walking. progressive Past Speech Future perfect will have been {PROGRESSIVE} She will have been progressive walking. Modal Infinitive to has/have been {PROGRESSIVE} To have been walking. Begin Future Speech “Began” began {} She began to walk. Present Speech “Begins” begins {INFINITIVE} She begins to walk. Past Speech “Will begin” will begin {INFINITIVE} She will begin to walk. Modal Infinitive to begin {PROGRESSIVE} To begin walking. Contains Future Speech {SIMPLE PAST} She walked. Present Speech Simple present {SIMPLE PRESENT} She walks. Past speech Simple future will {INFINITIVE} She will walk. Modal Infinitive {INFINITIVE} To walk. Before Future Speech “Posterior” was/were going {INFINITIVE} She was going to walk. Present Speech Future am/is/are going {INFINITIVE} She is going to walk. Past Speech Future-of- will be going {INFINITIVE} She will be going to future walk. Modal Infinitive to be going {INFINITIVE} To be going to walk.

Table 2: Tense/aspect assignment and realizer constructions for describing an action event from a partic- ular perspective and speech time. “Progressive” means “present participle.” mutation in Table 2. Not all permutations map to Steedman, 1988; Dorr and Gaasterland, 1995). In actual English tenses. Narration mode is shown as our case, we only distinguish between actions and Future Speech, in that S is in the future with re- statives, based on information from WordNet and spect to all events in the timeline. (This is the case VerbNet. We use a separate table for statives; it is even if E is unbounded, with E2 = ∞.) Snap- similar to Table 2, except the constructions replace shot mode is realized as Present Speech, in that conjugations with insertions of be, been, be- R = S. The fourth column indicates the syntac- ing, was, were, felt, and so on (with the latter ap- tic construction with which our system realizes the plying to affective states). We do not currently permutation. Each is a sequence of tokens that are distinguish between achievements and activities in either cue words (began, stopped, etc.) or conjuga- selecting tense and aspect, except that the anno- tions of the predicate’s verb. These constructions tator is tasked with “manually” indicating a new emphasize precision over fluency. state when an event culminates in one (e.g., The house was complete). Recognizing an achieve- As we have noted, theorists have distinguished ment action can benefit lexical choice (better to between “statives” that are descriptive (John was say John finished building the house than John hungry), “achievement” actions that culminate in stopped) and content selection for the discourse as a state change (John built the house), and “activi- a whole (the house’s completion is implied by fin- ties” that are more continuous and divisible (John ished and does not need to be stated separately). read a book for an hour) (Dowty, 1979). Prior work in temporal connectives has taken advantage To continue our running examples, suppose of lexical information to determine the correct sit- propositions (1) and (2) were viewed as a snap- uation and assign aspect appropriately (Moens and shot from state R = 2. Table 1 indicates Begin Diagram Relations Perspective Diagram Relations Perspective

R1 ≥ E2 After E2 > R2 During (a priori) E E E 1 2 2 E1 = −∞ R1 R2 R2 R1 = −∞

R2 > E2 After E 1 E 2 E 2 E1 = −∞ R1 R2 R2 R1 = −∞ R1 > E1 Finish R1 > E1 Contains E 1 E 2 E E2 > R1 1 E = ∞ R1 R2 2 R2 > E2 R1 R2 = ∞ E1 > R1 Before R1 ≤ E1 Contains E 1 E 1 E 2 E2 = ∞ R2 ≥ E2 R1 R1 R2 R2 = ∞

E 1 E 2 Table 4: Perspective assignment if event and ref- R1 R2 erence intervals are unbounded in like directions. E1 < R1 During E 1 E 2 E2 > R2 R1 R2 tive of an event from a reference interval. Table R1 < E1 Begin E 1 E 2 R2 > E1 2 then applies in the same manner. In snapshot R1 R2 E2 > R2 mode, the speech time S also occurs over an inter- val (namely, R), and Present Speech is still used. E1 ≥ R2 Before E 1 E 2 In narration mode, S is assumed to be a point fol- R1 R2 lowing all event and reference intervals. In our E 1 E 2 running example, narrating the interval (1,7) re- R1 R2 sults in Mary walked to the store and Julia began to be hungry, using the Contains and Begin per- Table 3: Perspective assignment for describing an spectives respectively. event from an assigned perspective. The notion of an unbounded reference interval, while unusual, corresponds to a typical perspec- to be the perspective for (1), since E1 = R, and tive if the event is either bounded or unbounded Table 2 calls for a “new” tense/aspect permutation in the opposite direction. These scenarios are il- that means “begins at the present time.” When the lustrated in Table 3. Less intuitive are the cases appropriate construction is inserted into the over- where event and reference intervals are unbounded all for walk(Agent, Destination), which we in the same direction. Perspective assignments for derive from the VerbNet frame for walk, the result these instances are described in Table 4 and em- is Mary begins to walk to the store; similarly, (2) is phasize the bounded end of R. These situations realized as Julia is hungry via the During perspec- occur rarely in this generation context. tive. Narration mode invokes past-tense verbs. 3.4 Event Subintervals 3.3 Reference interval We do not always want to refer to events in their Just as events occur over intervals, rather than sin- entirety. We may instead wish to refer to the be- gle points, so too can reference times. One may ginning, middle or end of an event, no matter when need to express what occurred when “Julia entered it occurs with respect to the reference time. This the room” (a non-instantaneous action) or “yes- invokes a second reference point in the same inter- terday evening.” Our system allows annotators to val (Comrie, 1985, p.128), delimiting a subinter- view intervals in snapshot mode to get a sense of val. Consider John searches for his glasses versus what happens over a certain time span. John continues to search for his glasses– both in- The of reference intervals have been dicate an ongoing process, but the latter implies a studied as extensions to Reichenbach’s point ap- subinterval during which time, we are expected to proach. Dowty (1979, p.152), for example, posits know, John was already looking for his glasses. that the progressive fits only if the reference in- Our handling of subintervals falls along four terval is completely contained within the event in- alternatives that depend on the interval E1..E2, 0 0 terval. Following this, we construct an alternate the reference R and the subinterval E1..E2 of E, 0 0 lookup table (Table 3) for assigning the perspec- where E1 ≥ E1 and E2 ≤ E2. 1. During-After. If E0 is not a final subinter- reality 0 0 val of E (E2 < E2), and R = E2 or R is a E R S 0 0 speech subinterval of E that is met by E (R1 = E ), 2 E the perspective of E0 is defined as During- hunger After. In Table 2, this invokes the perfect- progressive tense. For example, viewing ex- E′ ample (1) with E0 = (2, 4) from R = 4 in hunger narration mode (Future Speech) would yield R′ E′buy Mary had been walking to the store. alternate

0 2. Start. Otherwise, if E is an initial subin- Figure 2: Schematic of a speech attaching to 0 0 0 terval of E (E1 = E1 and E2 < E2), the a alternate timeline with a hypothetical action. R perspective is defined as Start. These rows and Espeech are attachment points. are omitted from Table 2 for space reasons, but the construction for this case reassigns the perspective to that between R and E0. Our the main timeline. This is primarily used in prac- realizer reassigns the verb predicate to begin tice for modeling dialogue acts, but it can also be (or become for statives) with a plan to render used to place real events at uncertain time states its only argument, the original proposition, in in the past (e.g., the present perfect is used in a the infinitive tense. For example, narrating reference story being encoded). (2) with E0 =(1,2) from R = 3 would yield Julia had become hungry. 4.1 Reassigning Temporal Focus Ogihara (1995) describes dialogue acts involving 3. Continue. Otherwise, and similarly, if E changes in temporal focus as “double-access sen- 0 0 0 strictly contains E (E1 > E1 and E2 < E2), tences.” We now consider a method for planning we assign the perspective Continue. To real- such sentences in such a way that the refocusing ize this, we reassign the perspective to that of time (the reassignment of R into a new con- 0 between R and E , and reassign the verb text) is clear, even if it means changing tense and predicate to continue (or was still for statives) aspect mid-sentence. Suppose Mary were to de- with a plan to render its only argument, the clare that she would buy some eggs because of original proposition, in the infinitive. Julia’s hunger, but before she returned from the store, Julia filled up on snacks. If this 4. End. Otherwise, if E0 is a final subinterval is described by a later in the story, then of E (E0 > E and E0 = E ), we assign the 1 1 2 2 we need to carefully separate what is known to perspective End. To realize this, we reassign Mary at the time of her speech from what is later the perspective to that between R and E0, and known at R by the teller of the episode. Mary reassign the verb predicate to stop (or finish sees her purchase of eggs as a possible future, even for cumulative achievements). Similarly, the though it may have already happened by the point predicate’s argument is the original proposi- of retelling, and Mary does not know that Julia’s tion rendered in the infinitive. hunger is to end before long. 4 Alternate timelines and modalities Following Hornstein’s treatment of these sce- narios (Hornstein, 1990), we attach R0, the ref- This section covers more complex situations in- erence time for Mary’s statement (in an alternate volving alternate timelines– the feature of our rep- timeline), to Espeech, the event of her speaking (in resentation by which a proposition in the main the main timeline). The act of buying eggs is a 0 0 timeline can refer to a second frame of time. Other hypothetical event Ebuy that falls after R on the models of time have supported similar encapsula- alternate (modal) timeline. S is not reassigned. tions (Crouch and Pulman, 1993; Mani and Puste- Figure 2 shows both timelines for this example. jovsky, 2004). The alternate timeline can contain The main timeline is shown on top; Mary’s speech to actual events or modal events (imag- act is below. The attachment point on the main ined, obligated, desired, planned, etc.) in the past timeline is, in this case, the speech event Espeech; the future with respect to its point of attachment on the attachment point on an alternate timeline is al- ways R0. The placement of R, the main refer- will have been going to have been taking) can be ence point, is not affected by the alternate time- generated with four timelines in a chain that in- line. Real events, such as Julia’s hunger, can be voke, in order and with Past Speech, the After, Be- invoked in the alternate timeline (as drawn with a fore, After and During perspectives. There are four 0 vertical line from Ehunger to an Ehunger without Rs, all but the main one attached to a previous E. 0 an E2 known to Mary) but they must preserve their order from the main timeline. 4.2 Subjunctives and Conditionals The tense assignment for the event intervals in We finally consider tense and aspect in the case of the alternate timeline then proceeds as normal, subjunctive and conditional statements (if-thens), R0 R with substituting for . The hypothetical “buy” which can appear in alternate timelines. The re- Before event is seen in perspective, but lationship between an if clause and a then clause (Future Speech), giving the “posterior” (future-of- is not the same as the relationship between two During a-past) tense. Julia’s hunger is seen as as clauses joined by because or when. The then per Table 1. Further, we assert that connectives clause– or set of clauses– is predicated on the truth Because R such as do not alter (or in this situation, of the if clause. As linguists have noted (Horn- R0 E0 E0 ), and that the buy is connected to hunger stein, 1990, p.74), the if clause serves as an adver- with a causality edge. (Annotators can indicate bial modifier, which has the effect of moving for- connectives between events for causality, motiva- ward the reference point to the last of the if event tion and other features of narrative cohesion.) intervals (provided that the if refers to a hypotheti- The result is: Mary had said that she cal future). Consider the sentence: If John were to was going to buy eggs because Julia was hungry. fly to Tokyo, he would have booked a hotel. A cor- The subordinate clause following that sees E0 0 0 buy rect model would place Ebook before Efly on an in the future, and E0 as ongoing rather than 0 hunger alternate timeline, with Efly as the if. Since were in the past. It is appropriately ambiguous in both 0 0 to fly is a hypothetical future, R < Efly. Dur- the symbolic and rendered forms whether E0 oc- 0 0 buy ing regeneration, we set R to Efly after rendering curs at all, and if so, whether it occurs before, dur- If John were to fly to Tokyo, because we begin to ing or after R. A discourse planner would have assume that this event transpired. If R0 is left un- the responsibility of pointing out Mary’s mistaken 0 changed, it may be erroneously left before Ebook: assumption about the duration of Julia’s hunger. Then he would be going to book a hotel. We assign tense and aspect for quoted speech Our encoding interface allows users to mark one differently than for unquoted speech. Instead of or more events in an alternate timeline as if events. holding S fixed, S0 is assigned to R0 at the attach- If at least one event is marked, all if events are ren- ment point of the alternate timeline (the “present dered in the subjunctive , and the remainder time” for the speech act). If future hypothetical are rendered in the conditional. For the if clauses events are present, they invoke the Past Speech that follow R0, S0 and R0 itself are reassigned to constructions in Table 2 that have not been used the interval for each clause in turn. R0 and S0 then by either narration or snapshot mode. The content remain at the latest if interval (if it is after the origi- of the quoted speech then operates totally indepen- nal R0) for purposes of rendering the then clauses. dently of the speech action, since both R0 and S0 In our surface realizer, auxiliary words were and are detached: Mary said/says/was saying, “I am would are combined with the Modal Infinitive con- going to buy eggs because Julia is hungry.” structions in Table 2 for events during or following The focus of the sentence can be subsequently the original attachment point. reassigned to deeper nested timelines as necessary As an example, consider an alternate timeline (attaching E0 to R00, and so on). Although the with two statives whose start and end points are the above example uses subordinate clauses, we can same: Julia is hungry and Julia is unhappy. The use this nesting technique to construct compos- former is marked if. Semantically, we are saying ite tenses such as those enumerated by Halliday that hungry(Julia)→unhappy(Julia). (1976). To this end, we conjugate the Modal In- If R0 were within these intervals, the rendering finitive construction in Table 2 for each alternate would be: If Julia is hungry, then she is unhappy timeline. For instance, Halliday’s complex form (Contains/Present Speech for both clauses). If “present in past in future in past in future” (as in R0 were prior to these intervals, the rendering would be: If Julia were to be hungry, then other time representations with similar specifica- she would be unhappy. This reassigns R0 to tions. Ehungry, using were as a futurative and would The second result is a set of syntactic construc- to indicate a conditional. Because R0 and S0 are tions for realizing these permutations in our story set to Ehungry, the perspective on both clauses encoding interface. Our focus here, as we have remains Contains/Present Speech. Finally, if both noted, is not fluency, but a surface-level render- intervals are before R0, describing Julia’s previous ing that reflects the relationships (and, at times, emotional states, we avoid shifting R0 and S0 the ) present in the conceptual encod- backward: If Julia had been hungry, then she had ing. We consider variations in modality, such as been unhappy (After perspective, Future Speech an indicative reading as opposed to a conditional for both statives). or subjunctive reading, to be at the level of the re- The algorithm is the same for event intervals. alizer and not another class of tenses. Take (1) and a prior event where Mary runs out of We have run a collection project with our en- eggs: coding interface and can report success in the tool’s usability (Elson and McKeown, 2009). Two runOut(Mary, eggs, 0, 1) (3) annotators each encoded 20 into the for- Suppose they are in an alternate timeline with mal representation, with their only exposure to the attachment point 00 and (1) marked if. We be- semantic encodings being through the reference gin by realizing Mary’s walk as an if clause: If text generator (as in Figure 1). Both annotators Mary were to walk to the store. We reassign R0 became comfortable with the tool after a period to Ewalk, (2,6), which diverts the perception of of training; in surveys that they completed after (3) from Begins to After: She would have run out each task, they gave Likert-scale usability scores of eggs. Conversely, suppose the conditional re- of 4.25 and 4.30 (averaged over 20 tasks, with lationship were reversed, with (3) as the only if 5 meaning “easiest to use”). These scores are action. If the attachment point is 30, we realize (3) not specific to the generation component, but they first in the After perspective, as R0 does not shift suggest that annotators could derive satisfactory backward: If Mary had run out of eggs. The re- tenses from their semantic structures. The most mainder is rendered from the During perspective: frequently cited deficiency in the model in terms She would be walking to the store. Note that in of time was the inability to assign reference times casual conversation, we might expect a speaker at to states and intervals (such as the next morning). R = 3 to use the past simple: If Mary ran out of eggs, she would be walking to the store. In this 6 Conclusion and Future Work case, the speaker is attaching the alternate timeline It has always been the goal in surface realization at a reference interval that subsumes (3), invoking to generate sentences from a purely semantic rep- the Contains perspective by casting a net around resentation. Our approach to the generation of the past. We ask our annotators to select the best tense and aspect from temporal intervals takes us attachment point manually; automatically making closer to that goal. We have applied prior work in this choice is beyond the of this paper. linguistics and interval theory and tested our ap- 5 Discussion proach in an interactive narrative encoding tool. Our method handles reference intervals and event As we mentioned earlier, we are describing two intervals, bounded and unbounded, and extends separate methods with a modular relationship to into subintervals, modal events, conditionals, and one another. The first is an abstract mapping from direct and indirect speech where the temporal fo- a conceptual representation of time in a narrative, cus shifts. including interval and modal logic, to a set of 11 In the future, we will investigate extensions perspectives, including the 7 listed in Table 2 and to the current model, including temporal adver- the 4 introduced in Section 3.4. These 11 are bials (which explain the relationship between two crossed with three scenarios for speech time to events), reference times, habitual events, achieve- give a total of 33 tense/aspect permutations. We ments, and discourse-level issues such as prevent- also use an infinitive form for each perspective. ing as to whether adjacent sentences oc- One may take these results and map them from cur sequentially (Nerbonne, 1986; Vlach, 1993). 7 Acknowledgments Christiane Fellbaum. 1998. WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA. This material is based on research supported in part by the U.S. National Science Foundation Michel Gagnon and Guy Lapalme. 1996. From con- ceptual time to linguistic time. Computational Lin- (NSF) under IIS-0935360. Any opinions, findings guistics, 22(1):91–127. and conclusions or recommendations expressed in this material are those of the authors and do not Brigitte Grote. 1998. Representing temporal discourse necessarily reflect the views of the NSF. markers for generation purposes. In Proceedings of the Discourse Relations and Discourse Markers Workshop, pages 22–28, Montreal, Canada.

References M.A.K. Halliday. 1976. The english verbal group. In G. R. Kress, editor, Halliday: System and Function James F. Allen. 1983. Maintaining knowledge about in Language. Oxford University Press, London. temporal intervals. Communications of the ACM, 26(11):832–843. Erhard W. Hinrichs. 1987. A compositional semantics of temporal expressions in english. In Proceedings James F. Allen. 1984. Towards a general theory of of the 25th Annual Conference of the Association for action and time. Artificial Intelligence, 23(2):123– Computational Linguistics (ACL-87), Stanford, CA. 154. Norbert Hornstein. 1990. As Time Goes By: Tense and Charles Callaway and James Lester. 2002. Nar- Universal Grammar. MIT Press, Cambridge, MA. rative prose generation. Artificial Intelligence, 139(2):213–252. Paul Kingsbury and Martha Palmer. 2002. From tree- bank to propbank. In Proceedings of the Third In- John Chen, Srinivas Bangalore, Owen Rambow, and ternational Conference on Language Resources and Marilyn Walker. 2002. Towards automatic gen- Evaluation (LREC-02), Canary Islands, Spain. eration of natural language generation systems. In Proceedings of the 19th International Conference on Mirella Lapata and Alex Lascarides. 2006. Learning Computational Linguistics (COLING 2002), Taipei, sentence-internal temporal relations. Journal of Ar- Taiwan. tificial Intelligence Research, 27:85–117.

Bernard Comrie. 1985. Tense. Cambridge University Benoit Lavoie and Owen Rambow. 1997. A fast and Press. portable realizer for text generation systems. In Pro- ceedings of the Fifth Conference on Applied Natural Richard Crouch and Stephen Pulman. 1993. Time and Language Processing, Washington, DC. modality in a natural language interface to a plan- ning system. Artificial Intelligence, pages 265–304. Gerard Ligozat and Michael Zock. 1992. How to vi- sualize time, tense and aspect? In Proceedings of Bonnie J. Dorr and Terry Gaasterland. 1995. Select- the 14th International Conference on Computational ing tense, aspect, and connecting words in language Linguistics (COLING ’92), pages 475–482, Nantes, generation. In Proceedings of the Fourteenth Inter- France. national Joint Conference on Artificial Intelligence (IJCAI-95), Montreal, Canada. Inderjeet Mani and James Pustejovsky. 2004. Tem- poral discourse models for . In David R. Dowty. 1979. Word Meaning and Montague Proceedings of the ACL Workshop on Discourse An- Grammar. D. Reidel, Dordrecht. notation, Barcelona, Spain.

Michael Elhadad and Jacques Robin. 1996. An Inderjeet Mani, Marc Verhagen, Ben Wellner, overview of surge: a reusable comprehensive syn- Chong Min Lee, and James Pustejovsky. 2006. Ma- tactic realization component. In INLG ’96 Demon- chine learning of temporal relations. In Proceedings strations and Posters, pages 1–4, Brighton, UK. of COLING/ACL 2006, pages 753–760, Sydney, Eighth International Natural Language Generation Australia. Workshop. Inderjeet Mani. 2004. Recent developments in tempo- David K. Elson and Kathleen R. McKeown. 2009. A ral information extraction. In Proceedings of the In- tool for deep semantic encoding of narrative texts. ternational Conference on Recent Advances in Nat- In Proceedings of the ACL-IJCNLP 2009 Software ural Language Processing (RANLP ’03), pages 45– Demonstrations, pages 9–12, Suntec, Singapore. 60, Borovets, Bulgaria.

David K. Elson and Kathleen R. McKeown. 2010. Christian M. I. M. Matthiessen and John A. Bateman. Building a bank of semantically encoded narratives. 1991. Text generation and systemic-functional lin- In Proceedings of the Seventh International Confer- guistics: experiences from English and Japanese. ence on Language Resources and Evaluation (LREC Frances Pinter Publishers and St. Martin’s Press, 2010), Malta. London and New York. Marc Moens and Mark Steedman. 1988. Temporal on- tology and temporal reference. Computational Lin- guistics, 14(2):15–28.

John Nerbonne. 1986. Reference time and time in nar- ration. Linguistics and Philosophy, 9(1):83–95.

Toshiyuki Ogihara. 1995. Double-access sentences and reference to states. Natural Language Seman- tics, 3:177–210.

Rebecca Passonneau. 1988. A computational model of the semantics of tense and aspect. Computational Linguistics, 14(2):44–60.

Hans Reichenbach. 1947. Elements of Symbolic Logic. MacMillan, London.

Fei Song and Robin Cohen. 1988. The interpretation of temporal relations in narrative. In Proceedings of the Seventh National Conference on Artificial Intel- ligence (AAAI-88), St. Paul, Minnesota.

Mark Steedman. 1995. for tense and aspect. In The 1995 International Joint Confer- ence on AI (IJCAI-95), Montreal, Quebec, Canada.

Frank Vlach. 1993. Temporal adverbials, tenses and the perfect. Linguistics and Philosophy, 16(3):231– 283.

Bonnie Lynn Webber. 1987. The interpretation of tense in discourse. In Proceedings of the 25th An- nual Meeting of the Association for Computational Linguistics (ACL-87), pages 147–154, Stanford, CA.

Guowen Yang and John Bateman. 2009. The chinese aspect generation based on aspect selection func- tions. In Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP (ACL- IJCNLP 2009), Singapore.