Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18) Event Coreference Resolution: A Survey of Two Decades of Research Jing Lu and Vincent Ng Human Language Technology Research Institute University of Texas at Dallas Richardson, TX 75083-0688 fljwinnie,[email protected] Abstract Recent years have seen a gradual shift of focus from entity-based tasks to event-based tasks in informa- tion extraction research. Being a core event-based task, event coreference resolution is less studied but arguably more challenging than entity corefer- ence resolution. This paper provides an overview of the major milestones made in event coreference research since its inception two decades ago. Figure 1: The standard information extraction pipeline 1 Introduction It should be easy to see from this example that to per- Compared to entity coreference resolution, event coreference form end-to-end event coreference resolution, one has to resolution is less studied but arguably more challenging. To build an information extraction (IE) pipeline (cf. Figure 1) see its difficulty, consider the following example on within- that involves (1) extracting the entity mentions from a given document event coreference resolution, whose goal is to de- document (the entity extraction component) and determining termine which event mentions in a document refer to the same which of them are coreferent (the entity coreference compo- real-world event: nent); (2) extracting the event mentions by identifying their trigger words/phrases and determining which entity mentions Georges Cipriani fleftg a prison in Ensisheim ev1 are their arguments (the event extraction component); and (3) in northern France on parole on Wednesday. He determining which event mentions are coreferent. fdepartedgev2 the prison in a police vehicle bound for an open prison near Strasbourg. There are at least two reasons why event coreference reso- lution is potentially more challenging than entity coreference In this example, there are two event mentions, ev1 and ev2, resolution. First, as we can see in Figure 1, while an entity which are triggered by the words left and departed respec- coreference resolver has to assume as inputs the noisy out- tively. These event mentions are coreferent because they both puts of an entity extraction component, an event coreference refer to the same event of Cipriani leaving the prison. resolver has to assume as inputs the noisy outputs of a larger Intuitively, for two event mentions to be coreferent, not set of upstream components in both the entity pipeline and only should they have the same event subtype, but their ar- the event pipeline, each of which involves challenging tasks guments should be compatible. In our example, ev1 and ev2 that are far from being solved. Second, while entity mentions have the same subtype, Movement.Transport-Person, thus are by and large noun phrases, event mentions are more di- satisfying the subtype agreement constraint. As far as argu- verse syntactic objects, including not only noun phrases (e.g., ment compatibility is concerned, note that an event mention gerunds, event-denoting nouns) but also verb phrases, sen- has zero or more arguments (the event’s participants), each tences and arguably whole paragraphs, which led the com- of which plays a role. For instance, ev1 has three arguments: munity to use both verbs and nouns as event mentions. Georges Cipriani is the PERSON argument, a prison is the Despite its difficulty, event coreference resolution is the ORIGIN argument, and Wednesday is its TIME argument. ev2 fundamental technology for consolidating the textual infor- also has three arguments, He, the prison, and a police vehi- mation about an event, which is crucial for essentially all cle, serving as its PERSON,ORIGIN, and INSTRUMENT argu- high-level natural language processing (NLP) applications. ments respectively. Since the two event mentions have two For example, in IE, events and event coreference information overlapping roles (i.e., PERSON and ORIGIN) and their argu- have been used for template filling [Humphreys et al., 1997] ments are (entity-)coreferent w.r.t. each of these roles, they and automated population of knowledge bases [Ji and Grish- satisfy the argument compatibility constraint. man, 2011]. In topic detection and tracking, event corefer- 5479 Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18) Event Ar- Event Corpora guments Language Document Type WD/CD Size (Approx. number of docs) type/subtypes labeled? MUC Typed Yes EN news WD 60 (MUC6), 50 (MUC7) news, broadcast conversations, web blogs, ACE 8 types, 33 subtypes Yes EN,CN WD ACE 2005: 600 (EN), 500 (CN) and conversational telephone speech OntoNotes Untyped No EN Wall Street Journal (WSJ) news WD,CD 600 ECB/ECB+ Untyped Yes EN Google news WD,CD 1000 (ECB+) KBP 9 types, 38 subtypes Yes EN,CN,ES news, discussion forum WD 1000 (EN), 800 (CN), 400 (ES) Table 1: Comparison of the publicly available event coreference-annotated corpora. ence information is needed to identify new events in a stream MUC does not define/evaluate event coreference officially: of broadcast news stories [Allan et al., 1998]. In event-based event coreference is (implicitly) a task that needs to be per- text summarization, event coreference information has been formed as part of the scenario template filling task. In this used to measure the similarity between two events, which in filling task, one has to fill one template (consisting of the turn can be used to determine whether a sentence is salient or various event roles/attributes) for each event mentioned in a not [Li et al., 2006]. Finally, event coreference information document. Hence, given two event mentions in a document, has also been used in other applications such as question an- one has to determine whether one or two templates should be swering [Narayanan and Harabagiu, 2004] and contradiction filled by determining whether they are coreferent. detection [De Marneffe et al., 2008]. The ACE corpora are produced as part of the Automatic Our goal in this paper is to provide the AI audience with an Content Extraction (ACE) evaluations. ACE 2005, the most overview of the major milestones made in event coreference widely used version of the ACE corpora for within-document research since its inception 20 years ago. Given the gradual event coreference evaluations, includes both English and Chi- shift of focus from entity-based tasks to event-based tasks in nese documents. In ACE, an event is defined as “a specific oc- IE research in recent years, we believe that this timely survey currence of something that happens, often a change of state, will be of interest to AI researchers. involving participants” [LDC, 2005]. As in MUC, only events We conclude this section by mentioning that while the ma- belonging to certain (sub)types are annotated in ACE, but jority of work on event coreference resolution focuses on ACE covers a larger set of (sub)types than MUC. the within-document version of this task, there has also been OntoNotes is a large-scale corpus that covers entities and work on cross-document event coreference, where the goal is events that are not limited to a predefined set of entity and to determine whether two event mentions in different docu- event types [Pradhan et al., 2007]. It provides both within- ments refer to the same real-world event. and cross-document entity and event coreference annotations. However, it does not specify which mentions are entity men- 2 Corpora tions and which are event mentions, nor does it annotate all event coreference chains: a chain is annotated if and only if In this section, we present five publicly-available corpora that at least one of its event mentions is nominal. It is partially for have been widely used for training and evaluating event coref- this reason that OntoNotes is less used for event coreference erence resolvers. Table 1 compares these corpora along six evaluation than the other corpora. dimensions, including (1) whether event (sub)types are anno- The EventCorefBank (ECB) corpus1 [Bejan and tated; (2) whether event arguments are annotated; (3) whether Harabagiu, 2008] and its revised version, ECB 0.12 the documents are in English (EN), Chinese (CN), and/or [Lee et al., 2012], follow the TimeML specification, where Spanish (ES); (4) the types of documents; (5) whether the events are characterized as “situations that happen or oc- documents are annotated with within-document (WD) and/or cur”. Events can be expressed as punctual, durational, or cross-document (CD) event coreference chains; and (6) the stative predicates describing “states or circumstances in approximate size in terms of the number of documents. which something obtains or holds true” [Pustejovsky et These corpora differ in another aspect that is not covered al., 2003]. Like OntoNotes, ECB contains both within- in the table: the definition of an event. As event coreference and cross-document event coreference links that belong to chains are annotated on top of event mentions, the definition one of 43 event types. Since its focus is cross-document of events (i.e., which event mentions to annotate) is relevant event coreference, within-document links are only partially to event coreference because changing the definition could annotated. The ECB+ corpus [Cybulska and Vossen, 2014] change the event coreference links in the resulting corpus. extends ECB 0.1 by incorporating more annotated docu- Below we discuss this and other aspects of the five corpora, ments and re-annotating the existing documents with the new presenting them in chronological order so that the reader can annotation style. It also expands the definition by modeling get a better idea of how the task has evolved over the years. events as a combination of four arguments, namely action, The Message Understanding Conferences (MUCs) pro- time, location, and participants.