Introduction General methodology Mention detection ML-based coreference resolution References

Coreference Resolution

Jordi Turmo TALP Research Center [email protected]

2014

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology General Goal Mention detection Types of coreference ML-based coreference resolution Identity noun phrase coreference References

Introduction General Goal Types of coreference Identity noun phrase coreference

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology General Goal Mention detection Types of coreference ML-based coreference resolution Identity noun phrase coreference References The goal of coreference resolution

Determining which mentions in a discourse refer to the same real world entity, property or situation. Example:

FC president Joan Laporta has warned Chelsea off star strike .

This warning has generated dicouragement in Chelsea.

Aware of Chelsea owner Roman Abramovich’s interest in the young Argentine, Laporta said last night: ” I

will answer as always, Messi is not for sale and we do not want to let him go.”

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology General Goal Mention detection Types of coreference ML-based coreference resolution Identity noun phrase coreference References The goal of coreference resolution

Determining which mentions in a discourse refer to the same real-world entity, property or situation. Example:

FC Barcelona president Joan Laporta has warned Chelsea off star strike Lionel Messi.

This warning has generated dicouragement in Chelsea.

Aware of Chelsea owner Roman Abramovich’s interest in the young Argentine, Laporta said last night: ” I

will answer as always, Messi is not for sale and we do not want to let him go.”

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology General Goal Mention detection Types of coreference ML-based coreference resolution Identity noun phrase coreference References The goal of coreference resolution

Determining which mentions in a discourse refer to the same real world entity, property or situation. Example:

FC Barcelona president Joan Laporta has warned Chelsea off star strike Lionel Messi.

This warning has generated dicouragement in Chelsea.

Aware of Chelsea owner Roman Abramovich’s interest in the young Argentine, Laporta said last night: ” I

will answer as always, Messi is not for sale and we do not want to let him go.”

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology General Goal Mention detection Types of coreference ML-based coreference resolution Identity noun phrase coreference References

Introduction General Goal Types of coreference Identity noun phrase coreference

Jordi Turmo TALP Research Center [email protected] Coreference Resolution I Cataphora (endophora):

I We do not want to let [him]1 go. [Messi]1 is not for sale. I Exophora:

I Smoking is forbidden [here]1. I [That chair]1 is broken.

Introduction General methodology General Goal Mention detection Types of coreference ML-based coreference resolution Identity noun phrase coreference References Types of coreference (positional viewpoint)

Given two mentions,

I Anaphora (endophora):

I [Messi]1 is not for sale. We do not want to let [him]1 go. I [Laporta warned Chelsea off Messi]1.[This warning]1 generated discouragement in Chelsea. I [The car]1 hit a tree. [The vehicle]1 was found one day later.

Jordi Turmo TALP Research Center [email protected] Coreference Resolution I Exophora:

I Smoking is forbidden [here]1. I [That chair]1 is broken.

Introduction General methodology General Goal Mention detection Types of coreference ML-based coreference resolution Identity noun phrase coreference References Types of coreference (positional viewpoint)

Given two mentions,

I Anaphora (endophora):

I [Messi]1 is not for sale. We do not want to let [him]1 go. I [Laporta warned Chelsea off Messi]1.[This warning]1 generated discouragement in Chelsea. I [The car]1 hit a tree. [The vehicle]1 was found one day later. I Cataphora (endophora):

I We do not want to let [him]1 go. [Messi]1 is not for sale.

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology General Goal Mention detection Types of coreference ML-based coreference resolution Identity noun phrase coreference References Types of coreference (positional viewpoint)

Given two mentions,

I Anaphora (endophora):

I [Messi]1 is not for sale. We do not want to let [him]1 go. I [Laporta warned Chelsea off Messi]1.[This warning]1 generated discouragement in Chelsea. I [The car]1 hit a tree. [The vehicle]1 was found one day later. I Cataphora (endophora):

I We do not want to let [him]1 go. [Messi]1 is not for sale. I Exophora:

I Smoking is forbidden [here]1. I [That chair]1 is broken.

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology General Goal Mention detection Types of coreference ML-based coreference resolution Identity noun phrase coreference References Coreference vs. anaphora: controversy

I Bound variable (semanticists): anaphoric mentions in which no particular real world entity is involved. They are not coreferent mentions.

I [Every dog]1 has [its]1 day. I Non-identity coreference relation: anaphoric coreferring mentions involving different entities (meronymy/holonymy)

I The boy entered [the room]1. The [door]1 closed automatically.

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology General Goal Mention detection Types of coreference ML-based coreference resolution Identity noun phrase coreference References Identity noun phrase coreference

I Determining which mentions in a discourse refer to the same real-world entity.

I A mention is an expression which refers to an entity. A noun phrase.

I An entity or coreference chain is the group of mentions that refer to the same real-word entity

I Most commonly investigated kind of coreference relation.

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology General Goal Mention detection Types of coreference ML-based coreference resolution Identity noun phrase coreference References Identity noun phrase coreference

I Included examples:

I [Messi]1 is not for sale. We do not want to let [him]1 go. I [The car]1 hit a tree. [The vehicle]1 was found one day later. I [Bruce Springsteen]1 will play in Barcelona. [The Boss]1 is well liked in that place. I We do not want to let [him]1 go. [Messi]1 is not for sale. I Excluded examples:

I [Every dog]1 has [its]1 day. I The boy entered [the room]1. The [door]1 closed automatically. I Smoking is forbidden [here]1.

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology General Goal Mention detection Types of coreference ML-based coreference resolution Identity noun phrase coreference References Identity noun phrase coreference

I Included examples:

I [Messi]1 is not for sale. We do not want to let [him]1 go. I [The car]1 hit a tree. [The vehicle]1 was found one day later. I [Bruce Springsteen]1 will play in Barcelona. [The Boss]1 is well liked in that place. I We do not want to let [him]1 go. [Messi]1 is not for sale.

WE WILL FOCUS ON THEM!! For simplicity, from now, we will refer to identity noun phrase coreference resolution simply as coreference resolution.

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Preprocessing:

I Mention detection:

I Detects the boundaries of the mentions in the input text. I system mentions vs true mentions I m = (m1, m2,..., mn) ordered as found in the document.

Introduction General methodology Mention detection ML-based coreference resolution References General methodology of a coreference solver

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention detection ML-based coreference resolution References General methodology of a coreference solver

Preprocessing:

I Mention detection:

I Detects the boundaries of the mentions in the input text. I system mentions vs true mentions I m = (m1, m2,..., mn) ordered as found in the document.

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention detection ML-based coreference resolution References General methodology of a coreference solver

Coreference resolution:

I find the coreference chains. I Heuristic-driven approaches: based on the centering theory of the discourse [Grosz et al., 83, 95]. See details in [Walker et al., 98]. I ML-based approaches: WE WILL FOCUS ON THEM!

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention detection ML-based coreference resolution References

Mention detection

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention detection ML-based coreference resolution References Mention detection

I Preprocess: POS-tagging, NERC and parsing. I Recursiverly visiting the parse tree, accept the following as mention

I Pronouns (filter out pleonastic pronouns, e.g., It is raining) I Proper names I Maximal noun phrase (NP) projections. I Coordinated NPs

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention detection ML-based coreference resolution References Mention detection

Examples of maximal NP projections:

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention detection ML-based coreference resolution References Mention detection

Examples of maximal NP projections:

keep the pronoun

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention detection ML-based coreference resolution References Mention detection

Examples of maximal NP projections:

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention detection ML-based coreference resolution References Mention detection

Examples of maximal NP projections:

drop out NPs sharing the same head

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention detection ML-based coreference resolution References Mention detection

Examples of maximal NP projections:

subordinate clause

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention detection ML-based coreference resolution References Mention detection

Examples of maximal NP projections:

subordinate clause

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References

ML-based coreference resolution Mention-Pair model Entity-Mention model Ranking models

Jordi Turmo TALP Research Center [email protected] Coreference Resolution - Decision Trees [McCarthy & Lehnert, 95], [Soon et al., 01] - Rule induction (RIPPER) [Ng & Cardie, 02] - Maximum Entropy [Denis & Baldrige, 07], [Ji et al., 05] - SVMs [Yang et al., 06]

- Closest-first strategy [Soon et al., 01] - Best-first strategy [Ng & Cardie, 02][Bengtson & Roth, 08] - Clustering [Klenner & Ailloud 2008]... - Global optimization (ILP) [Klenner, 07][Finkel & Manning, 08] - Graph partitioning [McCallum & Wellner,05][Nicolae & Nicolae, 06][Sapena et al, 10]...

I Two steps:

I Learn a classifier of mention pairs. Ex:

I Generate chains. Ex:

Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Mention-Pair model

I Examples: (mi , mj ) classified as CO/NC.

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Mention-Pair model

I Examples: (mi , mj ) classified as CO/NC. I Two steps:

I Learn a classifier of mention pairs. Ex: - Decision Trees [McCarthy & Lehnert, 95], [Soon et al., 01] - Rule induction (RIPPER) [Ng & Cardie, 02] - Maximum Entropy [Denis & Baldrige, 07], [Ji et al., 05] - SVMs [Yang et al., 06]

I Generate chains. Ex: - Closest-first strategy [Soon et al., 01] - Best-first strategy [Ng & Cardie, 02][Bengtson & Roth, 08] - Clustering [Klenner & Ailloud 2008]... - Global optimization (ILP) [Klenner, 07][Finkel & Manning, 08] - Graph partitioning [McCallum & Wellner,05][Nicolae & Nicolae, 06][Sapena et al, 10]...

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Mention-Pair model

I Examples: (mi , mj ) classified as CO/NC. I Two steps:

I Learn a classifier of mention pairs. Ex: - Decision Trees [McCarthy & Lehnert, 95], [Soon et al., 01] - Rule induction (RIPPER) [Ng & Cardie, 02] - Maximum Entropy [Denis & Baldrige, 07], [Ji et al., 05] - SVMs [Yang et al., 06]

I Generate chains. Ex: - Closest-first strategy [Soon et al., 01] - Best-first strategy [Ng & Cardie, 02][Bengtson & Roth, 08] - Clustering [Klenner & Ailloud 2008]... - Global optimization (ILP) [Klenner, 07][Finkel & Manning, 08] - Graph partitioning [McCallum & Wellner,05][Nicolae & Nicolae, 06][Sapena et al, 10]...

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Mention-Pair model

I Examples: (mi , mj ) classified as CO/NC. I Two steps:

I Learn a classifier of mention pairs. Ex: - Decision Trees [McCarthy & Lehnert, 95], [Soon et al., 01] - Rule induction (RIPPER) [Ng & Cardie, 02] - Maximum Entropy [Denis & Baldrige, 07], [Ji et al., 05] - SVMs [Yang et al., 06]

I Generate chains. Ex: - Closest-first strategy [Soon et al., 01] - Best-first strategy [Ng & Cardie, 02][Bengtson & Roth, 08] - Clustering [Klenner & Ailloud 2008]... - Global optimization (ILP) [Klenner, 07][Finkel & Manning, 08] - Graph partitioning [McCallum & Wellner,05][Nicolae & Nicolae, 06][Sapena et al, 10]...

Jordi Turmo TALP Research Center [email protected] Coreference Resolution I the classifier is biased to select the closest antecedent I problem: mi is a pronoun I [Ng & Cardie,02] Non-pronominal anaphoric mention mj and closest preceding non-pronominal antecedent mi =⇒ (mi , mj ) = CO Filtering training instances

I [Strube et al, 02][Yang et al, 03] if (mi , mj ) violates gender and number agreement.

Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Learn a classifier

Creating training examples

I [Soon et al, 01] Anaphoric mention mj and closest preceding antecedent mi =⇒ (mi , mj ) = CO =⇒ ∀k : i < k < j :(mk , mj ) = NC

Jordi Turmo TALP Research Center [email protected] Coreference Resolution I [Ng & Cardie,02] Non-pronominal anaphoric mention mj and closest preceding non-pronominal antecedent mi =⇒ (mi , mj ) = CO Filtering training instances

I [Strube et al, 02][Yang et al, 03] if (mi , mj ) violates gender and number agreement.

Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Learn a classifier

Creating training examples

I [Soon et al, 01] Anaphoric mention mj and closest preceding antecedent mi =⇒ (mi , mj ) = CO =⇒ ∀k : i < k < j :(mk , mj ) = NC I the classifier is biased to select the closest antecedent I problem: mi is a pronoun

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Filtering training instances

I [Strube et al, 02][Yang et al, 03] if (mi , mj ) violates gender and number agreement.

Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Learn a classifier

Creating training examples

I [Soon et al, 01] Anaphoric mention mj and closest preceding antecedent mi =⇒ (mi , mj ) = CO =⇒ ∀k : i < k < j :(mk , mj ) = NC I the classifier is biased to select the closest antecedent I problem: mi is a pronoun I [Ng & Cardie,02] Non-pronominal anaphoric mention mj and closest preceding non-pronominal antecedent mi =⇒ (mi , mj ) = CO

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Learn a classifier

Creating training examples

I [Soon et al, 01] Anaphoric mention mj and closest preceding antecedent mi =⇒ (mi , mj ) = CO =⇒ ∀k : i < k < j :(mk , mj ) = NC I the classifier is biased to select the closest antecedent I problem: mi is a pronoun I [Ng & Cardie,02] Non-pronominal anaphoric mention mj and closest preceding non-pronominal antecedent mi =⇒ (mi , mj ) = CO Filtering training instances

I [Strube et al, 02][Yang et al, 03] if (mi , mj ) violates gender and number agreement.

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Learn a classifier Mentions characterization (example of feature functions)

Type Feature Description Structural DIST SEN k distance in sentences is k: y,n DIST SEN >2 distance in sentences greater than 2: y,n DIST MEN k distance in mentions is k: y,n DIST MEN >2 distance in mentions greater than 2: y,n APPOSITIVE One mention in apposition with the other: y,n Lexical STR MATCH String matching: y,n ALIAS One mention is an alias of the other: y,n,u Morphological NUMBER The number of both mentions match: y,n,u GENDER The gender of both mentions match: y,n,u Syntactic DEF NP mj is a definitive NP: y,n DEM NP mj is a demonstrative NP: y,n Semantical SEMCLASS Semantic class match: y,n,u ANIMACY Animacy of both mentions match: y,n

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Learn a classifier Mentions characterization (example of feature functions)

Type Feature Description Structural DIST SEN k distance in sentences is k: y,n DIST SEN >2 distance in sentences greater than 2: y,n DIST MEN k distance in mentions is k: y,n DIST MEN >2 distance in mentions greater than 2: y,n APPOSITIVE One mention in apposition with the other: y,n Lexical STR MATCH String matching: y,n ALIAS One mention is an alias of the other: y,n,u Morphological NUMBER The number of both mentions match: y,n,u GENDER The gender of both mentions match: y,n,u Syntactic DEF NP mj is a definitive NP: y,n DEM NP mj is a demonstrative NP: y,n Semantical SEMCLASS Semantic class match: y,n,u ANIMACY Animacy of both mentions match: y,n

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Learn a classifier Mentions characterization (example of feature functions)

Type Feature Description Structural DIST SEN k distance in sentences is k: y,n DIST SEN >2 distance in sentences greater than 2: y,n DIST MEN k distance in mentions is k: y,n DIST MEN >2 distance in mentions greater than 2: y,n APPOSITIVE One mention in apposition with the other: y,n Lexical STR MATCH String matching: y,n ALIAS One mention is an alias of the other: y,n,u Morphological NUMBER The number of both mentions match: y,n,u GENDER The gender of both mentions match: y,n,u Syntactic DEF NP mj is a definitive NP: y,n DEM NP mj is a demonstrative NP: y,n Semantical SEMCLASS Semantic class match: y,n,u ANIMACY Animacy of both mentions match: y,n

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Learn a classifier Mentions characterization (example of feature functions)

Type Feature Description Structural DIST SEN k distance in sentences is k: y,n DIST SEN >2 distance in sentences greater than 2: y,n DIST MEN k distance in mentions is k: y,n DIST MEN >2 distance in mentions greater than 2: y,n APPOSITIVE One mention in apposition with the other: y,n Lexical STR MATCH String matching: y,n ALIAS One mention is an alias of the other: y,n,u Morphological NUMBER The number of both mentions match: y,n,u GENDER The gender of both mentions match: y,n,u Syntactic DEF NP mj is a definitive NP: y,n DEM NP mj is a demonstrative NP: y,n Semantical SEMCLASS Semantic class match: y,n,u ANIMACY Animacy of both mentions match: y,n

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Learn a classifier Mentions characterization (example of feature functions)

Type Feature Description Structural DIST SEN k distance in sentences is k: y,n DIST SEN >2 distance in sentences greater than 2: y,n DIST MEN k distance in mentions is k: y,n DIST MEN >2 distance in mentions greater than 2: y,n APPOSITIVE One mention in apposition with the other: y,n Lexical STR MATCH String matching: y,n ALIAS One mention is an alias of the other: y,n,u Morphological NUMBER The number of both mentions match: y,n,u GENDER The gender of both mentions match: y,n,u Syntactic DEF NP mj is a definitive NP: y,n DEM NP mj is a demonstrative NP: y,n Semantical SEMCLASS Semantic class match: y,n,u ANIMACY Animacy of both mentions match: y,n

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Learn a classifier Mentions characterization (example of feature functions)

Type Feature Description Structural DIST SEN k distance in sentences is k: y,n DIST SEN >2 distance in sentences greater than 2: y,n DIST MEN k distance in mentions is k: y,n DIST MEN >2 distance in mentions greater than 2: y,n APPOSITIVE One mention in apposition with the other: y,n Lexical STR MATCH String matching: y,n ALIAS One mention is an alias of the other: y,n,u Morphological NUMBER The number of both mentions match: y,n,u GENDER The gender of both mentions match: y,n,u Syntactic DEF NP mj is a definitive NP: y,n DEM NP mj is a demonstrative NP: y,n Semantical SEMCLASS Semantic class match: y,n,u ANIMACY Animacy of both mentions match: y,n

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Learn a classifier

Dataset for training 2 > 0 SEN MEN NP NP MATCH

.

Pair DIST . DIST APPOSITIVE STR ALIAS NUMBER GENDER DEF DEM SEMCLASS ANIMACY Corefer?

m1, m2 y n n n u n y n n y y N m1, m3 y n n n n u n n n n n N m1, m4 n n n y y y u n n y y Y m1, m3 n y n n n y n n n y y N . . mi , mj n y n n n y y n n y y Y

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Learn a classifier

Decision Tree [McCarthy & Lehnert, 95], [Soon et al., 01] Ex:

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Learn a classifier

Maximum Entropy [Denis & Baldrige, 07], [Ji et al., 05] Ex: P exp λk fk (xi,j , CO) P(CO|x) = i,j,k Z(x) Ex:  1 APPOSITIVE(x ) and corefer(x ) f (x , CO) = i,j i,j s i,j 0 otherwise

Maximum likelihood estimation of parameters λi (e.g. Improved Iterative scaling [Della Pietra et al., 96])

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

Closest-first strategy [Soon et al., 01]

if a probabilistic classifier is used then define a threshold above which a pair is considered coreferent (i.e, +pairs)

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

Closest-first strategy [Soon et al., 01]

for a given mj , select as antecedent the closest preceding mk from the +pairs

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

Best-first strategy [Ng & Cardie, 02][Bengtson & Roth, 08]

aims to improve the Precision of closest-first clustering by taking profit of the probabilities of the +pairs

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

Best-first strategy [Ng & Cardie, 02][Bengtson & Roth, 08]

for a given mj , select as antecedent the most probable precedent mk from the +pairs

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

Drawback due to the use of closest-first or best-first strategies:

I They only take profit of individual +pairs decisions of the mention-pair classifier

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

Drawback due to the use of closest-first or best-first strategies:

I They only take profit of individual +pairs decisions of the mention-pair classifier

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Approaches based on:

I Clustering [Klenner & Ailloud 2008]...

I Global optimization (ILP) [Klenner, 07][Finkel & Manning, 08]

I Graph partitioning algorithms [McCallum & Wellner,05][Nicolae & Nicolae, 06][Sapena et al, 10]...

Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

possible solution: take profit of groups of decisions of the mention-pair classifier.

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

possible solution: take profit of groups of decisions of the mention-pair classifier.

Approaches based on:

I Clustering [Klenner & Ailloud 2008]...

I Global optimization (ILP) [Klenner, 07][Finkel & Manning, 08]

I Graph partitioning algorithms [McCallum & Wellner,05][Nicolae & Nicolae, 06][Sapena et al, 10]...

Jordi Turmo TALP Research Center [email protected] Coreference Resolution I Find the most appropriated partition P in order to isolate the groups that represent independent entities. P can be learned from training data

Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

Graph-partitioning algorithms

I Ex: Alice Smith ... A. Smith ... He ... She ...

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

Graph-partitioning algorithms

I Ex: Alice Smith ... A. Smith ... He ... She ...

I Find the most appropriated partition P in order to isolate the groups that represent independent entities. P can be learned from training data

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

Graph-partitioning algorithms Ex: Alice Smith ... A. Smith ... He ... She ...

I Find the most appropriated partition P in order to isolate the groups that represent independent entities.

I P can be learned from training data

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

Graph-partitioning algorithms Ex: Constraint relaxation [Sapena et al., 10] A more flexible approach

I each mentions mi is a vertex in the graph

I each pair of mentions (mi , mj ) is connected by an edge eij I each edge eij is weighted by wij X wij = λk

k∈Cij

Cij : set of constraints that restrict the compatibility between mi and mj

λk : weight associated to the constraint k

λk and wij can be negative

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

Graph-partitioning algorithms Ex: Constraint relaxation [Sapena et al., 10] A more flexible approach

I each mentions mi is a vertex in the graph

I each pair of mentions (mi , mj ) is connected by an edge eij I each edge eij is weighted by wij X wij = λk

k∈Cij

Cij : set of constraints that restrict the compatibility between mi and mj

λk : weight associated to the constraint k

λk and wij can be negative

Jordi Turmo TALP Research Center [email protected] Coreference Resolution I λk = Pk − α

Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

Graph-partitioning algorithms Ex: Constraint relaxation [Sapena et al., 10]

I Cij : A Decision Tree (DT) is learned from mention pairs. Each rule in the DT is a constraint. Cij is the set of constraints satisfied between mi and mj .

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

Graph-partitioning algorithms Ex: Constraint relaxation [Sapena et al., 10]

I Cij : A Decision Tree (DT) is learned from mention pairs. Each rule in the DT is a constraint. Cij is the set of constraints satisfied between mi and mj .

I λk = Pk − α

Jordi Turmo TALP Research Center [email protected] Coreference Resolution i P i h = argmax i h Sil l hl l∈|Li | l S = P w xhi il eij ij l

Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

Graph-partitioning algorithms Ex: Constraint relaxation [Sapena et al., 10]

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

Graph-partitioning algorithms Ex: Constraint relaxation [Sapena et al., 10]

i P i h = argmax i h Sil l hl l∈|Li | l S = P w xhi il eij ij l

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

Graph-partitioning algorithms Ex: Constraint relaxation [Sapena et al., 10]

H = H(0); repeat for each mi do for each l ∈ |Li | do P i Sil = wij xh eij l end Normalize all Sil to [−1, 1]; for each l ∈ |Li | do i i P k h (t + 1) = (h (t)x(1 + Sil ))/( h (t)x(1 + Sik )) l l k∈|Li | l end end until H(t − 1) ≈ H(t);

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

Graph-partitioning algorithms Ex: Constraint relaxation [Sapena et al., 10]

H = H(0); repeat for each mi do for each l ∈ |Li | do P i Sil = wij xh eij l end Normalize all Sil to [−1, 1]; for each l ∈ |Li | do i i P k h (t + 1) = (h (t)x(1 + Sil ))/( h (t)x(1 + Sik )) l l k∈|Li | l end end until H(t − 1) ≈ H(t);

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Generate chains

Graph-partitioning algorithms Ex: Constraint relaxation [Sapena et al., 10]

I H(0): the probability of mi being in a new partition is double than the probabilities for the rest. (realistic situation: majority of mentions are singletons)

hi = 1 ∀ hi = 2 l |Li |+1 l∈[0,|Li |−1] |Li | |Li |+1

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Contradictions in classification.

I Generating chains by graph partitioning, ILP or clustering methods adress this problem within the mention-pair model paradigm.

I Entity-mention model and ranking models are different perspectives to deal with the problem.

Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Drawbacks of the Mention-Pair model

Lack of information.

Jordi Turmo TALP Research Center [email protected] Coreference Resolution I Generating chains by graph partitioning, ILP or clustering methods adress this problem within the mention-pair model paradigm.

I Entity-mention model and ranking models are different perspectives to deal with the problem.

Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Drawbacks of the Mention-Pair model

Lack of information. Contradictions in classification.

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Drawbacks of the Mention-Pair model

Lack of information. Contradictions in classification.

I Generating chains by graph partitioning, ILP or clustering methods adress this problem within the mention-pair model paradigm.

I Entity-mention model and ranking models are different perspectives to deal with the problem.

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Entity-Mention model

Entities characterization

I Feature functions used for the Mention-Pair models

I New feature functions implying mention groups

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Entity-Mention model

Entities characterization

I Feature functions used for the Mention-Pair models

I New feature functions implying mention groups

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Pros: improved expressiveness and results for some approaches

Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Entity-Mention model

Examples: Constraint relaxation [Sapena, 12] Global optimization [Luo et al., 04] Clustering [Ng., 08]

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Entity-Mention model

Examples: Constraint relaxation [Sapena, 12] Global optimization [Luo et al., 04] Clustering [Ng., 08] Pros: improved expressiveness and results for some approaches

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Ranking models take profit of decisions between mi and all its possible antecedents.

Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Ranking models

Mention-pair models take profit of independent mention pair decisions between mi and each possible antecedent.

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Ranking models

Mention-pair models take profit of independent mention pair decisions between mi and each possible antecedent.

Ranking models take profit of decisions between mi and all its possible antecedents.

Jordi Turmo TALP Research Center [email protected] Coreference Resolution I Resolution: Ai is the set of all possible antecedents of mi

Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Ranking models

Ex: rankers [Denis and Baldrige, 08]

I Learn a ranker from examples

I Example = (mi , αi , Ai ), where αi is the first antecedent of mi and Ai is the set of non-antecedents in a window context of 4 sentences

I Exponential model:

exp P λ f (m , α ) P(α |m ) = k k k i i i i P exp P λ f (m , m ) ms ∈Ai ∪{αi } k k k i s

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Ranking models

Ex: rankers [Denis and Baldrige, 08]

I Learn a ranker from examples

I Example = (mi , αi , Ai ), where αi is the first antecedent of mi and Ai is the set of non-antecedents in a window context of 4 sentences

I Exponential model:

exp P λ f (m , α ) P(α |m ) = k k k i i i i P exp P λ f (m , m ) ms ∈Ai ∪{αi } k k k i s

I Resolution: Ai is the set of all possible antecedents of mi

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Pros: take profit of decisions involving all the candidate antecedents. Cons: always pick an antecedent from the candidates, although the mention in course is not anaphoric. (a classifier of anaphoricity improves the results)

Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Ranking models

Examples: for mentions [Yang et al., 03] [Denis and Baldrige, 08] for partial entity pairs [Rahman and Ng, 09]

Jordi Turmo TALP Research Center [email protected] Coreference Resolution (a classifier of anaphoricity improves the results)

Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Ranking models

Examples: for mentions [Yang et al., 03] [Denis and Baldrige, 08] for partial entity pairs [Rahman and Ng, 09] Pros: take profit of decisions involving all the candidate antecedents. Cons: always pick an antecedent from the candidates, although the mention in course is not anaphoric.

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention-Pair model Mention detection Entity-Mention model ML-based coreference resolution Ranking models References Ranking models

Examples: for mentions [Yang et al., 03] [Denis and Baldrige, 08] for partial entity pairs [Rahman and Ng, 09] Pros: take profit of decisions involving all the candidate antecedents. Cons: always pick an antecedent from the candidates, although the mention in course is not anaphoric. (a classifier of anaphoricity improves the results)

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention detection ML-based coreference resolution References References

I B.J. Grosz, A.K. Joshi and S. Weinstein, Providing a unified account of definite noun phrases in discourse. Proceedings of ACL, 1983.

I B.J. Grosz, A.K. Joshi and S. Weinstein, Centering: A framework for modeling the local coherence of discourse. Computational Linguistics, 21(2), 1995.

I M. Walker, A. Joshi, and E. Prince, editors. 1998. Centering theory in discourse. Oxford University Press

I W.M. Soon, H.T. Ng and D.C.Y. Lim, A machine learning approach to coreference resolution of noun phrases. Computational Linguistics, 27(4), 2001

I V. Ng and C. Cardie, Improving machine learning approaches to coreference resolution. Proceedings of ACL, 2002

Jordi Turmo TALP Research Center [email protected] Coreference Resolution Introduction General methodology Mention detection ML-based coreference resolution References References

I C. Nicolae and A. Nicolae, Best Cut: A graph algorithm for coreference resolution. Proceedings of EMNLP, 2006.

I V. Ng, Supervised noun phrase coreference research: The First Fifteen Years. Proceedings of ACL, 2010.

I E. Sapena, L. Padr´oand J. Turmo. 2010. A global relaxation labeling. Proceedings of COLING 2010

I S. Pradhan, L. Ramshaw, M. Marcus, M. Palmer, R. Weischedel and N. Xue. 2011 CoNLL-2011 Shared Task: Modeling Unrestricted Coreference in OntoNotes

Jordi Turmo TALP Research Center [email protected] Coreference Resolution