Arxiv:2103.16929V1 [Cs.CL] 31 Mar 2021
Total Page:16
File Type:pdf, Size:1020Kb
Deep Neural Approaches to Relation Triplets Extraction: A Comprehensive Survey Tapas Nayaky, Navonil Majumder, Pawan Goyaly, Soujanya Poria y IIT Kharagpur, India Singapore University of Technology and Design, Singapore [email protected], fnavonil majumder,[email protected], [email protected] Abstract contain a large number of triplets, they remain incomplete. On the other hand, relation triplets Recently, with the advances made in contin- can be automatically distilled from the copious uous representation of words (word embed- amount of free text on the Web. This can be lever- dings) and deep neural architectures, many re- search works are published in the area of rela- aged for identifying missing links in the existing tion extraction and it is very difficult to keep KBs or build a KB from scratch without human track of so many papers. To help future re- intervention. search, we present a comprehensive review There are two distinct research paradigms of of the recently published research works in relation extraction: open information extraction relation extraction. We mostly focus on re- (Open IE) and supervised relation extraction. lation extraction using deep neural networks which have achieved state-of-the-art perfor- Banko et al.(2007), Christensen et al.(2011), Et- mance on publicly available datasets. In this zioni et al.(2011), and Mausam et al.(2012) use survey, we cover sentence-level relation ex- open information extraction (Open IE) to extract traction to document-level relation extraction, relation triplets from sentences where relations set pipeline-based approaches to joint extraction is open. Open IE systems like KnowItAll (Etzioni approaches, annotated datasets to distantly su- et al., 2004), TEXTRUNNER (Yates et al., 2007), pervised datasets along with few very recent REVERB (Etzioni et al., 2011), SRLIE (Chris- research directions such as zero-shot or few- shot relation extraction, noise mitigation in tensen et al., 2011), and OLLIE (Mausam et al., distantly supervised datasets. Regarding neu- 2012) use rule-based methods to extract entities ral architectures, we cover convolutional mod- from the noun phrases and relations from the verb els, recurrent network models, attention net- phrases present in sentences. These systems can work models, and graph convolutional models extract a large number of triplets of diverse re- in this survey. lations from text within a reasonable time frame. These models extract any verb phrase in the sen- 1 Introduction tences as a relation thus yielding too many un- A relation triplet consists of two entities and a re- informative triplets. Also, a relation can be ex- lation between them. We can find such triplets in pressed in sentences with different surface forms arXiv:2103.16929v1 [cs.CL] 31 Mar 2021 a structured format in several publicly available (lives in relation can be expressed with ‘lives in’, knowledge bases (KBs) such as, Freebase (Bol- ‘stays’, ‘settles’, ‘lodges’, ‘resident of’, etc) and lacker et al., 2008), DBpedia (Bizer et al., 2009), Open IE treats them as different relations which Wikidata (Vrandeciˇ c´ and Krotzsch¨ , 2014), etc. leads to duplication of triplets. These triplets are very useful for many natural lan- The problems of the Open IE can be addressed guage processing tasks such as machine reading using supervised relation extraction. In supervised comprehension (Qiu et al., 2019), machine trans- relation extraction, we consider a fixed set of re- lation (Zhao et al., 2020), abstractive summariza- lations, thus there is no need to do any normal- tion (huang et al., 2020), etc. However, building ization of the extracted relations. This approach such knowledge bases is a daunting task. The requires a large parallel corpus of text and rela- aforementioned KBs are built by crowdsourcing, tion triplets for training. There are some anno- which may not be scalable. Although these KBs tated and some distantly supervised parallel cor- Class Sentence Triplets The original Joy of Cooking was published in 1931 NEO <Irma Rombauer, St. Louis, place lived> by Irma Rombauer, a St. Louis housewife. <Germany, Berlin, capital> EPO Berlin is the capital of Germany. <Germany, Berlin, contains> <Berlin, Germany, country> Dr. C. V. Raman who was born in Chennai worked <Dr. C. V. Raman, Chennai, birth place> SEO mostly in Kolkata. <Dr. C. V. Raman, Kolkata, place lived> Table 1: Examples of different classes of overlapping relation triplets. This table is taken from Nayak(2020). pus of (text, triplets) available publicly that can Joint-extraction approaches, in contrast, jointly be used for training the models. Creating anno- find the entities and relations. Joint models extract tated corpus is difficult and time-consuming, so only the valid relational triplets and they do not datasets created in this way are relatively smaller need to extract the None triplets. Relation triplets in size. On the other hand, the distant supervision may share one or both entities among them and approach can be exploited to create a large training this overlapping of entities makes this task chal- corpus automatically, but these datasets contain a lenging. Based on the overlap of entities, we di- significant amount of noisy labels. These noisy vide the sentences into three classes: (i) No Entity labels in the distantly supervised datasets can af- Overlap (NEO): A sentence in this class has one fect the performance of the models in a negative or more triplets, but they do not share any enti- way. Several feature-based models and deep neu- ties. (ii) Entity Pair Overlap (EPO): A sentence ral network-based are proposed in the last decade in this class has more than one triplet, and at least for relation extraction. In this survey, we discuss two triplets share both the entities in the same or these datasets and models in detail in the remain- reverse order. (iii) Single Entity Overlap (SEO): ing part of the paper. A sentence in this class has more than one triplet Previously, Cui et al.(2017); Pawar et al. and at least two triplets share exactly one entity. It (2017); Kumar(2017); Shi et al.(2019); Han et al. should be noted that a sentence can belong to both (2020) presented survey of the research works in EPO and SEO classes. The goal is to extract all the relation extraction, but they mostly focused relation triplets present in a sentence. on pipeline-based relation extraction approaches at the sentence-level. Different from these survey 3 Scope of this Survey papers, we extend the survey to document-level re- In this survey, we focus on the relation triplets con- lation extraction and joint entity and relation ex- cerning PERSON, ORGANIZATION, and LO- traction approaches. We also survey very recent CATION mainly. Many research works are pub- research directions in this area such as zero-shot or lished for domain-specific relation extraction such few-shot relation extraction and noise mitigation scientific articles (Luan et al., 2017; Jain et al., in distantly supervised datasets. To the best of our 2020), medical (Gu et al., 2016; Li et al., 2017; knowledge, this is the first survey paper that cov- Choi, 2018; Thillaisundaram and Togia, 2019), le- ers so many different aspects of relation extraction gal (Andrew, 2018), finance (Vela and Declerck, in detail. 2009), etc. But in this survey, we do not include the research papers that only focus on a particular 2 Task Description domain. Also, we only focus on relation extraction Given a sentence and a set of relations R as input, for the English language. the task is to extract a set of relation triplets, with 4 Challenges of Dataset Annotation relations from R, from the sentence. Pipeline- based relation extraction approaches divide the Existing KBs, such as Freebase, Wikidata, and task into two sub-tasks: (i) entity recognition and DBpedia, are manually built which takes much ef- (ii) relation classification. In the first sub-task, all fort and time. However, these KBs still have a the candidate entities are identified in a sentence. large number of missing links. On the other hand, In the second sub-task, the relation between every we can find evidence of a large number of rela- possible ordered pair of candidate entities is deter- tion triplets in free texts. We have included some mined — this relation may not exist (None). examples of such triplets and texts in Table2. If Relation Entity 1 Entity 2 Text Meera Jasmine made her debut in the acted in Meera Jasmine Sootradharan Malayalam film “Soothradharan”. Chakkarakadavu is a small village to the east of the town of Cherai, on located in Chakkarakadavu Kerala Vypin Island in Ernakulam district, Kerala, India . birth place Barack Obama Hawaii Barack Obama was born in Hawaii. Fode´ Moussa Sylla is a Guinean plays for Moussa Sylla Horoya AC football player, who currently plays for Horoya AC. MTV Channel (Pvt) Ltd is a Sri Lankan media company which owns owns MTV Channel Shakthi TV three national television channels - Shakthi TV, Sirasa TV and TV 1 . Table 2: Examples of relation triplets found in free texts. This table is taken from Nayak(2020). we can extract relation triplets automatically from these two entities in the KB, that sentence is con- the text, we can build a KB from scratch or add sidered as a source of None triplet between the two new triplets to the existing KBs without any man- entities. These None samples are useful as dis- ual effort. But to achieve this goal, we need a large tantly supervised models consider only a limited number of texts annotated with relation triplets, set of positive relations. Any relation outside this and creating such a corpus manually is a daunting set is considered as None relation. This method task. One possible way to do the annotation is to can give us a large number of triplet-to-text map- identify the entities in the text and then for all pos- pings which can be used to build supervised mod- sible pairs of entities, identify the relations from a els for this task.