Abstractive Multi-document Summarization with Semantic Infor- mation Extraction Wei Li Key Lab of Intelligent Info. Processing, Institute of Computing Technology, CAS Beijing, 100190, China [email protected] analysis without semantic analysis and abstract Abstract representation. Fully abstractive summarization approach re- This paper proposes a novel approach to quires a separate process for the analysis of texts generate abstractive summary for multi- that serves as an intermediate step before the ple documents by extracting semantic in- generation of sentences (Genest and Lapalme, formation from texts. The concept of 2011). Statistics of words or phrases and syntac- Basic Semantic Unit (BSU) is defined to tical analysis that have been widely used in exist- describe the semantics of an event or ac- ing summarization approaches are all shallow tion. A semantic link network on BSUs is processing of text. It is necessary to explore constructed to capture the semantic in- summarization methods based on deeper seman- formation of texts. Summary structure is tic analysis. planned with sentences generated based We define the concept of Basic Semantic Unit on the semantic link network. Experi- (BSU) to express the semantics of texts. A BSU ments demonstrate that the approach is is an action indicator with its obligatory argu- effective in generating informative, co- ments which contain actor and receiver of the herent and compact summary. action. BSU is the most basic element of coher- ent information in texts, which can describe the 1 Introduction semantics of an event or action. The semantic information of texts is represented by extracting Most automatic summarization approaches are BSUs and constructing BSU semantic link net- extractive which leverage only literal or syntactic work (Zhuge, 2009). Semantic Link Network information in documents. Sentences are extract- consists of semantic nodes, semantic links and ed from the original documents directly by rank- reasoning rules (Zhuge, 2010; 2011; 2012; ing or scoring and only little post-editing is made 2015b). The semantic nodes can be any resources. (Yih et al., 2007; Wan et al., 2007; Wang et al., In this work, the semantic nodes are BSUs ex- 2008; Wan and Xiao, 2009). Pure extraction has tracted from texts. We use semantic relatedness intrinsic limits compared to abstraction (Carenini between BSUs as semantic links. Then summary and Cheung, 2008). can be generated based on the semantic link net- Abstractive summarization requires semantic work through summary structure planning. analysis and abstract representation of texts, The characteristics of our approaches are as which need knowledge on and beyond the texts follows: (Zhuge, 2015a). There are some abstractive ap- Each BSU describes the semantics of an proaches in recent years: sentence compression event or action. The semantic relatedness be- (Knight and Marcu, 2000; Knight and Marcu, tween BSUs can capture the context seman- 2002; Cohn and Lapata, 2009), sentence fusion tic relations of texts. (Barzilay and McKeown, 2005; Filippova and The BSU semantic link network is an ab- Strube, 2008), and sentence revision (Tanaka et stract representation of texts. Reduction on al., 2009). However, these approaches are sen- the network can obtain important infor- tence rewriting techniques based on syntactical mation of texts with no redundancy. 1908 Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1908–1913, Lisbon, Portugal, 17-21 September 2015. c 2015 Association for Computational Linguistics. Summary is built from sentence to sentence 2011). They extract binary relations from the to a coherent body of information based on web, which is different from our approach that the BSU semantic link network by summary extracts events or actions expressed in texts. structure planning. 3 The Summarization Framework 2 Related Work Our system produces an abstractive summary for There are some abstractive summarization ap- a set of topic related documents. It consists of proaches in recent years. An approach TTG at- two major components: Information extraction tempts to generate abstractive summary by using and summary generation. text-to-text generation to generate sentence for each subject-verb-object triple (Genest and 3.1 Information Extraction Lapalme, 2011). A system that attempts to gen- The semantic information of texts is obtained by erate abstractive summaries for spoken meetings extracting BSUs and constructing BSU semantic was proposed (Wang and Cardie, 2013). It iden- link network. A BSU is represented as an actor- tifies relation instances that are represented by a action-receiver triple, which can both detects lexical indicator with an argument constituent the crucial content and incorporates enough syn- from texts. Then the relation instances are filled tactic information to facilitate the downstream into templates which are extracted by applying sentence generation. Some actions may not have multiple sequence alignment. Both of these sys- the receiver argument. For example, “Flight tems need to select a subset of the large volumes MH370 – disappear” and “Flight MH370 - leave of generated sentences. However, our system - Kuala Lumpur” are two BSUs. generates summary directly by summary struc- BSU Extraction. BSUs are extracted from the ture planning. It can generate well-organized and sentences of the documents. The texts are pre- coherent summary more effectively. processed by name entity recognition (Finkel et A recent work aims to generate abstractive al., 2005) and co-reference resolution (Lee et al., summary based on Abstract Meaning Represen- 2011). Constituent and dependency parses are tation (AMR) (Liu et al., 2015). It first parses the obtained by Stanford parser (Klein and Manning, source text into AMR graphs, and then trans- 2003). The eligible action indicator is restricted forms them into a summary graph and plans to to be a predicate verb; the eligible actor and re- generate text from it. This work only focuses on ceiver arguments are noun phrase. Both the actor the graph-to-graph transformation. The module and receiver arguments take the form of constit- of text generation from AMR has not been de- uents in the parse tree. A valid BSU should have veloped. The nodes and edges of AMR graph are one action indicator and at least one actor argu- entities and relations between entities respective- ment, and satisfy the following constraints: ly, which are sufficiently different from the BSUs The actor argument is the nominal subject or semantic link network. Moreover, texts can be external subject or the complement of a pas- generated efficiently from the BSUs network. sive verb which is introduced by the preposi- Another recent abstractive summarization meth- tion “by” and does the action. od generates new sentences by selecting and The receiver argument is the direct object or merging phrases from the input documents (Bing the passive nominal subject or the object of et al., 2015). It first extracts noun phrases and a preposition following the action verb. verb-object phrases from the input documents, We create some manual rules and syntactic and then calculates saliency scores for them. An constraints to identify all BSUs based on the syn- ILP optimization framework is used to simulta- tactic structure of sentences in the input texts. neously select and merge informative phrases to maximize the salience of phrases and meanwhile Constructing BSU Semantic Link Network. satisfy the sentence construction constraints. As The semantic relatedness between BSUs contains the results show that the method is difficult to three parts: Arguments Semantic Relatedness generate new informative sentences really differ- (ASR), Action-Verbs Semantic Relatedness ent from the original sentences and may generate (VSR) and Co-occurrence in the Same Sentence some none factual sentences since phrases from (CSS). Arguments of BSUs include actors and different sentences are merged. receivers, which both are noun phrases and indi- Open information extraction has been pro- cate concepts or entities in the text. When com- posed by (Banko et al., 2007; Etzioni et al., puting ASR, the semantic relatedness between concepts must be measured. We use the explicit 1909 semantic analysis based on Wikipedia to com- Basic Features Number of words in actor/receiver pute semantic relatedness between concepts (Ga- Number of nouns in actor/receiver brilovich and Markovitch, 2007). When compu- Number of new nouns in actor/receiver Actor/receiver has capitalized word? ting VSR, WordNet-based measure is used to Actor/receiver has stopword? calculate the semantic relatedness between action Action is a phrasal verb? Content Features verbs (Mihalcea et al., 2006). CSS is measured Actor/receiver has name entity? TF/IDF/TF-IDF of action whether two different BSUs co-occur in the same TF/IDF/TF-IDF min max average of actor/receiver sentence. Semantic relations between BSUs are Syntax Features Constituent tag of actor/action/receiver computed by linearly combining these three parts. Dependency relation of action with actor Then BSUs that are extracted from the texts form Dependency relation of action with receiver a semantic link network. Table 1. Features for BSU summary-worthy Semantic Link Network Reduction. A dis- scoring. We use SVM-light with RBF kernel criminative ranker based on Support Vector Re- by default parameters (Joachims, 1999). gression (SVR) (Smola and Scholkopf, 2004) is utilized to assign