Textual Analogy Parsing: What’S Shared and What’S Compared Among Analogous Facts
Total Page:16
File Type:pdf, Size:1020Kb
Textual Analogy Parsing: What’s Shared and What’s Compared among Analogous Facts Matthew Lamm1;3 Arun Tejasvi Chaganty2;3∗ Christopher D. Manning1;2;3 Dan Jurafsky1;2;3 Percy Liang2;3 1Stanford Linguistics 2Stanford Computer Science 3Stanford NLP Group mlamm,jurafsky @stanford.edu chaganty,manning,pliangf g @cs.stanford.edu f g Abstract mention According to the U.S. Census , almost 10.9 million African parsing Americans , or 28% , live To understand a sentence like “whereas at or below the poverty line , analogy frame only 10% of White Americans live at or compared with 15% of Lati- source U.S. Census nos and approximately 10% below the poverty line, 28% of African quant live at or below the povery line of White Americans . Americans do” it is important not only to whole African Americans value 28% identify individual facts, e.g., poverty rates visualization whole Latinos of distinct demographic groups, but also the value 15% higher-order relations between them, e.g., the 25 whole White Americans value 10% disparity between them. In this paper, we 20 15 propose the task of Textual Analogy Parsing plotting (TAP) to model this higher-order meaning. 10 The output of TAP is a frame-style meaning % at/below poverty line Wh. Lat. AA representation which explicitly specifies what Figure 1: In textual analogy parsing (TAP), one is shared (e.g., poverty rates) and what is maps analogous facts to semantic role represen- compared (e.g., White Americans vs. African tations and identifies analogical relations between Americans, 10% vs. 28%) between its com- them. Automated chart generation from text is a ponent facts. Such a meaning representation motivating application of TAP. can enable new applications that rely on discourse understanding such as automated and C2 are juxtaposed (Kehler, 2002). Thus the chart generation from quantitative text. We author intends that we consider them in relation to present a new dataset for TAP, baselines, and a model that successfully uses an ILP to enforce each other, inviting us to note, for example, a dis- the structural constraints of the problem. parity of wealth distribution between demographic groups. To fail to capture this is to miss out on an 1 Introduction important aspect of text understanding. We propose the task of Textual Analogy Parsing The task of information extraction by and large (TAP) to explicitly capture such relational mean- seeks to populate a knowledge base with individ- ing between analogous facts in text. Concretely, uated facts extracted from text (Sarawagi, 2008). TAP first maps a set of analogous facts to semantic For example, given the sentence: role (SRL) representations, and then identifies the (E1) [According to the U.S. Census, whereas roles along which they are similar (the shared con- only 10% of White Americans live at or tent) and along which they are distinct (the com- below the poverty line today]C1, [28% pared content)—see Figure1. The resulting rep- 1 of African Americans do.]C2 resentation, the TAP frame, is a deeper represen- tation than the one output by shallow discourse one would extract two independent facts about parsers (Taboada and Mann, 2006; Prasad et al., voter registration, about the two distinct demo- 2007; Pitler et al., 2009; Prasad et al., 2010; Sur- graphic groups. On the other hand, the theory deanu et al., 2015). Given (E1) above, a shallow of discourse maintains that part of the above sen- discourse parser would classify the relation of con- tence’s meaning inheres in the fact that clauses C1 trast between C1 and C2—indicating that some ∗Author contributed significantly. salient differences exist in the meanings of the jux- 1Data in E1 and the figure sentence from Morris(2014). taposed phrases—but without identifying the na- mention analogy graph analogy frame White Americans W1 African Americans W2 source U.S. Census quant live at or below the poverty line do ≡ According to the U.S. Census S12 , whereas time today only 10% V1 of White Americans W1 live at or below the poverty line to- 10% 28% whole White Americans Q1 V1 V2 day , 28% of African Ameri- Tm12 V2 value 10% cans do . W2 Q2 U.S. Census S12 today Tm12 whole African Americans value 28% live at or below the poverty line Q1 do Q2 Figure 2: The mapping from utterance to TAP frame. Vertices in the graph are labeled with abbreviated semantic roles. Single lines represent edges between a VALUE and other roles in its associated fact. Dou- ble lines represent coreference and synonymy. Springs represent analogy. Note that vertices connected by equivalence arcs, or any span which connects to both V1 and V2 via fact relations (i.e., scope), map to the shared content of the TAP frame. Analogous spans map to the compared content. ture of those differences. frame 1 frame 2 source U.S. Census source U.S. Census We focus on applying TAP to quantitative facts, quant live at or below the poverty line quant do time today time today because TAP frames can be used to create graphi- whole White Americans whole African Americans cal plots from sentences with numbers, as in Fig- value 10% value 28% ure1. This new application could help to sim- plify complex quantitative text on the web (Bar- Figure 3: Two analogous quantitative facts represented independently, using the QSRL rio et al., 2016; Leonhardt et al., 2017). We thus schema (Lamm et al., 2018). created an expert-annotated dataset of TAP frames over quantitative facts in the Wall Street Journal they are analogous, i.e., structurally and semanti- corpus (Marcus et al., 1999). cally similar but distinct. We model TAP by jointly predicting SRL rep- resentations of facts in a sentence, and higher- Instead, we can explicitly show points of sim- order semantic relations between them. Our main ilarity and difference between them in the two- findings are that a neural architecture outperforms tiered frame structure in Figure2, which we call a log-linear baseline, well-chosen linguistic fea- a TAP frame. The outer tier of the TAP frame con- tures help performance, and so does the use of an tains shared content, or information pertinent to integer-linear programming (ILP) decoder that en- all of the facts in question, and the inner tier con- forces the structural constraints of the task. Nev- tains compared content, the information that varies ertheless, both quantitative and qualitative evalua- across the set of facts. tion reveal room for improvement on TAP. Mapping from an utterance to a TAP frame re- In sum, our main contributions are (1) a new quires three types of relational reasoning. Firstly, task, Textual Analogy Parsing (TAP), that com- one must decompose the utterance into a set of bines shallow semantic parsing with discourse facts, where a fact is represented as a set of se- meaning, (2) a dataset of TAP frames from quan- mantic roles. Then, one must identify the shared titative newswire, and (3) a preliminary study of a content across facts by aligning roles that are se- new application, automated chart generation from mantically equivalent, in the sense that they are ei- text. All data and code, including standardized ther the same span, are coreferent, or are synony- evaluation scripts, are made freely available. mous. For example, in Figure2 the phrase ‘U.S. Census’ occurs as the SOURCE of both facts be- 2 A Semantic Representation of Analogy cause it scopes over the entire sentence in which they appear. Additionally, one must identify the Let us revisit the example sentence from the previ- compared content by aligning roles that are analo- ous section (E1), where a pair of analogous quan- gous, in the sense that they are semantically sim- titative facts about poverty rates of different demo- ilar but nevertheless distinct. For example, the graphic groups are presented in contrast. Individ- phrases ‘White Americans’ and ‘African Ameri- ually, these can be represented using the semantic cans’ are analogous in our running sentence, play- role structures in Figure3, but representing them ing the same role in their respective facts, while separately in this way fails to capture the fact that signifying distinct demographic groups. (a) New England Electric A1 had offered Q1 $2 billion V1 to acquire PS of New Hampshire TH123 , well below the $2.29 billion value United Illuminating places on its bid and the $2.25 billion V2 A2 Q2 V3 Northeast says its bid is worth. A3 Q3 (b) First Boston S12 estimated that UAL TH12 was worth Q12 $ 250 to $ 344 a share V1 based on UAL’s results for the 12 months ending last June 30 , but only $ 235 to $ 266 based on a management estimate C1 V2 of results for 1989 C2 Table 1: Representative sentences from the Quantitative TAP dataset. Co-indexing (e.g., A1/Q1) indi- cates when spans are part of the same QSRL fact. Parentheses indicate shared content spans and brackets indicate compared content spans. To parse (a), one must recognize that ‘to acquire PS of New Hamp- shire’ is elided but nevertheless an implied TH(eme) in two of the clauses, and that ‘offered’ and ‘bid’ are contextually synonymous Q(uantities). Moreover, one must note that the A(gents) are analogous, and hence part of the compared content. In (b), ‘First Boston’, ‘UAL’ and ‘worth’, contribute a S(ource), TH(eme), and Q(uantity) to the shared content respectively. Here, C(ause) roles are compared content. Train (n = 1000) Test (n = 100) spans with like roles surrounded by brackets are av. max tot. av. max tot. compared content, meaning that they are analo- gous but semantically distinct.