Universal Semantic Parsing † ‡ ‡ †† †† Siva Reddy ∗ Oscar Tackstr¨ om¨ Slav Petrov Mark Steedman Mirella Lapata †Stanford University ‡ Google Inc. ††University of Edinburgh [email protected], oscart, slav @google.com, steedman, mlap @inf.ed.ac.uk { } { } Abstract Seeking to exploit the benefits of UD for natu- ral language understanding, we introduce UDEP- Universal Dependencies (UD) offer a uni- LAMBDA, a semantic interface for UD that maps form cross-lingual syntactic representation, natural language to logical forms, representing un- with the aim of advancing multilingual ap- derlying predicate-argument structures, in an al- plications. Recent work shows that se- most language-independent manner. Our frame- mantic parsing can be accomplished by work is based on DEPLAMBDA (Reddy et al., transforming syntactic dependencies to log- 2016) a recently developed method that converts ical forms. However, this work is lim- English Stanford Dependencies (SD) to logical ited to English, and cannot process de- forms. The conversion process is illustrated in pendency graphs, which allow handling Figure1 and discussed in more detail in Section2. complex phenomena such as control. In Whereas DEPLAMBDA works only for English, U- this work, we introduce UDEPLAMBDA, DEPLAMBDA applies to any language for which a semantic interface for UD, which maps UD annotations are available.1 Moreover, DEP- natural language to logical forms in an LAMBDA can only process tree-structured inputs almost language-independent fashion and whereas UDEPLAMBDA can also process depen- can process dependency graphs. We per- dency graphs, which allow to handle complex con- form experiments on question answering structions such as control. The different treatments against Freebase and provide German and of various linguistic constructions in UD compared Spanish translations of the WebQuestions to SD also require different handling in UDEP- and GraphQuestions datasets to facilitate LAMBDA (Section 3.3). multilingual evaluation. Results show that Our experiments focus on Freebase semantic UDEPLAMBDA outperforms strong base- parsing as a testbed for evaluating the framework’s lines across languages and datasets. For multilingual appeal. We convert natural language English, it achieves a 4.9 F1 point improve- to logical forms which in turn are converted to ma- ment over the state-of-the-art on Graph- chine interpretable formal meaning representations Questions. for retrieving answers to questions from Freebase. To facilitate multilingual evaluation, we provide arXiv:1702.03196v4 [cs.CL] 28 Aug 2017 1 Introduction translations of the English WebQuestions (Berant The Universal Dependencies (UD) initiative seeks et al., 2013) and GraphQuestions (Su et al., 2016) to develop cross-linguistically consistent annota- datasets to German and Spanish. We demonstrate tion guidelines as well as a large number of uni- that UDEPLAMBDA can be used to derive logical formly annotated treebanks for many languages forms for these languages using a minimal amount (Nivre et al., 2016). Such resources could advance of language-specific knowledge. Aside from devel- multilingual applications of parsing, improve com- oping the first multilingual semantic parsing tool parability of evaluation results, enable cross-lingual for Freebase, we also experimentally show that U- learning, and more generally support natural lan- DEPLAMBDA outperforms strong baselines across guage understanding. 1As of v1.3, UD annotations are available for 47 languages ∗ Work done at the University of Edinburgh at http://universaldependencies.org. languages and datasets. For English, it achieves the root strongest result to date on GraphQuestions, with nmod case competitive results on WebQuestions. Our imple- dobj det mentation and translated datasets are publicly avail- nsubj det compound able at https://github.com/sivareddyg/udeplambda. Disney won an Oscar for the movie Frozen propn verb det propn adp det noun propn EP AMBDA 2 D L (a) The dependency tree for Disney won an Oscar for the movie Frozen in the Universal Dependencies formalism. Before describing UDEPLAMBDA, we provide an overview of DEPLAMBDA (Reddy et al., 2016) (nsubj (nmod (dobj won (det Oscar an)) on which our approach is based. DEPLAMBDA (case (det (comp. Frozen movie) the) for)) Disney) converts a dependency tree to its logical form in (b) The binarized s-expression for the dependency tree. three steps: binarization, substitution, and com- λx. yzw.won(x ) Disney(y ) Oscar(z ) position, each of which is briefly outlined below. ∃ e ∧ a ∧ a Frozen(wa) movie(wa) Algorithm1 describes the steps of DEPLAMBDA ∧arg (x ,y ) ∧ arg (x ,z ) nmod.for(x ,w ) ∧ 1 e a ∧ 2 e a ∧ e a in lines 4-6, whereas lines 2 and 3 are specific to (c) The composed lambda-calculus expression. UDEPLAMBDA. Binarization A dependency tree is first mapped Figure 1: The mapping of a dependency tree to its to a Lisp-style s-expression indicating the order logical form with the intermediate s-expression. of semantic composition. Figure 1(b) shows the s-expression for the sentence Disney won an Os- of type Ind Event. Roughly speaking, proper car for the movie Frozen, derived from the depen- × nouns and adjectives invoke ENTITY expressions, dency tree in Figure 1(a). Here, the sub-expression verbs and adverbs invoke EVENT expressions, and (dobj won (det Oscar an)) indicates that the logi- common nouns invoke both ENTITY and EVENT ex- cal form of the phrase won an Oscar is derived by pressions (see Section 3.3), while remaining words composing the logical form of the label dobj with invoke FUNCTIONAL expressions. DEPLAMBDA the logical form of the word won and the logical enforces the constraint that every s-expression is of form of the phrase an Oscar, derived analogously. the type η = Ind Event Bool, which simpli- × → The s-expression can also be interpreted as a bi- fies the type system considerably. narized tree with the dependency label as the root Expressions for dependency labels glue the node, and the left and right expressions as subtrees. semantics of heads and modifiers to articulate A composition hierarchy is employed to impose predicate-argument structure. These expressions in a strict traversal ordering on the modifiers to each general take one of the following forms: head in the dependency tree. As an example, won has three modifiers in Figure 1(a), which according COPY λ f gx. y. f (x) g(y) rel(x,y) e.g. nsubj, dobj⇒ , nmod∃, advmod∧ ∧ to the composition hierarchy are composed in the INVERT λ f gx. y. f (x) g(y) reli(y,x) order dobj > nmod > nsubj. In constructions like e.g. amod, acl⇒ ∃ ∧ ∧ coordination, this ordering is crucial to arrive at MERGE λ f gx. f (x) g(x) e.g. compound⇒, appos, amod∧, acl the correct semantics. Lines 7-17 in Algorithm1 HEAD λ f gx. f (x) describe the binarization step. e.g. case, punct⇒ , aux, mark . Substitution Each symbol in the s-expressions As an example of COPY, consider the lambda is substituted for a lambda expression encoding expression for dobj in (dobj won (det Oscar an)): its semantics. Words and dependency labels are λ f gx. y. f (x) g(y) arg (x ,y ). This expres- ∃ ∧ ∧ 2 e a assigned different types of expressions. In general, sion takes two functions f and g as input, where words have expressions of the following kind: f represents the logical form of won and g repre- ENTITY λx.word(x ); e.g. Oscar λx.Oscar(x ) sents the logical form of an Oscar. The predicate- ⇒ a ⇒ a EVENT λx.word(x ); e.g. won λx.won(x ) ⇒ e ⇒ e argument structure arg2(xe,ya) indicates that the FUNCTIONAL λx. TRUE; e.g. an λx. TRUE ⇒ ⇒ arg2 of the event xe, i.e. won, is the individual ya, Here, the subscripts and denote the types i.e. the entity Oscar. Since arg (x ,y ) mimics the ·a ·e 2 e a of individuals (Ind) and events (Event), respec- dependency structure dobj(won, Oscar), we refer tively, whereas x denotes a paired variable (xa,xe) to the expression kind evoked by dobj as COPY. Expressions that invert the dependency direc- Algorithm 1: UDEPLAMBDA Steps tion are referred to as INVERT (e.g. amod in run- 1 Function UDepLambda(depTree): ning horse); expressions that merge two subexpres- 2 depGraph = Enhancement (depTree) sions without introducing any relation predicates #See Figure 2(a) for a depGraph. 3 bindedTree = SplitLongDistance (depGraph) are referred to as MERGE (e.g. compound in movie #See Figure 2(b) for a bindedTree. Frozen); and expressions that simply return the par- 4 binarizedTree = Binarization (bindedTree) #See Figure 1(b) for a binarizedTree. ent expression semantics are referred to as HEAD 5 logicalForm = Composition (binarizedTree) (e.g. case in for Frozen). While this generalization 6 return logicalForm applies to most dependency labels, several labels take a different logical form not listed here, some 7 Function Binarization (tree): of which are discussed in Section 3.3. Sometimes 8 parent = GetRootNode (tree); the mapping of dependency label to lambda expres- 9 (label1,child1),(label2,child2)... { = GetChildNodes (parent) } sion may depend on surrounding part-of-speech 10 sortedChildren = SortUsingLabelHierarchy ( (label1,child1),(label2,child2)... ) tags or dependency labels. For example, amod acts { } 11 binarziedTree.root = parent as INVERT when the modifier is a verb (e.g. in run- 12 for label, child sortedChildren: ∈ ning horse), and as MERGE when the modifier is 13 temp.root = label an adjective (e.g. in beautiful horse).2 Lines 26-32 14 temp.le ft = binarziedTree 15 temp.right = Binarization(child) in Algorithm1 describe the substitution procedure. 16 binarziedTree = temp 17 return binarizedTree Composition The final logical form is computed by beta-reduction, treating expressions of the form 18 Composition (f x y) as the function f applied to the arguments Function (binarizedTree): 19 mainLF = Substitution (binarizedTree.root) x and y. For example, (dobj won (det Oscar an)) 20 if binarziedTree has left and right children: 21 le ftLF = Composition (binarziedTree.le ft) results in λx.