Deep Linguistic Analysis for the Accurate Identification of Predicate

Deep Linguistic Analysis for the Accurate Identiﬁcation of Predicate-Argument Relations

Yusuke Miyao Jun'ichi Tsujii Department of Computer Science Department of Computer Science University of Tokyo University of Tokyo [email protected] CREST, JST [email protected]

Abstract obtained by deep linguistic analysis (Gildea and This paper evaluates the accuracy of HPSG Hockenmaier, 2003; Chen and Rambow, 2003). parsing in terms of the identification of They employed a CCG (Steedman, 2000) or LTAG predicate-argument relations. We could directly (Schabes et al., 1988) parser to acquire syntac- compare the output of HPSG parsing with Prop- tic/semantic structures, which would be passed to Bank annotations, by assuming a unique map- statistical classifier as features. That is, they used ping from HPSG semantic representation into deep analysis as a preprocessor to obtain useful fea- PropBank annotation. Even though PropBank tures for training a probabilistic model or statistical was not used for the training of a disambiguation model, an HPSG parser achieved the ac- classifier of a semantic argument identifier. These curacy competitive with existing studies on the results imply the superiority of deep linguistic anal- task of identifying PropBank annotations. ysis for this task. Although the statistical approach seems a reason- 1 Introduction able way for developing an accurate identifier of Recently, deep linguistic analysis has successfully PropBank annotations, this study aims at establish- been applied to real-world texts. Several parsers ing a method of directly comparing the outputs of have been implemented in various grammar for- HPSG parsing with the PropBank annotation in or- malisms and empirical evaluation has been re- der to explicitly demonstrate the availability of deep ported: LFG (Riezler et al., 2002; Cahill et al., parsers. That is, we do not apply statistical model 2002; Burke et al., 2004), LTAG (Chiang, 2000), nor machine learning to the post-processing of the CCG (Hockenmaier and Steedman, 2002b; Clark et output of HPSG parsing. By eliminating the effect al., 2002; Hockenmaier, 2003), and HPSG (Miyao of post-processing, we can directly evaluate the ac- et al., 2003; Malouf and van Noord, 2004). How- curacy of deep linguistic analysis. ever, their accuracy was still below the state-of-the- Section 2 introduces recent advances in deep lin- art PCFG parsers (Collins, 1999; Charniak, 2000) in guistic analysis and the development of semanti- terms of the PARSEVAL score. Since deep parsers cally annotated corpora. Section 3 describes the de- can output deeper representation of the structure of tails of the implementation of an HPSG parser eval- a sentence, such as predicate argument structures, uated in this study. Section 4 discusses a problem in several studies reported the accuracy of predicate- adopting PropBank for the performance evaluation argument relations using a treebank developed for of deep linguistic parsers and proposes its solution. each formalism. However, resources used for the Section 5 reports empirical evaluation of the accu- evaluation were not available for other formalisms, racy of the HPSG parser. and the results cannot be compared with each other. In this paper, we employ PropBank (Kingsbury 2 Deep linguistic analysis and and Palmer, 2002) for the evaluation of the accu- semantically annotated corpora racy of HPSG parsing. In the PropBank, semantic Riezler et al. (2002) reported the successful applica- arguments of a predicate and their semantic roles tion of a hand-crafted LFG (Bresnan, 1982) gram- are manually annotated. Since the PropBank has mar to the parsing of the Penn Treebank (Marcus been developed independently of any grammar for- et al., 1994) by exploiting various techniques for malisms, the results are comparable with other pub- robust parsing. The study was impressive because lished results using the same test data. most researchers had believed that deep linguistic Interestingly, several studies suggested that the analysis of real-world text was impossible. Their identification of PropBank annotations would re- success owed much to a consistent effort to main- quire linguistically-motivated features that can be tain a wide-coverage LFG grammar, as well as var- ARG0-choose S FrameNet (Baker et al., 1998) are large English cor- NP-1 VP pora annotated with the semantic relations of words they VP VP in a sentence. Figure 1 shows an example of the did n’t have S annotation of the PropBank. As the target text of NP VP the PropBank is the same as the Penn Treebank, a ARG0-choose *-1 to VP ARG1-choose syntactic structure is given by the Penn Treebank. choose NP The PropBank includes additional annotations rep- REL-choose this particular moment resenting a predicate and its semantic arguments in a syntactic tree. For example, in Figure 1, REL de- notes a predicate, “choose”, and ARG represents Figure 1: Annotation of the PropBank its semantic arguments: “they” for the 0th argument (i.e., subject) and “this particular moment” for the ious techniques for robust parsing. 1st argument (i.e., object). However, the manual development of wide- Existing studies applied statistical classifiers to coverage linguistic grammars is still a difficult task. the identification of the PropBank or FrameNet an- Recent progress in deep linguistic analysis has notations. Similar to many methods of applying ma- mainly depended on the acquisition of lexicalized chine learning to NLP tasks, they first formulated grammars from annotated corpora (Xia, 1999; Chen the task as identifying in a sentence each argument and Vijay-Shanker, 2000; Chiang, 2000; Hocken- of a given predicate. Then, parameters of the iden- maier and Steedman, 2002a; Cahill et al., 2002; tifier were learned from the annotated corpus. Fea- Frank et al., 2003; Miyao et al., 2004). This ap- tures of a statistical model were defined as a pat- proach not only allows for the low-cost develop- tern on a partial structure of the syntactic tree output ment of wide-coverage grammars, but also provides by an automatic parser (Gildea and Palmer, 2002; the training data for statistical modeling as a by- Gildea and Jurafsky, 2002). product. Thus, we now have a basis for integrating Several studies proposed the use of deep linguis- statistical language modeling with deep linguistic tic features, such as predicate-argument relations analysis. To date, accurate parsers have been devel- output by a CCG parser (Gildea and Hockenmaier, oped for LTAG (Chiang, 2000), CCG (Hockenmaier 2003) and derivation trees output by an LTAG parser and Steedman, 2002b; Clark et al., 2002; Hocken- (Chen and Rambow, 2003). Both studies reported maier, 2003), and LFG (Cahill et al., 2002; Burke et that the identification accuracy improved by inal., 2004). Those studies have opened up the appli- troducing such deep linguistic features. Although cation of deep linguistic analysis to practical use. deep analysis has not outperformed PCFG parsers in However, the accuracy of those parsers was still terms of the accuracy of surface structure, these re- below PCFG parsers (Collins, 1999; Charniak, sults are implicitly supporting the necessity of deep 2000) in terms of the PARSEVAL score, i.e., labeled linguistic analysis for the recognition of semantic bracketing accuracy of CFG-style parse trees. Since relations. one advantage of deep parsers is that they can out- However, these results do not directly reflect the put a sort of semantic representation, e.g. predicate- performance of deep parsers. Since these corpora argument structures, several studies have reported provide deeper structure of a sentence than surface the accuracy of predicate-argument relations (Hock- parse trees, they would be suitable for the evalua- enmaier and Steedman, 2002b; Clark et al., 2002; tion of deep parsers. In Section 4, we explore the Hockenmaier, 2003; Miyao et al., 2003). However, possibility of using the PropBank for the evaluation their evaluation employed a treebank developed for of an HPSG parser. a specific grammar formalism. Hence, those results cannot be compared fairly with parsers based on other formalisms including PCFG parsers. 3 Implementation of an HPSG parser At the same time, following the great success of machine learning approaches in NLP, many re- This study evaluates the accuracy of a general- search efforts are being devoted to developing vari- purpose HPSG parser that outputs predicate argu- ous annotated corpora. Notably, several projects are ment structures. While details have been explained underway to annotate large corpora with semantic in other papers (Miyao et al., 2003; Miyao et al., information such as semantic relations of words and 2004), in the remainder of this section, we briefly coreferences. review the grammar and the disambiguation model PropBank (Kingsbury and Palmer, 2002) and of our HPSG parser. S PHON “choose” arg head NP-1 VP HEAD verb arg HEAD noun head they VP VP SUBJ < SEM 1 > head mod head arg n’t have HEAD noun did S COMPS < > head SEM 2 NP VP arg REL choose *-1 tohead VP SEM ARG0 1 arg head ARG1 2 choose NP

this particular moment

HEAD verb Figure 3: Mapping from syntactic arguments to se- SUBJ < > COMPS < > subject-head mantic arguments HEAD noun HEAD verb 1 SUBJ < > SUBJ < > COMPS < > 2 head-comp HEAD verb HEAD verb we get lexical entries for all of the words in the tree they SUBJ < _ > SUBJ < 2 > head-comp (the bottom). did n’t have HEAD verb SUBJ < 1 > As shown in the ﬁgure, we can also obtain com- head-comp to HEAD verb plete HPSG derivation trees, i.e., an HPSG tree- SUBJ < 1 > head-comp bank. It is available for the machine learning of dis- choose HEAD noun SUBJ < > COMPS < > ambiguation models, and can also be used for the evaluation of HPSG parsing. this particular moment

In an HPSG grammar, syntax-to-semantics map-

HEAD verb pings are implemented in lexical entries. For exam- SUBJ < > COMPS < > ple, when we have a lexical entries for “choose” HEAD noun HEAD verb as shown in Figure 3, the lexical entry includes 1 SUBJ < > SUBJ < 1 > COMPS < > COMPS < > mappings from syntactic arguments (SUBJ and HEAD verb HEAD verb they SUBJ < 1 > 2 SUBJ < 1 > COMPS features) into a predicate-argument struc- COMPS < 2 > COMPS < > ture (ARG0 and ARG1 features). Argument labels did n’t HEAD verb HEAD verb SUBJ < 1 > 3 SUBJ < 1 > in a predicate-argument structure are basically de- COMPS < 3 > COMPS < > ﬁned in a left-to-right order of syntactic realizations, have to HEAD verb SUBJ < 1 > COMPS < > while if we had a cue for a movement in the Penn Treebank, arguments are put in its canonical posi- HEAD verb HEAD noun SUBJ < 1 > 4 SUBJ < > COMPS < 4 > COMPS < > tion in a predicate-argument structure.

choose this particular moment 3.2 Disambiguation model By grammar extraction, we are able to obtain a large Figure 2: Extracting HPSG lexical entries from the lexicon together with complete derivation trees of Penn Treebank-style parse tree HPSG, i.e, an HPSG treebank. The HPSG treebank can then be used as training data for the machine learning of the disambiguation model. 3.1 Grammar Following recent research about disambiguation The grammar used in this paper follows the theory models on linguistic grammars (Abney, 1997; John- of HPSG (Pollard and Sag, 1994), and is extracted son et al., 1999; Riezler et al., 2002; Clark and Cur- from the Penn Treebank (Miyao et al., 2004). In ran, 2003; Miyao et al., 2003; Malouf and van No- this approach, a treebank is annotated with partially ord, 2004), we apply a log-linear model or maxi- speciﬁed HPSG derivations using heuristic rules. mum entropy model (Berger et al., 1996) on HPSG

By inversely applying schemata to the derivations, derivations. We represent an HPSG sign as a tu- ¦

partially speciﬁed constraints are percolated and in- ple ¡£¢¥¤§¦©¨§ ¨ , where is a lexical sign of the tegrated into lexical entries, and a large HPSG-style head word, is a part-of-speech, and is a sym- lexicon is extracted from the treebank. bol representing the structure of the sign (mostly Figure 2 shows an example of extracting HPSG corresponding to nonterminal symbols of the Penn

lexical entries from a Penn Treebank-style parse Treebank). Given an HPSG schema and the dis-

tree. Firstly, given a parse tree (the top of the ﬁg- tance between the head words of the head/non- ure), we annotate partial speciﬁcations on an HPSG head daughter constituents, each (binary) branch-

derivation (the middle). Then, HPSG schemata are ing of an HPSG derivation is represented as a tuple

¤§¨ ¨¦¨§ ¨ ¨¦¨§ ¨ ¨! applied to each branching in the derivation. Finally, ¢ , where

¡ ¢¤£ ¥¦£ §¨£ ¢¤© ¥¦© § ©

ARG0-broke S ARG1-broke S

NP VP NP VP

ARG1-broke

– He broke NP the window broke

– – REL-broke the window REL-broke

– – –

– –

– – – Figure 4: Annotation of an ergative verb in the Prop-

– – – –

Bank – – – – – ARG1-broke S

NP VP

¢ ¥ §

£ £ £

the window broke PP ARG3-broke

REL-broke into NP

–

a million pieces

–

– – Figure 5: Annotation of another usage of “broke” Table 1: Feature function templates used in the disambiguation model of HPSG parsing: for binary the other features are 0:

schema applications (top) and for unary ones (bot- C

D D D D D H head comp DFEGD trans VB VP noun NNS NP

tom) C

D D D D D D H

head comp D trans VB VP noun NNS NP

D D D D D H

head comp DIE*D VB VP NNS NP

D D D D D D H head comp D VB VP NNS NP

denote head/non-head daughters.1 Since an HPSG C

D D D D D H

head comp DIE*D trans VB noun NNS C

derivation is represented by a set of B, a prob-

D D D D D D H

head comp D trans VB noun NNS

ability of assigned to sentence is deﬁned as

D D D D D H

head comp DFEGD VB NNS

D D D D D D H follows: head comp D VB NNS

Given the HPSG ) treebank as training data, the

¢ ¨

model parameters + are efﬁciently estimated using

).-/)

"!#"$ $*),+ 0 1 ¢ a dynamic programming algorithm for maximum %&(' entropy estimation (Miyao and Tsujii, 2002; Geman

and Johnson, 2002). 2 is a probability of a sequence of lexical en-

4 Evaluating HPSG parsing with

) ) tries, and is ) deﬁned as the product of unigram prob-

semantically annotated corpora

§¦ 3 ¦

abilities ) , where is a lexical entry assigned

to word 3 . We divided the probability into Our study aims toward the fair evaluation of deep

and 45 in order to accelerate the estimation linguistic parsers, thus we want to directly compare 2 of the probability model by using as a ref- the output of HPSG parsing with hand-annotated erence distribution (Miyao et al., 2003), because the test data. However, disagreements between the out-

direct estimation of 45 was computationally put of HPSG parser and the PropBank prevents us

expensive. from a direct comparison.

) - Feature function returns 1 when a cer- In the PropBank annotation, semantic arguments tain part of tuple is observed. Table 1 can occur in multiple syntactic realizations, as in the lists templates of feature functions used in the following example (Figure 4). disambiguation model, where a check means 1. He broke the window. that the corresponding element in the tuple is seen. For example, when we have a branching 2. The window broke.

¨76 ¨ ¨ ¨ ¨ ¨ ¨ ¤ head comp trans VB VP noun NNS NP , In the ﬁrst example, a semantic object appears in a the following feature functions return 1, while all syntactic object position, while in the second sentence it becomes the syntactic subject. This alterna- 1 A unary branching is represented by 8¤9*:<;¤=:¤>*=?:<@A=#B . tion is caused by two reasons: syntactic alternations 2 In this example, head comp and trans stand for the Head such as passive constructions and long-distance de- Complement Schema and a transitive verb. In our probabilistic model, lexical entry templates are more ﬁne-grained (as shown pendencies, and lexical alternations such as erga- in Section 5, a grammar has more than 1,000 templates), while tive verbs. It should also be noted that the assign- we used a simple example here. ment of argument labels have some arbitrariness.

¡ ¡ For example, Figure 5 shows the PropBank annota- penn prop tion for “The window broke into a million pieces.”, # words 8,539 8,496 where a phrase “a million pieces” is annotated with # lexical entry template 1,106 1,178 ARG3, not with ARG2. This is because ARG2 is # template per word 3.00 3.16 reserved for an instrument argument (e.g. “with a # features 50,158 52,151 rock”). However, the choice of selecting ARG2 or Size of the training data 124 MB 131 MB ARG3 for “a million pieces” is arbitrary. Existing Estimation time 68 min 51 min studies exploited statistical methods to mend these alternations and arbitrariness. Basically, deep linguistic parsers derived from Table 2: Specifications of the HPSG grammar and the Penn Treebank can handle syntactic alternations the disambiguation model owing to trace annotation in the treebank. However, lexical alternations and arbitrariness of assignments 5 Experimental results of argument labels will be a problem when we di- In this section, we evaluate the accuracy of HPSG rectly compare the output of an HPSG parser with parsing using the November 2002 release of Prop- the PropBank. Bank (Kingsbury and Palmer, 2002). An HPSG However, we can see that the remaining disagree- grammar was extracted from Section 02-21 and a ments are about the labels of argument labels. In disambiguation model was trained using the same general, we can assume that argument labels can data. Table 2 shows specifications of the grammar be uniquely determined if a syntactic class of the 3 and the disambiguation model, where the size of the predicate is given. In the example given in Sec- training data shows the file size of a compressed tion 2, “the window” always occurs in the object training data and the estimation time represents a

position when “broke” is transitive, while it appears

¨ user time required for estimating 4 . We in the subject position when it is intransitive. Since ¡ prepared two grammars for the evaluation: penn syntactic classes are expressed by lexical entries in was extracted from the Penn Treebank with the orig- HPSG, this indicates that we can establish a unique ¡ inal algorithm (Miyao et al., 2004), and prop was mapping from an HPSG lexical entry into PropBank extracted using the PropBank annotations for ar- semantic roles. gument/modifier distinction by a method similar to Following this idea, we developed a mapping Chen and Rambow (2003). That is, constituents an- from HPSG argument labels into PropBank argu- notated with ARG were treated as an argument ment labels. This mapping was developed with a ¡ in the grammar extraction. In penn, prepositional very simple algorithm as follows. We first com- phrases are basically treated as modifiers since we puted predicate-argument structures from an HPSG have no cue to detect argument/modifier distinc- treebank. We then compared the obtained predicate- tion in the original Penn Treebank. Section 02-21 argument structures with the PropBank annotations, was also used for developing HPSG-to-PropBank and for each pair of a surface form of a word and its mapping. Note that the PropBank annotation was syntactic class, the mapping from argument labels used only for this purpose, and was not used for of a predicate-argument structure into those of Prop- training a statistical disambiguation model. This is Bank was registered. When we found a conflict, that very different from existing methods of identifying is, multiple mappings were found for a pair, a map- PropBank-style annotations where they trained the ping found later was simply discarded. identification model using the PropBank. In the fol- Our method is much simpler than existing stud- lowing, Section 22 of the PropBank was used for ies, and it should be noted that PropBank was not the development of the parser, while Section 23 was used for training the probabilistic model or statis- used for the final evaluation. tical identifier. This might be a handicap for our The accuracy of HPSG parsing was measured evaluation, but this method can clearly show the against the core-argument annotations (i.e., ARG0, lower bound of the accuracy that has been attained ..., ARG5) of the PropBank. Each predicate-

by HPSG parsing. argument relation output by the parser was rep-

3There exist some exceptions as follows:

£¢¥¤§¦ ¨¢ ¦¨¤¦ was a predicate, was the label of an

“He opened the bottles.” argument position (i.e., one of ARG0, ..., ARG5),

£¢¥¤¦ “The can opener opens the bottles.” and ¨ ¢ was the head word of the argument of . In the PropBank, “he” is assigned ARG0, while “the can Each tuple was compared to the annotations in the opener” is assigned ARG2 (instrument). PropBank. We used a mapping table described in

LP LR UP UR with another work using LTAG (Chen and Rambow, ¡

penn 70.3 56.0 86.7 69.2 2003) because their target annotations were limited ¡ prop 68.3 59.0 85.6 73.9 to those localized in an elementary tree, consider- Gold parses 79.5 67.1 97.2 82.0 ing that their target annotations were 87% of core- arguments, our results are competitive with their results (82.57/71.41). Table 3: Accuracy of PropBank annotations (head words of core arguments, without HPSG-to- 6 Conclusion PropBank mapping) In this paper, the accuracy of HPSG parsing was

LP LR UP UR evaluated in terms of the identiﬁcation of predicate- ¡

penn 80.3 64.1 86.7 69.2 argument relations. By assuming unique mapping ¡ prop 79.6 68.7 85.6 73.9 from HPSG predicate argument structures into the Gold parses 91.2 76.9 97.2 82.0 PropBank annotation of semantic arguments, we could directly compare the output of an HPSG parser with PropBank. Despite not using Prop- Table 4: Accuracy of PropBank annotations (head Bank for the training of a disambiguation model, words of core arguments, with HPSG-to-PropBank the HPSG parser achieved a high accuracy compet- mapping) itive with the previous studies on the identiﬁcation of PropBank annotations. This result reveals the ac- Section 4 for mapping the argument labels of HPSG curate identiﬁcation of predicate-argument relations into the PropBank-style. by HPSG parsing. Table 3 shows the accuracy of semantic argu- Although this study directly compared the HPSG ments output by the HPSG parser without map- output with PropBank, we may require an addi- ping HPSG outputs to PropBank-style, while Ta- tional machine learning step as in the existing stud- ble 4 shows the accuracy with the HPSG-to- ies to obtain higher accuracy because the accu- PropBank mapping. LP/LR columns represent la- racy attained by gold parses showed a limitation beled precision/recall while UP/UR represent unla- of our approach. Another possibility is to directly beled precision/recall. “Labeled” here means the extract PropBank-style semantic representations by label of argument positions. That is, a predicate- reforming the grammar extraction algorithm (Chen argument relation was judged to be correct if and Rambow, 2003), and to estimate a disambigua-

tion model using the PropBank.

£¢¥¤§¦ ¨©¨ ¢ ¦ ¨¤ ¦©¨©¨ ¢ ¤ was correctly output. “Un- labeled” means that the head word of the argument References was correctly output regardless of the argument po-

Steven P. Abney. 1997. Stochastic attribute-value

£¢¥¤§¦ ¨ ¢ sition, i.e., and were correctly output. The “Gold parses” row represents the accuracy attained grammars. Computational Linguistics, 23(4). when correct HPSG derivations are given. That is, Collin F. Baker, Charles J. Fillmore, and John B. it represents the accuracy when Section 23 of the Lowe. 1998. The Berkeley FrameNet project. In HPSG treebank was given. This represents the up- Proc. COLING/ACL 1998, pages 86–90. per bound of this measure in this evaluation. Adam L. Berger, Stephen A. Della Pietra, and Vin- First of all, we can see that labeled preci- cent. J. Della Pietra. 1996. A maximum entropy sion/recall signiﬁcantly increased with the HPSG- approach to natural language processing. Com- to-PropBank mapping. This means that the low ac- putational Linguistics, 22(1):39–71. curacy of the naive evaluation (Table 3) was mainly Joan Bresnan, editor. 1982. The Mental Repre- due to the disagreements of the representation of se- sentation of Grammatical Relations. MIT Press, mantic structures. Cambridge, MA. As shown in Table 4, despite not employing the Michael Burke, Aoife Cahill, Ruth O’Donovan, PropBank for the machine learning of a disambigua- Josef van Genabith, and Andy Way. 2004.

tion model, the labeled precision/recall attained by Treebank-based acquisition of wide-coverage, ¡ prop were superior to an existing study using probabilistic LFG resources: Project overview, the Collins parser (75.9/69.6) (Gildea and Hock- results and evaluation. In Proc. IJCNLP-04 enmaier, 2003), and the results were approaching Workshop “Beyond Shallow Analyses”. existing studies on the same task using a CCG Aoife Cahill, Mairead McCarthy, Josef van Gen- parser (76.1/73.5) (Gildea and Hockenmaier, 2003). abith, and Andy Way. 2002. Parsing with PCFGs Although the results cannot directly be compared and automatic f-structure annotation. In Proc. 7th International Lexical-Functional Grammar Con- models of predicate-argument structure. In Proc. ference. 41st ACL, pages 359–366. Eugene Charniak. 2000. A maximum-entropy- Mark Johnson, Stuart Geman, Stephen Canon, inspired parser. In Proc. NAACL-2000, pages Zhiyi Chi, and Stefan Riezler. 1999. Estimators 132–139. for stochastic “uniﬁcation-based” grammars. In John Chen and Owen Rambow. 2003. Use of deep Proc. ACL ’99, pages 535–541. linguistic features for the recognition and label- Paul Kingsbury and Martha Palmer. 2002. From ing of semantic arguments. In Proc. EMNLP Treebank to PropBank. In Proc. 3rd LREC. 2003. Robert Malouf and Gertjan van Noord. 2004. Wide John Chen and K. Vijay-Shanker. 2000. Automated coverage parsing with stochastic attribute value extraction of TAGs from the Penn Treebank. In grammars. In Proc. IJCNLP-04 Workshop “Be- Proc. 6th IWPT. yond Shallow Analyses”. David Chiang. 2000. Statistical parsing with an Mitchell Marcus, Grace Kim, Mary Ann automatically-extracted tree adjoining grammar. Marcinkiewicz, Robert MacIntyre, Ann Bies, In Proc. 38th ACL, pages 456–463. Mark Ferguson, Karen Katz, and Britta Schas- Stephen Clark and James R. Curran. 2003. Log- berger. 1994. The Penn Treebank: Annotating linear models for wide-coverage CCG parsing. In predicate argument structure. In ARPA Human Proc. EMNLP 2003, pages 97–104. Language Technology Workshop. Stephen Clark, Julia Hockenmaier, and Mark Steed- Yusuke Miyao and Jun’ichi Tsujii. 2002. Maxi- mum entropy estimation for feature forests. In man. 2002. Building deep dependency structures with a wide-coverage CCG parser. In Proc. 40th Proc. HLT 2002. ACL, pages 327–334. Yusuke Miyao, Takashi Ninomiya, and Jun’ichi Tsujii. 2003. Probabilistic modeling of argument Michael Collins. 1999. Head-Driven Statistical structures including non-local dependencies. In Models for Natural Language Parsing. Ph.D. Proc. RANLP 2003, pages 285–291. thesis, Univ. of Pennsylvania. Yusuke Miyao, Takashi Ninomiya, and Jun’ichi Anette Frank, Louisa Sadler, Josef van Genabith, Tsujii. 2004. Corpus-oriented grammar develop- and Andy Way. 2003. From treebank resources ment for acquiring a Head-driven Phrase Struc- to LFG f-structures: Automatic f-structure an- ture Grammar from the Penn Treebank. In Proc. notation of treebank trees and CFGs extracted IJCNLP-04. from treebanks. In Anne Abeille, editor, Build- Carl Pollard and Ivan A. Sag. 1994. Head- ing and Using Syntactically Annotated Corpora, Driven Phrase Structure Grammar. University of pages 367–389. Kluwer Academic Publishers. Chicago Press. Stuart Geman and Mark Johnson. 2002. Dy- Stefan Riezler, Tracy H. King, Ronald M. Kaplan, namic programming for parsing and estimation of Richard Crouch, John T. Maxwell III, and Mark stochastic uniﬁcation-based grammars. In Proc. Johnson. 2002. Parsing the Wall Street Jour- 40th ACL, pages 279–286. nal using a Lexical-Functional Grammar and dis- Daniel Gildea and Julia Hockenmaier. 2003. Iden- criminative estimation techniques. In Proc. 40th tifying semantic roles using Combinatory Cate- ACL. gorial Grammar. In Proc. EMNLP 2003. Yves Schabes, Anne Abeille,´ and Aravind K. Joshi. Daniel Gildea and Daniel Jurafsky. 2002. Auto- 1988. Parsing strategies with ‘lexicalized’ gram- matic labeling of semantic roles. Computational mars: Application to tree adjoining grammars. In Linguistics, 28(3):245–288. Proc. 12th COLING, pages 578–583. Daniel Gildea and Martha Palmer. 2002. The ne- Mark Steedman. 2000. The Syntactic Process. The cessity of parsing for predicate argument recog- MIT Press. nition. In Proc. 40th ACL. Fei Xia. 1999. Extracting tree adjoining grammars Julia Hockenmaier and Mark Steedman. 2002a. from bracketed corpora. In Proc. 5th NLPRS. Acquiring compact lexicalized grammars from a cleaner treebank. In Proc. 3rd LREC. Julia Hockenmaier and Mark Steedman. 2002b. Generative models for statistical parsing with Combinatory Categorial Grammar. In Proc. 40th ACL, pages 335–342. Julia Hockenmaier. 2003. Parsing with generative