Towards a Semantic Annotation of English Television News - Building and Evaluating a Constraint Grammar FrameNet Eckhard Bick Institute of Language and Communication, University of Southern Denmark Campusvej 55, DK 5230 Odense M [email protected] Abstract The UCLA Communications Studies Archive (UCLA CSA) is a so-called monitor This paper presents work on the semantic corpus of television news, where newscasts annotation of a multimodal corpus of from a large number of channels are recorded English television news. The annotation is daily in high-quality video mode, amounting performed on the second-by-second- to ~ 150.000 hours of recorded news, and aligned transcript layer, adding verb frame growing by 100 programs a day (DeLiema, categories and semantic roles on top of a Steen & Turner 2012). To date only English morphosyntactic analysis with full dependency information. We use a rule- language channels have been targeted, but the based method, where Constraint Grammar author's institution has plans to join the mapping rules are automatically generated project with matching data for first the from a syntactically anchored Framenet Scandinavian languages and German, then with about 500 frame types and 50 further European languages. This paper semantic role types. We discuss design focuses on the linguistic annotation of the decisions concerning the Framenet, and time-stamp-aligned textual layer of the evaluate the coverage and performance of corpus. Optimally, such annotation should the pilot system on authentic news data. address the following issues • robustness in the face of spoken language 1 Introduction and methodological data focus • low error rate for basic morphosyntactic Because the communicative information annotation contained in a multi-modal corpus is • conservation/integration of non-linguistic distributed across different channels, it is meta-annotation (speaker, source, time ...) much more difficult to process automatically • unified tag system across languages to than a classical text corpus. Large multi- facilitate comparative studies modal corpora, in particular, constitute a • a semantic annotation layer to support challenge to quantitative-statistical higher-level communicative studies exploration or even comparative qualitative studies, because they may be too big for A well-established annotation format is the complete inspection, let alone extensive assignment of feature-attribute pairs to word manual mark-up. In some types of multi- tokens, expressed as tag fields and convertible modal corpora, however, such as a film- to xml structures. A list of tokens with tags subtitle corpus, or the television news corpus guarantees that all information is local and that is the object of this study, aligned easy to filter or search, with meta-information transcripts or captions offer at least a partial carried along on separate lines between solution, because this textual layer can be tokens. For the tagging/parsing task as such used to search the corpus and extract we have chosen the Constraint Grammar (CG) matching sections for closer inspection, formalism (Karlsson et al. 1995, Bick 2000) comparison or even quantitative analysis. which has proven robust enough for a large 60 Copyright 2012 by Eckhard Bick 26th Pacific Asia Conference on Language,Information and Computation pages 60–69 variety of corpus annotation task, including speech own semantic frame. Depending, for instance, on annotation (Bick 2012). An added advantage is the the number of obligatory arguments, several fact that comparable CG systems, with similar tag valency or semantic frames may share the same sets and annotation conventions, already exist not verb sense, but two different verb senses will only for English, but also for many other European almost always differ in at least one syntactic or languages, among them almost all Germanic and semantic aspect of their argument frame - Romance languages (http://visl.sdu.dk/ guaranteeing that all senses can in principle be constraint_grammar.html). CG systems are disambiguated exploiting a parser's argument tags modular, hierarchical sets of rule-based grammars and dependency links. targeting different linguistic levels, and while higher level analysis can be performed within the Currently, the EngGram FrameNet (EFN) contains same formalism, it is a challenging task. Thus, 7820 verb sense for 4774 verb types, with 10.800 most of the existing CG systems perform only valency frames. For each frame, we provide a list morphosyntactic and dependency annotation, with of arguments with the following information: some notable exceptions in the area of NER and semantic role annotation. The system that comes 1. Thematic role (Table 1) closest to the task at hand, is the Danish DanGram 2. Syntactic function (Table 2) system which implements a framenet-based verbal 3. Morphosyntactic form (Table 4) classification and semantic role annotation (Bick 4. for np's, a list of typical semantic prototypes 2011), with a category inventory of ~500 verb to fill the slot (Table 3) frames and ~50 semantic roles. For our present 5. An English language gloss / skeleton sentence task, we have attempted to port lexical material from this system, and adopted its verb For about 2/3 of the frames, a best-guess link to a classification scheme, which in turn was inspired BFN verb sense is also provided, based on semi- by the VerbNet classes proposed by Kipper et al. automatic valency matches on EngGram-parsed (2006), ultimately with roots in (Levine 1993), and BFN example sentences. a smaller and thus more tractable granularity than Our FrameNet uses ca. 35 core thematic roles PropBank (Palmer et al. 2005). Our semantic role (or case/semantic roles, Fillmore 1968), with a inventory, following the one implemented for further 10-15 adverbial roles that are added by the Portuguese by (Bick 2007), is also much smaller semantic tagger based on syntactic context without than PropBank's, the rationale being that medium- the need of a verb frame entry (e.g. subclause sized category sets allow for a reasonable level of function based on conjunction type). These roles abstraction compared to the underlying lexical are far from evenly distributed in running text. items, and by roughly matching the granularity of Table 1 provides some live corpus data, showing other linguistic abstractions (syntactic function that the top 5 roles account for over half of all role inventory, PoS/morphological categories) are well taggings in running text. Note that the distribution suited to be integrated with the latter in automatic is for all roles, not just verb frame roles, since the disambuguation systems. semantic tagger also tags some semantic relations based on nominal or adjectival valency (e.g. 2 Frame role distinctors: valency, abolition of X, full of Y). syntactic function and semantic classes Table 1: Top 25 Semantic (Thematic) Roles In this vein, the distinctional backbone of our Thematic Role in corpus frame inventory are syntactic valency frames like §TH Theme 21.91% <vt> (monotransitive), <vdt> (ditransitive), §ATR Attribute 13.76% <to^vp-forward> (prepositional transitive with the §AG Agent 7.07% preposition “to” and a verb-incorporated 'forward'- §LOC Location 6.78% adverb). Each of these valency frames is assigned 1 §LOC-TMP Point in time 5.44% at least one (or more ) verb senses, each with its §PAT Patient 4.20% §DES Destination/Goal 3.56% 1In 717 cases, there is more than one role combination for the §MES Message 3.13% same sense with the same valency, and in 11.2% multiple verb senses share the same valency frame, reflecting cases where semantic prototype or other slot filler information is needed to make the distinction. 61 §COG Cognizer 3.00% @OC Object complem. §SP Speaker 2.58% ATR (80.7%) > RES §BEN Beneficiary 2.48% The prototypical verb frame consists of a full verb §ID Identity 2.16% and its nominal, adverbial or subclause §TP Topic 1.97% complements. Like most other languages, §ACT Action 1.91% however, English has also verb incorporations that §INC Incorporated particle 1.91% are not, in the semantical sense, complements. The §EXP Experiencer 1.73% simplest kind are adverb incorporates, which we §RES Result 1.49% mark in the valency frame, but not in the argument §STI Stimulus 1.37% list: §FIN Purpose 1.31% §EV Event 1.56% give up - <vi-up>, turn off - <vt-off> §CAU Cause 0.98% More complicated are support verb constructions, §ORI Origin 0.97% where the semantic weight and - to a certain degree §REC Recipient 0.80% - valency reside in a nominal element, typically a §EXT-TMP Duration 0.74% noun that syntactically fills a (direct or §INS Instrument/Tool 0.62% prepositional) object slot, but semantically orchestrates the other complements. While adverb Other roles: §COND condition, §COM co-agent, §HOL incorporates are marked as such by the EngGram whole, §VOC vocative, §COMP comparison, §SOA state of affairs, §MNR manner, §PART part, §VAL parser already at the syntactic level (@MV<), noun value, §ASS asset, §EXT extension, §PATH path, §DON or adjective incorporates receive an ordinary donor, §CONT contents, §CONC concession, §REFL syntactic tag (@ACC, @SC), but are marked with reflexive, §POSS possessor, §EFF effect, §ROLE role, an empty §INC (incorporate) role tag at the §MAT material, §ROLE role, §DES-TMP temp. semantic level. This is why, currently, about 14.6% destination, §ORI-TMP temp. origin of EFN valency entries include incorporated Even in a case-poor language like English, we material, but the percentage of non-adverbial found some clear likelihood relations between incorporates is still small (about a 1/10 of all thematic roles and syntactic functions (table 2). incorporations). Thus, agents (§AG, §COG, §SP) are typical The examples below also show the subject roles, while patients (§PAT), messages corresponding valency tags, where 'vt' means (§MES) and results (§RES) are typical direct transitive and 'vi' intransitive. Governed object roles, and recipients (§REC) and prepositions are prefixed (e.g. <of^...>) and beneficiaries (§BEN) call for dative object incorporated material is postfixed (e.g.