EACL 2009
Proceedings of the EACL 2009 Workshop on Language Technology and Resources for Cultural Heritage, Social Sciences, Humanities, and Education
LaTeCH – SHELT&R 2009
30 March 2009 Megaron Athens International Conference Centre Athens, Greece Production and Manufacturing by TEHNOGRAFIA DIGITAL PRESS 7 Ektoros Street 152 35 Vrilissia Athens, Greece
c 2009 The Association for Computational Linguistics
Order copies of this and other ACL proceedings from:
Association for Computational Linguistics (ACL) 209 N. Eighth Street Stroudsburg, PA 18360 USA Tel: +1-570-476-8006 Fax: +1-570-476-0860 [email protected]
ii Preface
The LaTeCH–SHELT&R workshop aims to present in one forum two strands of research in language technology, which we believe have many common concerns, but also complementary viewpoints.
Museums, archives, and libraries around the world maintain large collections of cultural heritage objects, such as archaeological artefacts, sound recordings, historical manuscripts, or preserved animal specimens. Large scale digitization projects are currently underway to make these collections more accessible to the public and to research. The natural next step after digitization is the development of powerful tools to search, link, enrich, and mine the digitized data. Language technology has an important role to play in this endeavor, even for collections which are primarily non-textual, since text is the pervasive medium used for metadata. Two previous LaTeCH (Language Technology for Cultural Heritage) workshops (at ACL 2007 in Prague and LREC 2008 in Marrakech) have shown that there is an interest among language technology researchers in providing intelligent infrastructure and tools for working with cultural heritage data.
Similarly, in research in the Social Sciences, Humanities and Education (SHE), text – and speech, i.e., language – are central as both primary and secondary research data sources. In today’s world, the normal mode of access to text, speech, images and video is in digital form. Modern material is born digital, and, as already mentioned, older material is being digitized on a vast scale in cultural heritage and digital library projects. Language technology and language resources has an equally important role to play here, as in the cultural heritage area, and for more or less the same purposes. A clear sign of this is the newly launched European research infrastructure initiative CLARIN, which addresses exactly the use of language technology and language resources as research infrastructure in the humanities and social sciences. Against this background, it now seems natural to add a component to the workshop reflecting this development: SHELT&R (Language Technology and Resources infrastructure for text- based research in the Social Sciences, Humanities and Education).
The CH and SHE domains are not mere passive consumers of ready-made language technology solutions. Rather, they make up interesting and challenging testbeds, where the robustness and the generality of existing language technology are subjected to the acid test of messy and multilingual reality, more so than in many other application areas, since they have to deal with, inter alia, historical, non-standardized language varieties in addition to a number of modern standard languages. Our workshop thus aims to foster interaction between researchers working on all aspects of language technology applied to the CH and SHE domains, and experts from institutions who are testing deployed technologies and formulating improved use cases.
The papers accepted for the LaTeCH–SHELT&R workshop after a thorough peer-review process give a good sense of the current breadth of this exciting and expanding area. We are happy that our keynote speakers, Dr. Martin Doerr and Dr. Tamas´ Varadi,´ agreed to join us and deliver excellent topics for a complete workshop program. We would like to thank all authors who submitted papers for the hard work that went into their submissions. We are also extremely grateful to the members of the programme committee for providing thorough reviews and multi-faceted input.
Some of the workshop costs were covered from a project grant awarded to Lars Borin by the Swedish Research Council (VR Dnr 2007-7430: Safeguarding the future of Sprakbanken˚ ), which is hereby gratefully acknowledged.
Lars Borin • Piroska Lendvai
iii
LaTeCH – SHELT&R 2009
Workshop Chairs: Lars Borin, University of Gothenburg (Sweden) Piroska Lendvai, Tilburg University (The Netherlands)
Organizing Committe: Piroska Lendvai (Co-chair), Tilburg University (The Netherlands) Lars Borin (Co-chair), University of Gothenburg (Sweden) Antal van den Bosch, Tilburg University (The Netherlands) Martin Reynaert, Tilburg University (The Netherlands) Caroline Sporleder, Saarland University (Germany)
Program Committee Members: Ion Androutsopoulos, Athens University of Economics and Business (Greece) Timothy Baldwin, University of Melbourne (Australia) David Bamman, Perseus (USA) Lars Borin, University of Gothenburg (Sweden) Antal van den Bosch, Tilburg University (The Netherlands) Andrea Bozzi, ILC-CNR, Pisa (Italy) Paul Buitelaar, DERI Galway (Ireland) Kate Byrne, University of Edinburgh (UK) Claire Cardie, Cornell University (USA) Paul Clough, Sheffield University (UK) Milena P. Dobreva, CDLR, University of Strathclyde (UK) Mick O’Donnell, Universidad Autonoma de Madrid (Spain) Claire Grover, University of Edinburgh (UK) Ben Hachey, University of Edinburgh (UK) Erhard Hinrichs, Tubingen¨ University (Germany) Graeme Hirst, University of Toronto (Canada) Christer Johansson, University of Bergen (Norway) Jaap Kamps, University of Amsterdam (The Netherlands) Dimitrios Kokkinakis, University of Gothenburg (Sweden) Stasinos Konstantopoulos, NCSR Demokritos (Greece) Piroska Lendvai, Tilburg University (The Netherlands) Christina Lioma, University of Leuven (Belgium) Anke Ludeling,¨ Humboldt University (Germany) Veronique Malaise,´ Free University of Amsterdam (The Netherlands) Steven van der Mije, Trezorix (The Netherlands) John Nerbonne, Rijksuniversiteit Groningen (The Netherlands) Marco Pennacchiotti, Saarland University/Yahoo! Research (Germany) Georg Rehm, vionto GmbH, Berlin (Germany) Martin Reynaert, Tilburg University (The Netherlands) Michael Rosner, University of Malta (Malta) Caroline Sporleder, Saarland University (Germany) Tamas´ Varadi,´ Hungarian Academy of Sciences (Hungary) Andreas Witt, Tubingen¨ University (Germany) Svitlana Zinger, Eindhoven University of Technology (The Netherlands)
v
Table of Contents
Content Analysis of Museum Documentation in a Transdisciplinary Perspective Guenther Goerz and Martin Scholz ...... 1
An Intelligent Authoring Environment for Abstract Semantic Representations of Cultural Object Descrip- tions Stasinos Konstantopoulos, Vangelis Karkaletsis and Dimitris Bilidas ...... 10
Multiple Sequence Alignments in Linguistics Jelena Prokic,´ Martijn Wieling and John Nerbonne ...... 18
Evaluating the Pairwise String Alignment of Pronunciations Martijn Wieling, Jelena Prokic´ and John Nerbonne ...... 26
A Web-Enabled and Speech-Enhanced Parallel Corpus of Greek-Bulgarian Cultural Texts Voula Giouli, Nikos Glaros, Kiril Simov and Petya Osenova ...... 35
The Development of the “Index Thomisticus” Treebank Valency Lexicon Barbara McGillivray and Marco Passarotti ...... 43
Applying NLP Technologies to the Collection and Enrichment of Language Data on the Web to Aid Linguistic Research Fei Xia and William Lewis ...... 51
Instance-Driven Discovery of Ontological Relation Labels Marieke van Erp, Antal van den Bosch, Sander Wubben and Steve Hunt...... 60
The Role of Metadata in the Longevity of Cultural Heritage Resources Milena Dobreva and Nikola Ikonomov ...... 69
vii
Conference Program
Monday, March 30, 2009
9:00–9.15 Opening
9:15–10:15 Invited Talk by Martin Doerr
10:15–10:30 Moderated discussion
10:30–11:00 Coffee break
11:00–11:30 Content Analysis of Museum Documentation in a Transdisciplinary Perspective Guenther Goerz and Martin Scholz
11:30–12:00 An Intelligent Authoring Environment for Abstract Semantic Representations of Cul- tural Object Descriptions Stasinos Konstantopoulos, Vangelis Karkaletsis and Dimitris Bilidas
12:00–12:20 Multiple Sequence Alignments in Linguistics Jelena Prokic,´ Martijn Wieling and John Nerbonne
12:20–12:45 Evaluating the Pairwise String Alignment of Pronunciations Martijn Wieling, Jelena Prokic´ and John Nerbonne
12:45–14:00 Lunch
14:00–15:00 Invited talk by Tamas´ Varadi´
15:00–15:20 A Web-Enabled and Speech-Enhanced Parallel Corpus of Greek-Bulgarian Cultural Texts Voula Giouli, Nikos Glaros, Kiril Simov and Petya Osenova
15:20–15:40 The Development of the “Index Thomisticus” Treebank Valency Lexicon Barbara McGillivray and Marco Passarotti
15:40–16:00 Applying NLP Technologies to the Collection and Enrichment of Language Data on the Web to Aid Linguistic Research Fei Xia and William Lewis
16:00–16:30 Coffee break
ix Monday, March 30, 2009 (continued)
16:30–16:55 Instance-Driven Discovery of Ontological Relation Labels Marieke van Erp, Antal van den Bosch, Sander Wubben and Steve Hunt
16:55–17:20 The Role of Metadata in the Longevity of Cultural Heritage Resources Milena Dobreva and Nikola Ikonomov
17:25–18:00 Moderated discussion; closing
x Content Analysis of Museum Documentation with a Transdisciplinary Perspective
Gunther¨ Goerz, Martin Scholz University of Erlangen-Nuremberg, Computer Science Department (8) Erlangen, Germany [email protected]
Abstract With the introduction of information technol- ogy in museums and cultural heritage institu- In many cases, museum documentation tions, such records have been stored in (relational) consists of semi-structured data records database systems and content management sys- with free text fields, which usually refer tems. At the beginning — with the exception to contents of other fields, in the same of bibliographic records — there were no meta- data record, as well as in others. Most of data standards at all in the museum world. Since these references comprise of person and the 1990s, many metadata schemata have been place names, as well as time specifica- proposed for the field of cultural heritage, some tions. It is, therefore, important to rec- with very detailed classification features for spe- ognize those in the first place. We report cific object types1. There is still an active dis- on techniques and results of partial pars- cussion about metadata schemata and their stan- ing in an ongoing project, using a large dardisation, as can be seen with recent proposals database on German goldsmith art. The for CDWA Lite, museumdat and their combination texts are encoded according to the TEI (Stein and Coburn, 2008). guidelines and expanded by structured de- Today, access to museum documentation via the scriptions of named entities and time spec- World Wide Web has become a matter of course, ifications. These are building blocks for in particular, if the documentation has been the event descriptions, at which the next step result of publicly funded research projects. Nat- is aiming. The identification of named en- urally, printed editions are still a very important tities allows the data to be linked with var- medium of publication. However, in many cases ious resources within the domain of cul- the data are too voluminous, which means only tural heritage and beyond. For the latter abridged versions are published in print, while case, we refer to a biological database and the full data are available only in digital form. present a solution in a transdisciplinary Web access allows many means to retrieve and perspective by means of the CIDOC Con- print the data, with very little cost involved. Us- ceptual Reference Model (CRM). ing controlled language defined in terminologies and formal ontologies, different forms of “intelli- 1 Specific Goals of Content Analysis gent search” come within reach as well as inter- When we speak of museum documentation, we active evaluation and visualisation methods. But address a wide variety of document types. First it is not only access to the data alone; interactiv- of all, there are acquisition and inventory lists or ity opens up possibilites for Wiki-style annotation index cards, which contain more or less detailed and scholarly communication, as well as forums records of museum objects. Often these are ac- for the general public. Furthermore, the technol- companied by photographs, restoration records, ogy provides methods to link the data with other and further archival records. If curators prepare resources, e.g. authority files containing biograph- exhibitions, usually they provide catalogs by com- ical or geographical data. piling data from sources, such as those just men- tioned, and by contributing short articles on the ex- 1cf. Getty Foundation’s Metadata Crosswalk http: //www.getty.edu/research/conducting_ hibits. Last but not least there are scholarly mono- research/standards/intrometadata/ graphs on museum objects. crosswalks.html ;visited 03.12.2008.
Proceedings of the EACL 2009 Workshop on Language Technology and Resources for Cultural Heritage, Social Sciences, Humanities, and Education –LaTeCH – SHELT&R 2009, pages 1–9, Athens, Greece, 30 March 2009. c 2009 Association for Computational Linguistics
1 A common situation in museum documenta- an example between biology and cultural history. tion is characterized by the fact that it is cen- tered around museum objects, i.e. there is a 2 Characteristics of the Data database system or content management system, 2 which contains structured descriptions of museum We are working with data which resulted from a objects and further information about their cre- project on goldsmith art in Nuremberg, executed ators, provenance, use, and so forth, according at the German National Museum, providing de- to given descriptive and administrative metadata scriptions of more than 6700 objects, 2290 artists, schemata. Besides fields in such data records en- many locations, etc. Furthermore, with the mu- forcing (more or less strictly defined) data types, seum’s content management system we can access e.g. for inventory numbers, there are free text many more data records on sculptures and paint- fields which contain important background infor- ings — with a particular emphasis on the work of mation about persons, objects, materials, stylistic Albrecht Durer¨ — up to 1800. The latter corpora festures, etc. without any further tagging. Ba- were accessed primarily to verify the general use- sically, the free text fields are open for any kind fulness of the approach that will be presented in of information which cannot be expressed in the the following sections. strictly defined parts of the schema. Therefore, For many projects in the field of cultural her- overall, the given data records at best provide a itage in Germany, a condition for public fund- 3 semi-structured representation. ing has been to use the MIDAS data schema (Heusinger, 1989) in combination with a spe- The free text fields and their relations to other cific database implementation (HiDA). MIDAS fields, in particular, indicate a clear need for con- defines a framework of record types with appro- tent analysis. Firstly, named entitities must be priate properties for terms (thesauri), time, place, identified, in particular person and geographic artists, other persons and organizations, objects, place names. For instance, there may be a data content and signs, events, sources, and adminis- field for the creator of a work of art and another trative data. The goal of MIDAS was to establish one for the place where this work was created, ad- a de facto standard based on the current documen- ditionally one or more free text fields which talk tation practice in museums. Depending on what about the artist’s family relations, when he came to is to be documented, the appropriate record types the mentioned place and how long he stayed there, can be selected. HiDA is a data administration sys- etc. As this example indicates, at least a second tem, which provides a graphical user interface for type of linguistic expressions, time specifications data input, editing, and search; it stores the records in a variety of forms, ought to be recognized. In not in a database system, but in a system of files, the future, we would like to identify event descrip- one for each type, in a proprietary format. For tions and how they are related among each other, this reason and problems in handling the user in- for which the recognition of named entitites and terface, many HiDA-encoded data are now being time specifications is a first step. converted to an XML representation. For the free In the following sections we describe our ap- texts, we decided to follow the encoding rules of proach to address these problems. The next sec- the Text Encoding Initiative (TEI) (Ide and Vero- tion outlines characteristic features of the data nis, 1995)4 for text bodies. with a reflection on their typicality. Section three The actual encoding of the XML-transformed is the central technical part presenting the shallow data sets is still very close to HiDA as far as text analysis techniques we use — word class tag- the “classes” and properties are concerned. Cur- ging, recognition of temporal specifications, place rently, the data are in the process of being trans- and person names — and the utilization of name formed to the emerging museumdat/CDWA Lite authorities for lexical and semantic enrichment. 2 In the fourth section we show how the results Research project “WissKI — Wissenschaftliche Kom- munikationsInfrastruktur”; funding provided by the German achieved so far can be used to construct event- Research Council (DFG) based shallow semantic representations related to 3Acronym for “Marburger Informations-, the CIDOC CRM. Furthermore, the CRM is also Dokumentations- und Administrations-System”, not to be confused with the MIDAS heritage standard in the UK. the key to transdisciplinary approaches in museum 4Website: http://www.tei-c.org/index.xml ; documentation as outlined in the final section with visited 15.12.2008
2 standard (Stein and Coburn, 2008)5, which in turn tags of the module namesdates for person and is compatible with CIDOC’s Conceptual Refer- place names. For processing, all texts in the free ence Model (Doerr, 2003)6. The CRM is the for- text fields of a MIDAS file — e.g., the “object” file mal reference ontology, which defines the con- containing all object descriptions in the “database” ceptual background for the semantic representa- — are merged in a single multi-text TEI file. Each tions resulting from content analysis. We refer to text from a data field is represented as a
3 Content Analysis: Shallow Parsing and First of all, the texts encoded conforming to the Semantic Representation TEI guidelines are annotated with word class tags and lemmata (base forms) by a POS tagger. Lem- The texts contained in the free text fields are en- matisation is very useful in languages with a rich coded with the TEI Lite tag set, supplemented by inflectional system, such as German. For POS tag- ging, we use the Stuttgart TreeTagger9 with the 5cf. slide set by Georg Hohmann: http: //www8.informatik.uni-erlangen.de/IMMD8/ STTS tagset which provides categories for Ger- Services/transdisc/cidoc2008_hohmann.pdf man words and delimiters. ; visited 03.12.2008 6The actual version of the ISO standard and a lot of 8based on previous work by (Tantzen, 2004). accompanying materials can be retrieved from http:// 9Institute for Automatic Language Processing of the cidoc.ics.forth.gr/ ; visited 03.12.2008. University of Stuttgart. The tagger is available at http: 7http://www8.informatik.uni-erlangen. //www.ims.uni-stuttgart.de/projekte/ de/IMMD8/Services/cidoc-crm/ ; visited 05.02. corplex/TreeTagger/DecisionTreeTagger. 2009 html ; visited 03.12.2008.
3 The resulting TEI tags express morphosyntac- incompatible types as in die Stadte¨ Munchen¨ und tic descriptions. Of particular interest are the tags Berlin und Finnland (the cities Munich, Berlin and
4 small and have been assembled manually. For Pattern Example first and surnames, several name lists were com- s Jamnitzer piled into one dictionary file from lists available s g Jamnitzer II via Name Authorities and also from the WWW. f + s Hans Jamnitzer To recognize person names containing very rare f + g c s *Hans II von Jamnitzer first and surnames, as well as writing variants f + g s Hans II Jamnitzer which are not contained in the lexicon, we use a f + c s *Hans von Jamnitzer system of syntactic and semantic cues — based on f + g Hans II statistical analyses of the texts — which indicate f + Hans the occurence of a name at a specific location (cf. s , f + g Jamnitzer, Hans II table). s , f + c *Jamnitzer, Hans von + syntax of the trigger Example s , f g c *Jamnitzer, Hans II von + profession name Goldschmied Samuel Klemm s , f Jamnitzer, Hans appellation name Frau Martha preposition relation name mit Meister Silvester Table 2: Recognized name patterns with examples showing the name of the goldsmith “Hans II Jamnitzer”. s stands for possessive pron. rel. name Seine Tochter Katharina surname, f for forename, c for copula and g for generation relation des/der name Tochter des Christian Mahler particle. The ‘+’ sign expresses one or more occurences; the von relation name Sohn von Peter v. Quickelberg asterisk indicates that the name has been modified to fit the : relation name Lehrling: Lang, Joh. Christoph pattern with “von”.
Table 1: Rules for person name triggers. Words to be in- serted for profession, appellation and relation are taken from strengths and weaknesses. Up to now, we have hand-made lexica. used the following interfaces — however, fur- ther interfaces are in preparation: BGN: Board Statistical analysis of the goldsmith corpus has on Geographic Names (German places File)12, given clear evidence for three groups of words Diskus “Geographie-Datei” (distributed with MI- whose occurrence indicates an immediate follow- DAS)13, Orbis Latinus (Graesse)14, Getty TGN ing person name: Appellations of professions, ap- (Thesaurus of Geographic Names)15, PKNAD pellations plus titles, and relations between per- (Person Names) by prometheus e.V.16, and Getty sons. A relation between persons is regarded as a ULAN (United List of Artist Names)17 cue only if certain particles occur immediately be- There are two modes of use for name authorities fore or after it. The word sequence “Tochter von” in the process of named entity recognition: (daughter of) is a good example of such a cue for a subsequent person name. 1. Decision making: The data are used as dic- In a first step, the name parts and the cues are tionaries for the person name and place name labelled separately. In a second pass, whenever a parsers. cue or a name part is encountered, an algorithm to assemble the parts into complete person names is 2. Enrichment with metadata in a second phase run. It tries to match the current word sequence once the named entities are identified. with different patterns of name parts which con- As there are not yet unique formats and inter- stitute valid person names, i.e. it applies different finite state machines11 to the word sequence. The 12http://earth-info.nga.mil/gns/html/ longest sequence recognized by a finite state ma- namefiles.htm ; visited 17.12.2008 13http://museum.zib.de/ chine is assumed to be a name (see Table 2). museumsvokabular/index.php?main=download ; visited 17.12.2008 3.5 Name Authorities 14http://www.columbia.edu/acis/ets/ Graesse/contents.html ; visited 17.12.2008 To achieve a normalization of appellations, per- 15http://www.getty.edu/research/ son and place names are looked up in name au- conducting_research/vocabularies/tgn/ ; visited 17.12.2008 thorities. There are several authorities, none of 16http://www.prometheus-bildarchiv. which can claim completeness, and each has its de/index.php?id=56\&L=0\&skin=0 ; visited 17.12.2008 11Finite State Machines are formal automata which recog- 17http://www.getty.edu/research/ nize regular expression patterns; i.e., both notions are equiv- conducting_research/vocabularies/ulan/ alent. ; visited 17.12.2008
5 faces for the mentioned name authorities, we im-
3.6 TEI-Encoding of Named Entities 4 Building Blocks for Event Recognition Temporal expressions are encoded with the With parsing results for person and place names
6
Figure 1: Parsing result: annotated text in TEI encoding. (Layout has been rearranged for readability.)
7 lar in combination with text mining.18 In previous these resources use idiosyncratic data formats just work (Fischer et al., 1996; Bucher¨ et al., 2002), we as MIDAS mentioned above. At best, they refer augmented a chart-based chunk parser with an in- to a formal representation of their respective do- cremental construction procedure for (partial) Dis- main, in terms of a so-called “formal domain on- course Representation Structures (DRSs). DRSs tology”, a representative hierarchical structure of are semantic representations which contain a list concepts, properties and constraints of the domain. of discourse referents, introduced by named enti- However, to satisfy the desideratum of linking di- ties or definite noun phrases, and a body, which verse data collections, an intermediate level of in- consists of a possibly complex logical form repre- teroperability is required. A well proven approach senting the meaning of the given part of speech19. for such information integration tasks is to link the For events, we use a neo-Davidsonian represen- different domain ontologies to a generic reference tation, i.e., the corresponding verb is a one-place ontology, which contains just the fundamental and predicate whose argument is a discourse referent most general concepts and properties for a wide representing an event, conjoined with binary re- variety of applications. In fact, for the field of lations for the thematic roles. For example, the cultural heritage, CIDOC’s Conceptual Reference sentence “Albrecht Durer¨ painted a self-portrait Model (CRM) is such a reference ontology. It is in 1500 in Nuremberg” would get a semantic rep- worthwhile to notice that, among other things, the resentation in which — extremely simplified — CRM emphasizes the event-driven perspective, in e would be the discourse referent for the event, fact, events are the glue in CRM which connects paint(e) the representation of the event, and ac- all documentation elements. As a first step, we tor(e,a), object(e,s), time(e,1500), etc. constitute have already implemented a generator for CRM the body, where a and s are the discourse referents instances from TEI-conformant texts with named for the artist and the self-portrait, resp. DRSs are entity annotations. reaching beyond sentence limits and can in prin- ciple be combined into larger and larger discourse 5 Transdisciplinary Aspects structures. Therefore, they are appropriate repre- sentations on which reference resolution mecha- Coming back to our project on goldsmith art doc- nisms, such as those described in (Fischer et al., umentation, we recognize clues in the data, which 1996) can be built. In our current work, a cen- point beyond the domain of cultural history: there tral activity is to port this method and its imple- are goblets and centerpieces (epergnes) showing mentation to the museum documentation domain sculptered animals, such as lizards and beetles. and enrich it by collocational analysis as in (Smith, Two of the documented objects exhibit a beau- 2002). tiful stag beetle, which induced interesting ques- The representation of events is not only an ex- tions about those insects, not only on their icono- tremely important key to content analysis, but also graphic significance, but also on their determina- the pivot which connects various objects, persons, tion and classification in biology, the distribution places, with each other, making a variety of con- of species, etc. This illustrates that there is a nections explicit, which are implicitly contained need to connect with further knowledge sources, in the data fields and free texts of records of dif- such as resources from biology, biodiversity re- ferent types. It, therefore, becomes an obvious search, etc. For example, we may want to con- goal to enrich such relational structures with fur- sult a database such as BIODAT, maintained by ther information elements from other cultural her- the natural history museum Koenig in Bonn. Con- itage resources — beyond name authorities. In our sidering the completely different scientific back- particular application, access to Getty’s Art and ground and the different perspectives in descrip- Architecture Thesaurus (AAT), to other museum tion, this task seems to be very ambitious, to say and collection databases or online auction cata- the least. Whereas the stag beetle in the foot of the logs would be obvious. Unfortunately, many of goblet is described in terms of art history and met- allurgy, we find a completely different description 18To quote just one prominent example, cf. the TERQAS of a pinned stag beetle in the BIODAT data base. (Time and Event Recognition for Question Answering) We may be lucky to identify it there if we know Symposium, 2002, http://www.timeml.org/site/ terqas/index.html ; visited 05.02.2009 the precise species name in advance, but in many 19cf. (Kamp and Reyle, 1993) cases, there is a significant chance that the match-
8 ing task will fail. At this point in time, we can only References provide a sketch in terms of an example how we Kerstin Bucher,¨ Gunther¨ Goerz, and Bernd Ludwig. would approach this challenge. But it seems ob- 2002. Corega Tabs: Incremental semantic compo- vious if we could find a general way to connect to sition. In Gunther¨ Goerz and et al., editors, KI-2002 different description systems, we would approach Workshop on Applications of Description Logics, Proceedings, volume 63 of CEUR Workshop Pro- the long-term goal of an “epistemic web”. ceedings, Aachen, September. Gesellschaft fur¨ In- Recent efforts showed that there is in fact a formatik e.V. way to a solution, indicated by the term “trans- disciplinarity”; first results have been presented Martin Doerr. 2003. The CIDOC Conceptual Refer- ence Model: an ontological approach to semantic in- at the first meeting of the CIDOC working group teroperability of metadata. AI Magazine, 24(3):75– on “Transdisciplinary Approaches in Documen- 92, September. tation”20. Originating from philosophy of sci- Ingrid Fischer, Bernd Geistert, and Gunther¨ Goerz. ence (Mittelstrass, 2002), transdisciplinarity con- 1996. Incremental semantics construction and centrates on problems, which cannot be solved anaphora resolution using Lambda-DRT. In S. Bot- within a single disciplinary framework. It takes ley and J. Glass, editors, Proceedings of DAARC- a new view on the unity of science, focussing on 96 — Discourse Anaphora and Anaphor Resolution Colloquium, pages 235–244, Lancaster, July. scientific rationality, not systems. Taking into ac- count that for all sciences there are common ele- Gunther¨ Goerz, Martin Oischinger, and Bernhard ments in the practice of argumentation and justifi- Schiemann. 2008. An Implementation of the cation, transdisciplinarity is a research principle in CIDOC Conceptual Reference Model (4.2.4) in OWL-DL. In Proceedings of the 2008 Annual Con- the first place. Its emphasis on rational language ference of CIDOC — The Digital Curation of Cul- use in science offers a clue to the field of docu- tural Heritage, pages 1–14, Athens, Benaki Mu- mentation; as a starting point, our methodological seum, September 15–18. focus is first of all on data integration . Taking Lutz Heusinger. 1989. Marburger Informations-, into account that transdisciplinarity addresses the Dokumentations- und Administrations-System (MI- practice of research, this framework should sup- DAS) / [1,2]. Saur, Munchen.¨ port an action and event perspective on a generic Nancy Ide and Jean Veronis, editors. 1995. Text En- level, i.e. for the tasks of classification, represen- coding Initiative. Background and Context. Kluwer, tation, annotation, linking, etc. Dordrecht. Also in: Computers and the Humanities. In fact, we claim that the CIDOC CRM can play Vol. 29, No. 1–3 (1995). the role of such a transdisciplinary framework; Hans Kamp and Uwe Reyle. 1993. From Discourse to at least for the stag beetle on goblets and still Logic. Kluwer, Dordrecht. life paintings, some other insects and also birds on drawings and paintings, the modelling task Jurgen¨ Mittelstrass. 2002. Transdisciplinarity — new structures in science. In Innovative Structures in has already been performed successfully. For the Basic Research. Ringberg-Symposium, 4–7 October birds — hooded crows in Dutch winter scenes in 2000, number 5 in Max Planck Forum, pages 43–54, Brueghel paintings — our transdisciplinary mod- Munchen.¨ elling effort provided a nice result for biodiversity David A. Smith. 2002. Detecting events with date research as a side effect: During the “little ice age” and place information in unstructured text. In Pro- hooded crows lived in Western Europe, whereas ceedings of the 2nd ACM+IEEE Joint Conference today they can only be found east of the Elbe river. on Digital Libraries, pages 191–196, Portland, OR. Regine Stein and Erin Coburn. 2008. CDWA Lite Acknowledgments and museumdat: New developments in metadata standards for cultural heritage information. In Pro- The authors are grateful for valuable hints and ceedings of the 2008 Annual Conference of CIDOC, discussions to Siegfried Krause, Georg Hohmann, Athens, September 15–18. Karl-Heinz Lampe, and Bernhard Schiemann and Svenja Tantzen. 2004. Ein Prologparser fur¨ temporale to the anonymous reviewers for valuable sugges- und lokale Ausdrucke¨ in einem ‘Geosem-System’ tions. fur¨ das Deutsche. Technical report, Friedrich- 20at the CIDOC 2008 conference; online materi- Alexander-Universitat¨ Erlangen-Nurnberg,¨ als are available via http://www8.informatik. Philosophische Fakultat¨ II, Erlangen. Master uni-erlangen.de/IMMD8/Services/ Thesis. transdisc/ ; visited 03.12.2008.
9 An intelligent authoring environment for abstract semantic representations of cultural object descriptions
Stasinos Konstantopoulos and Vangelis Karkaletsis and Dimitris Bilidas Institute of Informatics & Telecommunications NCSR ‘Demokritos’, Greece {konstant,vangelis,dbilid}@iit.demokritos.gr
Abstract More specifically, we present ELEON, an au- thoring environment for creating abstract concep- In this paper we describe an authoring tual representations of cultural heritage object de- environment for the creation of cultural- scriptions, as well as the linguistic and profil- domain ontologies and the associated lin- ing models necessary to realize those into con- guistic and profile annotations, for dy- crete natural-language descriptions exploiting nat- namically generating adaptable natural- ural language generation technology. The advan- language descriptions of the cultural ob- tages of this approach, as opposed to directly au- jects in the ontology. Adaptation is thoring natural language descriptions, are mani- achieved at the expense of considerable fold: authoring effort, since it relies on pro- viding numerical parameters for each on- • Abstract descriptions constitute machine- tological entity. To assist the authoring readable and reusable models of the cultural process, we provide an intelligent author- heritage collection. Besides deriving natural ing back-end that completes manually au- language descriptions, such models can be thored models by inferring missing values. used for the semantic indexing and search- This intelligent authoring support facility, ing of the collection. This can also be seen combined with immediate previews, can from the reverse perspective: the natural lan- considerably reduce the effort required to guage descriptions can be derived from ex- create a fully functional model as the au- isting conceptual models created for the pur- thor can iterate through cycles of provid- pose of semantic indexing and searching. ing information, previewing the generated text, and only elaborating the model where • The conceptual descriptions are realized us- the text is unsatisfactory. ing domain-independent, reusable linguistic models. By clearly separating the conceptual 1 Introduction and linguistic models, the same conceptual descriptions can be realized in different lan- Cultural heritage organizations create and main- guages and the same linguistic models can be tain repositories of (digital representations of) arti- used to realize descriptions of different col- facts, including extensive semantic knowledge and lections. meta-data about the cultural objects in the collec- tion. Such semantic repositories are typically seen • The dynamic generation of the description as an opportunity to catalogue, index, and classify is driven by profiles that personalize the de- the cultural content, for the purpose of providing scriptions for different audiences, but also semantic searching and browsing facilities to pro- adapt them to different contexts and situa- fessional users as well as to the general public. tions. In this article we discuss another unique op- portunity that cultural heritage repositories offer: ELEON provides great flexibility in finely the opportunity to automatically generate adapt- parametrizing how the generated descriptions are able and customizable textual descriptions of the adapted to different audiences and situations. Fur- cultural objects for a variety of audiences and pur- thermore, the authoring environment is backed by poses. Artificial Intelligence tools that assist the author
Proceedings of the EACL 2009 Workshop on Language Technology and Resources for Cultural Heritage, Social Sciences, Humanities, and Education –LaTeCH – SHELT&R 2009, pages 10–17, Athens, Greece, 30 March 2009. c 2009 Association for Computational Linguistics
10 by automatically inferring missing profile param- stances. eters, alleviating the burden of explicitly providing Figure1 illustrates part of such an ontology all necessary details for large numbers of objects. that encodes knowledge about the ancient Agora Although the system can be used in a variety of of Athens. This ontology is used in the INDIGO domains and Human-Computer interaction appli- project,1 implementing a use case where the sys- cations, it is particularly pertinent to cultural her- tem guides visitors through an exhibition on the itage content, which is interesting for wide ranges ancient Agora of Athens, introducing the build- of age groups, levels of expertise, cultural and edu- ings to them before they attend a virtual 3D tour cational backgrounds, situations and contexts, em- of the Agora hosted at the Foundation of the Hel- phasising the need for personalized and custom- lenic World. The examples used in this paper are tailored text. drawn from this domain. In the rest of this article we first set the back- In the example of Figure1, ‘stoa-of-attalus’, ground by describing the authoring environment is an instance of the entity type Stoa, a sub- and particularly the way in which it can be used to type of Building which is a sub-type of Archi- create the conceptual model of the collection and tecturalConstruction, a sub-type of PhysicalOb- populate it with data (Section2) and then proceed ject. Properties and relationships are expressed to to describe how adaptation parameters are rep- using fields. For any entity type, it is possible resented and used by human-computer interaction to introduce new fields which then become avail- systems (Section3). We then focus on the main able to all the entities that belong to that type and contribution of this paper by describing the intel- its subtypes. In Figure1, the field locatedIn is ligence mechanism behind the environment (Sec- introduced at the ArchitecturalConstruction en- tion4), discuss related work (Section5), and con- tity type and is defined as a relationship between clude (Section6). ArchitecturalConstruction and Place, while the using-period field defines a property of the Phys- 2 Authoring Domain Ontologies icalObject entity type. Consequently, all enti- ties of type PhysicalObject and its subtypes, i.e. ELEON enables its authors—i.e., persons that have ArchitecturalConstruction and ArtObject inherit domain expertise but no technological expertise— these fields. Furthermore, all the instances of these to create a new application domain, defining the entity types and their subtypes also inherit these ontology of the new domain, as well as the cor- fields. responding language resources and the profiling The proposed system expresses such ontologi- models. All these elements are used by a natu- cal conceptual models in OWL [11], an ontology ral language generation (NLG) engine in order to representation language that is one of the core se- derive natural language descriptions from the con- mantic web technologies. OWL models can be ceptual representations in the ontology. The envi- created from scratch in the authoring tool or im- ronment also enables authors to generate text pre- ported, facilitating the use of well-established con- views using the NLG engine in order to examine ceptual models in the cultural heritage domain, as the effect of their updates to the domain ontology, almost all can be (or already are) expressed as on- the language resources and the profiling parame- tologies. The CIDOC conceptual reference model, ters. for example, also provides an official OWL ver- Concerning the language resources, these affect sion.2 Most other cultural heritage vocabularies, the content and the surface form of the derived thesauri, and classification schemes using XML texts, leading to more varied texts and contain en- or relational database data models are consistent tries for nouns and verbs for each supported lan- with the Simple Knowledge Organization System guage. With regard to the profiling parameters, (SKOS) and can be automatically converted to on- these are used by ELEON to adapt the generated tologies.3 descriptions to the users’ preferences and knowl- 1See http://www.ics.forth.gr/indigo/ and edge. Acknowledgements section. ELEON ontologies encode domain knowledge in 2http://cidoc.ics.forth.gr/official_ the form of concepts, instances of concepts, (en- release_cidoc.html 3See http://www.w3.org/2004/02/skos/ tity types and entities respectively in ELEON ter- about SKOS. A variety of tools exist for con- minology), and relations between concepts and in- verting SKOS data models to, or aligning with,
11 flecting a synthetic personality in the description, as well as personalizing it for a particular audi- ence. Adaptation parameters are provided in the form of profile attributes that control aspects of the text plan such as how many and which of the facts known about an object should be used to describe it, as discussed in more detail below.
3.1 Personalization and personality The system supports authoring the adaptation pro- files that control the dynamic adaptation of the generated descriptions. Profiles permit the au- thor to specify, for example, that technical vocab- Figure 1: ELEON screen, showing the class hi- ulary be used when generating for experts, or that erarchy and the individuals of each class (left), shorter and simpler sentences are generated for the properties of the currently selected individual children. This is achieved by providing a variety (right top), and a preview of the description of the of generation parameters though user profiles, in- individual (right bottom). The preview language cluding a numerical interest attribute of the prop- and profile can be seen on (and selected from) the erties of the ontology. bar at the bottom of the screen. Isard et al. [7] describe how interest is used to impose a preference ordering of the properties of 3 Description Adaptation ontological entities, controlling which facts will be used when describing each entity. In the work de- Besides modelling the cultural heritage domain scribed here, we have extended profiles in two re- itself, ELEON supports annotating the objects, spects: classes, and properties of the domain with adapta- tion and linguistic information. Such information • by generalizing interest into arbitrary, author- is used by NLG engines to (a) plan the descrip- defined profile attributes; and tion that will be generated, adapting it to the cur- • by permitting profile attributes to apply not rent audience and circumstance, and (b) realize the only to ontological properties, but also to in- planned description in a particular language. dividuals and classes. Realization is based on clause plans (micro- plans) that specify how an ontological property Using these extensions, authors can define person- can be expressed in each supported natural lan- ality profiles for generating text, managing dia- guage. The author specifies the clause to be gen- logue, and simulating emotional variation in a way erated in abstract terms, by specifying, for ex- that reflects a certain personality on behalf of the ample, the verb to be used, the voice and tense system. of the resulting clause, etc. Similar annota- In the INDIGO project we use these profiles tions for instances and classes specify how they in a human-robot interaction application, where should be realized as noun phrases that fill slots a robotic tour guide that gives the impression of in the property-generated clauses. Micro-plan an- empathizing with the visitor is perceived as more notations also comprise several other language- natural and user-friendly. But the methodology specific parameters, such as whether the resulting is generally interesting in any context of generat- clause can be aggregated into a longer sentence or ing descriptions of cultural heritage content, es- not, its voice and tense, and so on, as described in pecially if the individual descriptions are aggre- more detail by Androutsopoulos et al. [1], Sect. 3. gated in a tour of the collection. In such contexts, Adaptive planning, on the other hand, operates dialogue-management adaptivity can vary the ex- at the abstract level and does not involve specifics hibits included in personalized tours and emo- of the target language. It is rather aimed at re- tional state variation can match the described con- tent and make the tour more engaging and lively. ontological models. See, for example, http: //www.heppnetz.de/projects/skos2gentax/ The way in which personality profiles are used and http://annocultor.sourceforge.net/ to parametrize dialogue management and simu-
12 lated emotions are discussed in more detail else- where [9], so we shall only briefly outline it here. Emotional variation is achieved by using the personality profile to estimate the emotional ap- praisal of dialogue acts and update the mood and emotional state of artificial agents. Dialogue man- agement is affected both directly, by taking exhibit preference into account when deliberating over di- alogue acts, and indirectly, by being influenced by the artificial agent’s current mood; and, as already mentioned above, NLG is adapted by using prop- erty preference to plan a description In the Konstantopoulos et al. [9] model, prefer- ence is calculated based on a logic model of the Figure 2: Screen fragment, showing the pop-up robot’s personality traits and also on ground facts window for providing profile attribute values for regarding objective attributes of the content—such an exhibit. Automatically inferred attribute values as the importance of an exhibit—but also subjec- are displayed in red, to stand out from explicitly tive attributes that reflect the robot’s perception of provided ones which are displayed in black. the content—such as how interesting an exhibit is. With the work described here, we alleviate the bur- itself provides a serialization of OWL ontologies den of manually providing all the ground param- as RDF for transport and data interchange pur- eters, exploiting the fact that these parameters are poses. Since ELEON uses this OWL/RDF repre- strongly inter-related and can, to a large extend, be sentation for the domain ontology, linguistic and automatically inferred. More specifically, ELEON profile annotations can be directly represented as backs the profile authoring process by reasoning RDF triples of extra-ontological properties of the over manually provided exhibit attributes in order ontological instances. to infer what the values of the missing attributes The RDF vocabulary used defines a prop- should be. The author can inspect the explicitly erty that relates ontological entities (individuals, provided as well as the automatically inferred val- classes, and properties) with profile attribute nodes ues and make corrections where necessary (Fig- that involve: ure2). Manual corrections trigger a re-estimation of the missing values, so that after each round of • the profile to which they are pertinent, e.g., corrections the overall model is a closer approxi- ‘expert’; mation of the author’s intention. • the attribute, e.g., ‘interest’ or ‘importance’; 3.2 Representation and interoperability and
Linguistic and profile annotations are represented • the numerical value of the attribute for this in RDF, the Resource Description Framework entity in this profile. (RDF) [5]. RDF is a knowledge representa- tion technology built around the concept of us- When applied to ontology properties, profile at- ing subject-predicate-object triples to describe ab- tribute nodes can be further elaborated to apply stract entities, resources. RDF triples assign to only to properties of instances of a particular class. their subject resource the property of being related For example, one can express that users find it to the object through the predicate resource. Pred- more interesting to know the architectural style icates can be data properties, in which case their when discussing temples than when discussing objects are concrete values (numbers, strings, time stoas. periods, and so on), or object properties, in which Using RDF is motivated by the usage of OWL case their objects are abstract resources. to represent the domain ontology as well as the Although OWL is not formally defined in RDF, availability of natural language generation (NLG) it is defined in such a way that it can be repre- engines that support it. More specifically, as al- sented within RDF. In fact, the OWL specification ready discussed, OWL ontologies and RDF anno-
13 tations can be easily merged in a combined model since OWL ontologies can be expressed in RDF. An alternative approach would have been to in- corporate profile attributes in the OWL ontology. Since profile attributes refer to classes and proper- ties as well as individuals, profile attributes would, then, have to be interpreted as second-order on- tological properties. Although second-order con- structs can be represented in OWL-Full (the most expressive ‘dialect’ of OWL), logical inference over OWL-Full ontologies is a challenging and computationally inefficient task. In fact, second- order inference is only supported by research pro- totypes and only for restricted fragments, often excluding binary second-order predicates (second- Figure 3: System architecture showing the inter- order properties). facing with NLG and Inference engines By contrast, the chosen approach restricts the ontology within the computationally efficient logics and many-valued valuations. OWL-DL dialect, for which multiple stable and highly optimized inference engines have been de- 4.1 Integrating reasoning in ELEON veloped. Profile attributes are provided as extra- ELEON specifies a Java interface through which ontological properties, without tying them to a inference results can be requested and retrieved: particular logical interpretation. We shall revisit OWL domain models and RDF profile annotations this point in the following section. are passed to an inference engine, which responds The second motivating factor behind RDF pro- with the numerical values of profile attributes for file attributes is interoperability with NLG en- all ontological entities in the domain (individuals, gines. The RDF vocabulary used to assign linguis- classes, and properties). tic and profile attributes is understood by the NAT- ELEON also extends and uses the TransOnto se- URALOWL[6] and METHODIUS [10] generation mantic knowledge migration system4 to perform engines (Figure3). all the necessary transformations for using many- valued DL reasoners, i.e., transforming the OWL 4 Intelligent Authoring Support and RDF models into many-valued DL assertions, We have previously discussed how profile at- as well as transforming logical query answers into tributes were not directly incorporated in the do- the numerical profile-attribute values. main ontology as second-order statements, but are Furthermore, as depicted in Figure3, the im- rather represented as extra-logical RDF annota- plementation includes the necessary API calls for tions. While avoiding forcing a second-order in- using either of two many-valued DL reasoners, terpretation of profile attributes is a definite ad- YADLR [8] or FUZZYDL[4]. Support for alter- vantage from a computational-complexity point of native many-valued DL reasoners can be easily view, this choice leaves profile attributes outside added, by using such reasoners to implement the the scope of OWL reasoning tools. reasoning API expected by the authoring system. In order to be able to efficiently reason over and 4.2 Many-valued DL Reasoning draw inferences about profile attributes, we have chosen to interpret profile attributes within many- Description Logics (DL) [2] are a family of first- valued description logics. Using description log- order logics; their main characteristic is decidabil- ics has the advantage of direct access to the do- ity, attained by being restricted to concepts (unary main ontology; using many-valued valuations has predicates, sets of individuals) and relations (bi- the advantage of providing a means to represent nary predicates, sets of pairs of individuals). Of and reason over numerical values. particular importance is the DL called SHOIN , This section describes this interpretation and which covers OWL-DL. how it is used, after first introducing description 4See http://transonto.sourceforge.net/
14 DL statements, concept descriptions, use logi- Ontology Interesting cal connectives to define concepts by combining Instance membership (a) other concepts, and (b) relation constructs that Doric style 0.8 describe the set of individuals that have a certain Ionic style 0.7 relation with a certain set of fillers (relation ob- Pergamene style 0.3 jects). Relation descriptions are not supported, Attalus 0.9 and membership in a relation can only be ex- plicitly asserted, except for a limited set of rela- Table 1: Profile fragment. tion axioms such as inversion, subordination, and transitivity. Most DL reasoners also provide lim- Resource Property Value ited support for reasoning over concrete domains Stoa of Attalus style Doric (numbers, strings, etc.) through data properties Stoa of Attalus style Ionic that relate abstract individual subjects with con- Stoa of Attalus style Pergamene crete value objects. Stoa of Attalus orderedBy Attalus Many-valued logics in general, and conse- quently many-valued DLs, extend the binary true- Table 2: Ontology fragment, showing the proper- false valuations of logical formulae into many- ties of the ‘Stoa of Attalus’ instance. valued numerical valuations, denoting the degree to which formulae hold. Such many-valued mod- 0.6, which, under Łukasiewicz-Tarski semantics, els receive their semantics not from set theory, as means that being a stoa implies being interesting is the case with binary valuations, but from al- at a loss of 0.4 of a degree. Thus individuals that gebraic norms that assign semantics to the logi- are members of the Stoa class at a degree of 1.0, cal connectives. These norms are used to calcu- are implicitly interesting at a degree of 0.6. Al- late the degree at which complex logical proposi- though this is not identical to saying that the class tions hold, given the degrees of their constituent itself is interesting, it clearly captures the intention elementary propositions. behind the original RDF annotation. In the work described here we use Profile attributes can also characterize proper- Łukasziewicz-Tarski algebra to provide many- ties, like orderedBy, creationEra or style, en- valued semantics [9, Sect. 3]. Although there is coding the information that it might, for example, nothing in ELEON itself that forces this choice, be more interesting to describe the artistic style of Łukasiewicz-Tarski algebra is well-suited to an exhibit rather than provide historical data about inferring profile attribute values, as it is founded it. This is interpreted as the strength of the con- on neither probability nor uncertainty, which nection between how interesting an exhibit is, and would be inappropriate in our case, but on the how interesting its properties are. In other words, notion of relevance. if having an interesting filler for style also makes the exhibit interesting, this is taken as an indica- 4.3 Inferring missing attribute values tion that the style relation itself is an interesting Profile attributes of individuals are captured by one. Formulated in logical terms, having interest- normalizing in the [0, 1] range and then using the ing relation fillers implies being interesting, and normalized value as a class membership degree. the implication holds at a degree provided by the So, for example, if interesting is such an at- interest level of the relation itself. tribute of individual exhibits, then an exhibit with For example, consider the assertion at 0.8 that a (normalized) interest level of 0.7 is an instance the class of things that are related to at least of the Interesting class at a degree of 0.7. one Interesting instance with the style property, Attributes of classes are reduced to attributes of are themselves Interesting and the assertion at the members of the class, expressed by a class sub- 0.4 that the class of things that are related to at sumption assertion at the degree of the attribute. least one Interesting instance with the orderedBy So, if the class of stoas is interesting at a degree property, are themselves Interesting. of 0.6, this is expressed by asserting that being a Given a profile fragment like the one in Table1 member of Stoa implies being a member of Inter- and a domain ontology including the factual infor- esting. The implication is asserted at a degree of mation in Table2, ‘Stoa of Attalus’ has an interest-
15 ing style at a degree of 0.8, which is the maximum the concrete linguistic information, facilitating the among the three architectural styles found in the easy reuse of the ontological information to gener- stoa (Doric, Ionic, and Pergamene). Since style ate descriptions in multiple languages, as well as fillers transfer interest at a loss of 0.2, style con- using an external NLG engine to provide previews tributes 0.6 to the stoa’s Interesting-ness. By con- of the descriptions from within the authoring envi- trast, the filler of orderedBy (which is more inter- ronment. esting in this profile than any of the architectural The system presented here extends a previous styles) only contributes 0.3 of a degree, because version of ELEON [3], which supports using an orderedBy is annotated as uninteresting and in- external DL reasoner to catch logical errors by terest transfers across it at a heavy loss. checking the consistency of the authored ontology. We have so far discussed how to infer profile In the work described here, the intelligence behind attribute values for the individuals of the domain. the tool is substantially extended by using logical Classes and relations receive the value of the min- inference to predict values that have not been ex- imal instance of the class (relation). That is to say, plicitly entered by the user, alleviating the need to the individual (pair of individuals) for which noth- manually provide large volumes of numerical data. ing else is known, except that it is a member of the A parallel line of development of the origi- class (relation). nal Androutsopoulos et al. tool is based on the As an example, consider a DoricBuilding class Proteg´ e´ ontology authoring and management en- which is a subclass of Building that only admits vironment.5 Galanis and Androutsopoulos [6] de- instances that have a style relation with ‘Doric’. veloped a Proteg´ e´ plug-in that builds upon the ex- The minimal instance of this class is a member of tensive ontology authoring features of Proteg´ e´ to Interesting through having an interesting property provide an environment for creating cultural her- as discussed above, even though nothing else is itage ontologies and the associated linguistic and known about it. This membership degree in Inter- profiling annotations. It does not, however, of- esting is taken to be an attribute of the class itself fer the flexibility to define new profile attributes rather than any one of its members, and is used as as ELEON does, and is restricted to specifying the the attribute value for the class itself. level of interest of the various ontological entities. For relations, two minimal instances of the rela- Furthermore, it only uses logic inference to catch tion’s domain and range are created. The attribute ontological inconsistencies in a manner similar to value for the property is the degree of the impli- that described by Bilidas et al. [3] without any pre- cation that having this property makes the domain diction facilities. individual have the attribute. For example, in order to infer how interesting the property devotedTo is, 6 Conclusion we first observe that it relates Temple instances In this article we have presented an authoring envi- with MythicalPerson instances, and create bare ronment for the creation of domain ontologies and instances of these two classes. The implication the associated linguistic and profile annotations. that having a devotedTo relation to an Interest- Annotated ontologies can be used to automatically ing individual leads to being member of Interest- generate natural-language descriptions of the enti- ing holds to a degree that can be calculated, given ties of the ontology, dynamically adapting the gen- the Interesting degrees of the Temple and Mythi- eration engine to the audience and context of the calPerson instances involved in the relation. The description. degree of the implication is then used as the value The advantages of using ELEON instead of of the interesting attribute. generic knowledge tools, such as Proteg´ e,´ stem from the ability to couple ELEON with external en- 5 Related Work gines that provide important conveniences to the author. More specifically, ELEON can invoke a nat- ELEON is based on the authoring tool described ural language generation engine in order to display by Androutsopoulos et al. [1], which was also tar- previews of the description based on the informa- geted at creating ontologies for generating person- tion currently provided about an object. Further- alized descriptions of the individuals in the ontol- more, logical inference is used to provide an intel- ogy. ELEON inherits from that tool the idea of separating the abstract ontological relations from 5See http://protege.stanford.edu/
16 ligent authoring back-end that completes the cur- [3] Dimitris Bilidas, Maria Theologou, and Van- rent model by inferring missing values based on gelis Karkaletsis. 2007. Enriching OWL on- what has already been provided. tologies with linguistic and user-related an- This intelligent authoring support facility, com- notations: the ELEON system. In Proc. bined with the immediate previews, can consid- 19th IEEE Intl. Conf. on Tools with Artificial erably reduce the effort required to create a fully Intelligence (ICTAI-2007), Patras, Greece, functional model as the author can iterate through Oct. 2007. IEEE Computer Society. cycles of providing information, previewing the [4] Fernando Bobillo and Umberto Straccia. generated text, and only elaborating the model 2008. fuzzyDL: an expressive fuzzy Descrip- where the text is unsatisfactory. This iterative pro- tion Logic reasoner. In Proceedings of the cess converges to satisfactory descriptions much 2008 International Conference on Fuzzy Sys- faster than having to manually enter all adaptation tems (FUZZ-08). parameters, especially for large and complex do- [5] Dan Brickley and R. V. Guha. 2004. RDF mains. Primer. W3C Recommendation. In the context of the XENIOS project,6 the pre- vious version of ELEON has been evaluated by cu- [6] Dimitris Galanis and Ion Androutsopoulos. rators of the Foundation of the Hellenic World, 2007. Generating multilingual descriptions who used it to create an ontology of the buildings, from linguistically annotated OWL ontolo- rooms, and exhibitions of the Foundation. In the gies: the NaturalOWL system. In Pro- context of creating the Agora of Athens ontology ceedings of the 11th European Workshop on and annotations for INDIGO, we are planning to Natural Language Generation (ENLG 2007), extend this evaluation to include the new intelli- Schloss Dagstuhl, Germany, pages 143–146. gent authoring features. [7] Amy Isard, Jon Oberlander, Ion Androut- sopoulos, and Colin Matheson. 2003. Speak- Acknowledgements ing the users’ languages. IEEE Intelligent Systems, 18(1):40–45. The work described here is supported by the FP6- IST project INDIGO7 which aims to advance [8] Stasinos Konstantopoulos and Georgios human-robot interaction methodology and tech- Apostolikas. 2007. Fuzzy-DL reasoning nology, enabling robots to perceive natural human over unknown fuzzy degrees. In Proc. Intl. behaviour, as well as making them act in ways that IFIP Workshop of Semantic Web and Web are more familiar to humans. The authors also Semantics (IFIP-SWWS 07), Nov. 2007, wish to acknowledge the help of the colleagues Vilamoura, Portugal. LNCS 4806, Springer, from the Foundation of the Hellenic World who Berlin/Heidelberg. used ELEON to create the initial version of the on- [9] Stasinos Konstantopoulos, Vangelis tology for the ancient Agora of Athens. Karkaletsis, and Colin Matheson. 2008. Robot personality: Representation and exter- References nalization. In Proceedings of Intl. Workshop on Computational Aspects of Affective and [1] Ion Androutsopoulos, Jon Oberlander, and Emotional Interaction (CAFFEi 08), Patras, Vangelis Karkaletsis. 2007. Source authoring Greece, July 2008. for multilingual generation of personalised object descriptions. Journal of Natural Lan- [10] Matthew Marge, Amy Isard, and Johanna guage Engineering, 13(3):191–233. Moore. 2008. Creation of a new do- main and evaluation of comparison gener- [2] Franz Baader, Ian Horrocks, and Ulrike Sat- ation in a natural language generation sys- tler. 2003. Description logics as ontology tem. In Proceedings of the Fifth Inter- languages for the semantic web. In Steffen national Language Generation Conference Staab and Rudi Studer, editors, Lecture Notes (INLG08), June 2008, Salt Fork, Ohio, USA. in Artificial Intelligence. Springer Verlag. [11] Michael K. Smith, Chris Welty, and Debo- 6See http://www.ics.forth.gr/xenios/ (in rah L. McGuinness. 2004. OWL web ontol- Greek) ogy language. W3C Recommendation. 7See http://www.ics.forth.gr/indigo/
17 Multiple sequence alignments in linguistics
Jelena Prokic´ Martijn Wieling John Nerbonne University of Groningen University of Groningen University of Groningen The Netherlands The Netherlands The Netherlands [email protected] [email protected] [email protected]
Abstract We present work in a research line which seeks to In this study we apply and evaluate an submit pronunciation data to phylogenetic analy- iterative pairwise alignment program for sis (Gray and Atkinson, 2003) and which requires producing multiple sequence alignments, an alignment of the (phonological) segments of ALPHAMALIG (Alonso et al., 2004), us- cognate words. We focus in this paper on evalu- ing as material the phonetic transcriptions ating the quality of multi-aligned pronunciations. of words used in Bulgarian dialectological In bioinformatics, sequence alignment is a way research. To evaluate the quality of the of arranging DNA, RNA or protein sequences in multiple alignment, we propose two new order to identify regions of similarity and deter- methods based on comparing each column mine evolutionary, functional or structural simi- in the obtained alignments with the cor- larity between the sequences. There are two main responding column in a set of gold stan- types of string alignment: pairwise and multiple dard alignments. Our results show that the string alignment. Pairwise string alignment meth- alignments produced by ALPHAMALIG ods compare two strings at a time and cannot di- correspond well with the gold standard rectly be used to obtain multiple string alignment alignments, making this algorithm suitable methods (Gusfield, 1997, 343-344). In multiple for the automatic generation of multiple string alignment all strings are aligned and com- string alignments. Multiple string align- pared at the same time, making it a good technique ment is particularly interesting for histor- for discovering patterns, especially those that are ical reconstruction based on sound corre- weakly preserved and cannot be detected easily spondences. from sets of pairwise alignments. Multiple string comparison is considered to be the holy grail of 1 Introduction molecular biology (Gusfield, 1997, 332): Our cultural heritage is studied today not only in It is the most critical cutting-edge tool for ex- museums, libraries, archives and their digital por- tracting and representing biologically important, tals, but also through the genetic and cultural lin- yet faint or widely dispersed, commonalities eaments of living populations. Linguists, popula- from a set of strings. tion geneticists, archaeologists, and physical and cultural anthropologists are all active in research- Multiple string comparison is not new in lin- ing cultural heritage on the basis of material that guistic research. In the late 19th century the may or may not be part of official cultural heritage Neogrammarians proposed the hypothesis of the archives. The common task is that of understand- regularity of sound change. According to THE ing the histories of the peoples of the world, espe- NEOGRAMMARIANHYPOTHESIS sound change cially their migrations and contacts. To research occurs regularly and uniformly whenever the ap- and understand linguistic cultural heritage we re- propriate phonetic environment is encountered quire instruments which are sensitive to its signals, (Campbell, 2004). Ever since, the understand- and, in particular sensitive to signals of common ing of sound change has played a major role in provenance. The present paper focuses on pronun- the comparative method that is itself based on the ciation habits which have been recognized to bear simultaneous comparison of different languages, signals of common provenance for over two hun- i.e. lists of cognate terms from the related lan- dred years (since the work of Sir William Jones). guages. The correct analysis of sound changes
Proceedings of the EACL 2009 Workshop on Language Technology and Resources for Cultural Heritage, Social Sciences, Humanities, and Education –LaTeCH – SHELT&R 2009, pages 18–25, Athens, Greece, 30 March 2009. c 2009 Association for Computational Linguistics
18 requires the simultaneous examination of corre- The structure of this paper is as follows. An sponding sounds in order to compare hypotheses example of a multiple alignment and a discus- about their evolution. Alignment identifies which sion of the advantages over pairwise alignment sounds correspond. Historical linguists align the is given in the next section, after which we dis- sequences manually, while we seek to automate cuss our data set in section 3. Section 4 explains this process. the iterative pairwise alignment algorithm and the In recent years there has been a strong fo- program ALPHAMALIG. Section 5 discusses the cus in historical linguistics on the introduction gold standard and two baselines, while section 6 of quantitative methods in order to develop tools discusses the novel evaluation procedures. The re- for the comparison and classification of lan- sults are given in section 7 and we end this paper guages. For example, in his PhD thesis, Kondrak with a discussion in section 8. (2002) presents algorithms for the reconstruction 2 Example of Multiple Sequence of proto-languages from cognates. Warnow et al. (2006) applied methods taken from phylogenet- Alignment ics on Indo-European phonetic data in order to In this section we will give an example of the au- model language evolution. Heeringa and Joseph tomatically multi-aligned strings from our data set (2007) applied the Levensthein algorithm to the and point out some important features of the si- Dutch pronunciation data taken from Reeks Ned- multaneous comparison of more than two strings. erlandse Dialectatlassen and tried to reconstruct a ‘proto-language’ of Dutch dialects using the pair- village1 j "A ---- wise alignments. village2 j "A z e -- Studies in historical linguistics and dialectome- village3 - "A s --- try where string comparison is used as a basis for village4 j "A s --- calculating the distances between language vari- village5 j "A z e k a eties will profit from tools to multi-align strings village6 j "E ---- automatically and to calculate the distances be- village7 - "6 s --- tween them. Good multiple alignment is of ben- efit to all those methods in diachronic linguistics Figure 1: Example of multiple string alignment such as the comparative reconstruction method or the so-called CHARACTER-BASEDMETHODS In Figure 1 we have multi-aligned pronuncia- taken from phylogenetics, which have also been tions of the word az ’I’ automatically generated successfully applied in linguistics (Gray and Jor- by ALPHAMALIG. The advantages of this kind dan, 2000; Gray and Atkinson, 2003; Atkinson of alignment over pairwise alignment are twofold: et al., 2005; Warnow et al., 2006). The multi- • First, it is easier to detect and process corre- alignment systems can help historical linguistics sponding phones in words and their alterna- by reducing the human labor needed to detect the tions (like ["A] and ["E] and ["6] in the second regular sound correspondences and cognate pairs column in Figure 1). of words. They also systematize the linguistic knowledge in intuitive alignments, and provide a • Second, the distances/similarities between basis for the application of the quantitative meth- strings can be different in pairwise compari- ods that lead to a better understanding of language son as opposed to multiple comparison. This variation and language change. is so because multi-aligned strings, unlike In this study we apply an iterative pairwise pairwise aligned strings, contain information alignment program for linguistics, ALPHAMA- on the positions where phones were inserted LIG, on phonetic transcriptions of words used in or deleted in both strings. For example, dialectological research. We automatically multi- in Figure 1 the pairwise alignment of the align all transcriptions and compare these gener- pronunciations from village 1 and village 3 ated alignments with manually aligned gold stan- would be: dard alignments. At the same time we propose two methods for the evaluation of the multiple se- village1 j "A - quence alignments (MSA). village3 - "A s
19 These two alignments have one matching el- at the time to the already aligned strings.3 It was ement out of three in total, which means originally developed to align corresponding words that the similarity between them is 1/3 = in bilingual texts, i.e. with textual data, but it func- 0.33. At the same time the similarity be- tions with any data that can be represented as a tween these two strings calculated based on sequence of symbols of a finite alphabet. In addi- the multi-aligned strings in Figure 1 would tion to the input sequences, the program needs to be 4/6 = 0.66. The measurement based on know the alphabet and the distances between each multi-alignment takes the common missing token pair and each pair consisting of a token and material into account as well. a gap. In order to perform multiple sequence align- 3 Data set ments of X-SAMPA word transcriptions we modi- fied ALPHAMALIG slightly so it could work with The data set used in this paper consists of pho- the tokens that consist of more than one symbol, netic transcriptions of 152 words collected from such as ["e], ["e:] and [t_S]. The distances be- 197 sites evenly distributed all over Bulgaria. It tween the tokens were specified in such a way that is part of the project Buldialect—Measuring lin- vowels can be aligned only with vowels and con- guistic unity and diversity in Europe.1 Pronun- sonants only with consonants. The same tokens ciations of almost all words were collected from are treated as identical and the distance between all the sites and for some words there are mul- them is set to 0. The distance between any token tiple pronunciations per site. Phonetic transcrip- in the data set to a gap symbol has the same cost tions include various diacritics and suprasegmen- as replacing a vowel with a vowel or a consonant tals, making the total number of unique characters with a consonant. Except for this very general lin- (types) in the data set 98.2 guistic knowledge, no other data-specific informa- tion was given to the program. In this research we 4 Iterative pairwise alignment do not use any phonetic features in order to define Multiple alignment algorithms iteratively merge the segments more precisely and to calculate the two multiple alignments of two subsets of strings distances between them in a more sensitive way into a single multiple alignment that is union of than just making a binary ’match/does-not-match- those subsets (Gusfield, 1997). The simplest ap- distinction’, since we want to keep the system lan- proach is to align the two strings that have the guage independent and robust to the highest pos- minimum distance over all pairs of strings and it- sible degree. eratively align strings having the smallest distance to the already aligned strings in order to generate 5 Gold standard and baseline a new multiple alignment. Other algorithms use In order to evaluate the performance of AL- different initializations and different criteria in se- PHAMALIG, we compared the alignments ob- lecting the new alignments to merge. Some begin tained using this program to the manually aligned with the longest (low cost) alignment instead of strings, our gold standard, and to the alignments the least cost absolutely. A string with the smallest obtained using two very simple techniques that edit distance to any of the already merged strings are described next: simple baseline and advanced is chosen to be added to the strings in the multiple baseline. alignment. In choosing the pair with the minimal distance, all algorithms are greedy, and risk miss- 5.1 Simple baseline ing optimal alignments. The simplest way of aligning two strings would be ALPHAMALIG is an iterative pairwise align- to align the first element from one string with the ment program for bilingual text alignment. It uses first element from the other string, the second el- the strategy of merging multiple alignments of ement with the second and so on. If two strings subsets of strings, instead of adding just one string are not of equal length, the remaining unaligned 1The project is sponsored by Volkswagen Stiftung. tokens are aligned with the gap symbol which rep- More information can be found at http://sfs.uni- tuebingen.de/dialectometry. 3More information on ALPHAMALIG can be found 2The data is publicly available and can be found at at http://alggen.lsi.upc.es/recerca/align/alphamalig/intro- http://www.bultreebank.org/BulDialects/index.html. alphamalig.html.
20 resents an insertion or a deletion. This is the align- j "A z e j "A z e ment implicit in Hamming distance, which ignores j "A -- - "A S - insertions and deletions. By applying this simple method, we obtained j "A -- multiple sequence alignments for all words in our j "A z e data set. An example of such a multiple sequence - "A S - alignment is shown in Figure 2. These align- ments were used to check how difficult the mul- Figure 3: Advanced baseline. The top two align- tiple sequence alignment task is for our data and ments each contain two alignment lines, and the how much improvement is obtained using more bottom one contains three. advanced techniques to multi-align strings. 6 Evaluation j "A -- j "A z e Although multiple sequence alignments are "A S -- broadly used in molecular biology, there is still no widely accepted objective function for evaluating Figure 2: Simple baseline the goodness of the multiple aligned strings (Gusfield, 1997). The quality of the existing methods used to produce multiple sequence 5.2 Advanced baseline alignments is judged by the ’biological meaning Our second baseline is more advanced than the of the alignments they produce’. Since strings first and was created using the following proce- in linguistics cannot be judged by the biological dure: criteria used in string evaluation in biology, we were forced to propose evaluation methods that 1. for each word the longest string among all would be suitable for the strings in question. One pronunciations is located of the advantages we had was the existence of 2. all strings are pairwise aligned against the the gold standard alignments, which made our longest string using the Levensthein algo- task easier and more straightforward—in order to rithm (Heeringa, 2004). We refer to both se- determine the quality of the multi-aligned strings, quences in a pairwise alignment as ALIGN- we compare outputs of the different algorithms to MENTLINES. Note that alignment lines in- the gold standard. Since there is no off-the-shelf clude hyphens indicating the places of inser- method that can be used for comparison of multi- tions and deletions. aligned strings to a gold standard, we propose two novel methods—one sensitive to the order of 3. the alignment lines—all of equal length—are columns in two alignments and another that takes extracted into account only the content of each column. 4. all extracted alignment lines are placed below 6.1 Column dependent method each other to form the multiple alignment The first method we developed compares the con- An example of combining pairwise alignments tents of the columns and also takes the column se- against the longest string (in this case [j"aze]) is quence into account. The column dependent eval- shown in Figure 3. uation (CDE) procedure is as follows: 5.3 Gold standard • Each gold standard column is compared to Our gold standard was created by manually cor- the most similar column out of two neigh- recting the advanced baseline alignments de- boring columns of a candidate multiple align- scribed in the previous section. The gold stan- ment. The two neighboring columns depend dard results and both baseline results consist of on the previous matched column j and have 152 files with multi-aligned strings, one for each indices j + 1 and j + 2 (at the start j = 0). It word. The pronunciations are ordered alphabeti- is possible that there are columns in the can- cally according to the village they come from. If didate multiple alignment which remain un- there are more pronunciations per site, they are all matched, as well as columns at the end of the present, one under the other. gold standard which remain unmatched.
21 • The similarity of a candidate column with a (0.75 + 1 + 1 + 1 + 1) gold standard column is calculated by divid- = 0.792 ing the number of correctly placed elements (5 + 1) in every candidate column by the total num- It is clear that this method punishes unmatched ber of elements in the column. A score of columns by increasing the value of the denomina- 1 indicates perfect overlap, while a score of tor in the similarity score calculation. As a conse- 0 indicates the columns have no elements in quence, swapped columns are punished severely, common. which is illustrated in Figure 5.
j j • The similarity score of the whole multiple "o r @ j - "o r @ - j j j alignment (for a single word) is calculated by "o r @ - u "o r @ u - j j summing the similarity score of each candi- "o r @ f - "o r @ - f date column and dividing it by the total num- Figure 5: Two alignments with swapped columns ber of matched columns plus the total num- ber of unmatched columns in both multiple In the alignments in Figure 5, the first three alignments. columns of GS would be matched with the first three columns of GA with a score of 1, the fourth • The final similarity score between the set of would be matched with the fifth, and two columns gold standard alignments with the set of can- would be left unmatched: the fifth GS column and didate multiple alignments is calculated by the fourth GA column yielding a total similarity averaging the multiple alignment similarity score of 4/6 = 0.66. Especially in this case this is scores for all strings. undesirable, as both sequences of these columns As an example consider the multiple alignments represent equally reasonable multiple alignment in Figure 4, with the gold standard alignment (GS) and should have a total similarity score of 1. on the left and the generated alignment (GA) on We therefore need a less strict evaluation method the right. which does not insist on the exact ordering. An alternative method is introduced and discussed in w rj "E m e w - rj "E m e the following section. v r "e m i v - r "e m i u rj "e m i - u rj "e m i 6.2 Modified Rand Index v rj "e m i v - rj "e m i In developing an alternative evaluation we pro- Figure 4: GS and ALPHAMALIG multiple string ceeded from the insight that the columns of a mul- alignments, the gold standard alignment left, the tiple alignment are a sort of PARTITION of the el- ALPHAMALIG output right. ements of the alignment strings, i.e., they consti- tute a set of disjoint multi-sets whose union is the The evaluation starts by comparing the first col- entire multi-set of segments in the multiple align- umn of the GS with the first and second column ment. Each column effectively assigns its seg- of the GA. The first column of the GA is the best ments to a partition, which clearly cannot overlap match, since the similarity score between the first with the elements of another column (partition). columns is 0.75 (3 out of 4 elements match). In Since every segment must fall within some col- similar fashion, the second column of the GS is umn, the assignment is also exhaustive. compared with the second and the third column of Our second evaluation method is therefore the GA and matched with the third column of GA based on the modified Rand index (Hubert and with a similarity score of 1 (all elements match). Arabie, 1985). The modified Rand index is used The third GS column is matched with the fourth in classification for comparing two different parti- GA column, the fourth GS column with the fifth tions of a finite set of objects. It is based on the GA column and the fifth GS column with the sixth Rand index (Rand, 1971), one of the most popular GA column (all three having a similarity score of measures for comparing the degree to which parti- 1). As a consequence, the second column of the tions agree (in classification). GA remains unmatched. In total, five columns are Given a set of n elements S = o1, ...on and two matched and one column remains unmatched. The partitions of S, U and V , the Rand index R is de- total score of the GA equals: fined as:
22 We would like to emphasize that it is clear that a + b R = the set of columns of a multi-alignment have more a + b + c + d structure than a partition sec, in particular because where: the columns (subpartitions) are ordered, unlike the subpartitions in a partition. But we shall compen- • a: the number of pairs of elements in S that sate for this difference by explicitly marking order. are in the same set (column) in U and in the same set in V "o [1] rj [2] @ [3] j [4] - j • b: the number of pairs of elements in S that "o [5] r [6] @ [7] - u [8] j are in different sets (columns) in U and in dif- "o [9] r [10] @ [11] f [12] - ferent sets in V Figure 6: Annotated alignment • c: the number of pairs of elements in S that are in the same set in U and in different sets In our study, each segment token in each tran- in V scription was treated as a different object (see Fig- ure 6), and every column was taken to be a sub- • d: the number of pairs of elements in S that partition to which segment tokens are assigned. are in different sets in U and in the same set Both alignments in Figure 5 have 12 phones that in V are put into 5 groups. We “tag” each token sequen- Consequently, a and b are the number of pairs of tially in order to distinguish the different tokens of elements on which two classifications agree, while a single segment from each other, but note that the c and d are the number of pairs of elements on way we do this also introduces an order sensitivity which they disagree. In our case classifications in the measure. The two partitions obtained are: agree about concrete segment tokens only in the cases where they appear in the same columns in GS1 = {1,5,9} GA1 = {1,5,9} the alignments. GS2 = {2,6,10} GA2 = {2,6,10} The value of Rand index ranges between 0 and GS3 = {3,7,11} GA3 = {3,7,11} 1, with 0 indicating that the two partitions (multi- GS4 = {4,12} GA4 = {8} alignments) do not agree on any pair of points and GS5 = {8} GA5 = {4,12} 1 indicating that the data partitions are exactly the 4 same. A problem with the Rand index is that it Using the modified Rand index the quality of does not return a constant value (zero) if two par- each column is checked, regardless of whether the titions are picked at random. Hubert and Arabie columns are in order. The MRI for the alignments (1985) suggested a modification of the Rand in- in Figure 5 will be 1, because both alignments dex (MRI) that corrects this property. It can be group segment tokens in the same way. Even expressed in the general form as: though columns four and five are swapped, in both Rand index − Expected index classifications phones [j] and [f] are grouped to- MRI = gether, while sound [u] forms a separate group. Maximum index − Expected index The MRI itself only takes into account the The expected index is the expected number of quality of each column separately since it sim- pairs which would be placed in the same set in U ply checks whether the same elements are together and in the same set in V by chance. The maximum in the candidate alignment as in the gold-standard index represents the maximum number of objects alignment. It is therefore insensitive to the order- that can be put in the same set in U and in the ing of columns. While it may have seemed coun- same set in V . The MRI value ranges between −1 terintuitive linguistically to proceed from an order- and 1, with perfect overlap being indicated by 1 insensitive measure, the comparison of “tagged to- and values ≤ 0 indicating no overlap. For a more kens” described above effectively reintroduces or- detailed explanation of the modified Rand index, der sensitivity. please refer to Hubert and Arabie (1985). In the next section we describe the results of ap- 4In dialectometry, this index was used by Heeringa et al. plying both evaluation methods on the automati- (2002) to validate dialect clustering methods. cally generated multiple alignments.
23 7 Results v l "7 k v l "7 - k v "7 l k v - "7 l k After comparing all files of the baseline algo- rithms and ALPHAMALIG against the gold stan- Figure 7: Two alignments with metathesis dard files according to the column dependent eval- uation method and the modified Rand index, the ing vowels and/or consonants with a gap (see for average score is calculated by summing up all example Figure 4). scores and dividing them by the number of word The other type of error present in the AL- files (152). PHAMALIG alignments is caused by the fact The results are given in Table 1 and also in- that all vowel-vowel and consonant-consonant dis- clude the number of words with perfect multi- tances receive the same weight. In Figure 8 alignments (i.e. identical to the gold standard). the alignment of word bjahme ’were’ produced Using CDE, ALPHAMALIG scored 0.932 out of by ALPHAMALIG is wrong because instead of 1.0 with 103 perfectly aligned files. The result aligning [mj] with [m] and [m] it is wrongly for the simple baseline was 0.710 with 44 per- aligned with [x] and [x], while [x] is aligned with fectly aligned files. As expected, the result for [S] instead of aligning it with [x] and [x]. the advanced baseline was in between these two results—0.869 with 72 files that were completely b "E S u x - m e - identical to the GS files. Using MRI to eval- bj "A - - x - m i - uate the alignments generated we obtained gen- b "e x - mj - - 7 - erally higher scores for all three algorithms, but with the same ordering. ALPHAMALIG scored Figure 8: Alignment error produced by AL- 0.982, with 104 perfectly aligned files. The ad- PHAMALIG vanced baseline had a lower score of 0.937 and 74 perfect alignments. The simple baseline per- formed worse, scoring 0.848 and having 44 per- 8 Discussion and future work fectly aligned files. In this study we presented a first attempt to auto- The scores of the CDE evaluation method are matically multi-align phonetic transcriptions. The lower than the MRI scores, which is due to the first algorithm we used to generate alignments has been method’s problematic sensitivity to column order- shown to be very reliable, produce alignments of ing in the alignments. It is clear that in both evalu- good quality, with less than 2% error at the seg- ation methods ALPHAMALIG outperforms both ment level. In this study we used only very sim- baseline alignments by a wide margin. ple linguistic knowledge in order to align strings. It is important to notice that the scores for the The only restriction we imposed was that a vowel simple baseline are reasonably high, which can should only be aligned with a vowel and a con- be explained by the structure of our data set. sonant only with a consonant. The system has The variation of word pronunciations is relatively shown to be very robust and to produce good qual- small, making string alignment easier. However, ity alignments with a very limited information on ALPHAMALIG obtained much higher scores us- the distances between the tokens. However, in the ing both evaluation methods. future we would like to apply this algorithm using Additional qualitative error analysis reveals that more detailed segment distances, so that we can the errors of ALPHAMALIG are mostly caused by work without vowel-consonant restrictions. Using the vowel-vowel consonant-consonant alignment more detailed language specific feature system for restriction. In the data set there are 21 files that each phone, we believe we may be able to improve contain metathesis. Since vowel-consonant align- the produced alignments further. This especially ments were not allowed in ALPHAMALIG, align- holds for the type of errors illustrated in Figure 8 ments produced by this algorithm were different where it is clear that [mj] is phonetically closer to from the gold standard, as illustrated in Figure 7. [m] than to [x] sound. The vowel-consonant restriction is also respon- As our data set was relatively simple (indicated sible for wrong alignments in some words where by the reasonable performance of our simple base- metathesis is not present, but where the vowel- line algorithm), we would very much like to eval- consonant alignment is still preferred over align- uate ALPHAMALIG against a more complex data
24 CDE CDE perfect columns MRI MRI perfect columns Simple baseline 0.710 44 0.848 44 Advanced baseline 0.869 72 0.937 74 ALPHAMALIG 0.932 103 0.982 104
Table 1: Results of evaluating outputs of the different algorithms against the GS set and try to replicate the good results we ob- Lyle Campbell. 2004. Historical Linguistics: An In- tained here. On one hand, high performance of troduction. Edinburgh University Press, second edi- both baseline algorithms show that our task was tion. relatively easy. On the other hand, achieving per- Russel D. Gray and Quentin D. Atkinson. 2003. fect alignments will be very difficult, if possible at Language-tree divergence times support the Ana- all. tolian theory of Indo-European origin. Nature, 426:435–339. Additionally, we proposed two methods to eval- uate multiple aligned strings in linguistic research. Russel D. Gray and Fiona M. Jordan. 2000. Lan- Although these systems could be improved, both guage trees support the express-train sequence of Austronesian expansion. Nature, 405:1052–1055. of them are giving a good estimation of the qual- ity of the generated alignments. For the examined Dan Gusfield. 1997. Algorithms on Strings, Trees and data, we find MRI to be better evaluation tech- Sequences: Computer Science and Computational Biology. Cambridge University Press. nique since it overcomes the problem of swapped columns. Wilbert Heeringa and Brian Joseph. 2007. The rela- In this research we tested and evaluated AL- tive divergence of Dutch dialect pronunciations from their common source: An exploratory study. In John PHAMALIG on the dialect phonetic data. How- Nerbonne, T. Mark Ellison, and Grzegorz Kondrak, ever, multiple sequence alignments can be also editors, Proceedings of the Ninth Meeting of the ACL applied on the sequences of sentences and para- Special Interest Group in Computational Morphol- graphs. This makes multiple sequence alignment ogy and Phonology. algorithm a powerful tool for mining text data in Wilbert Heeringa, John Nerbonne, and Peter Kleiweg. social sciences, humanities and education. 2002. Validating dialect comparison methods. In Wolfgang Gaul and Gunter Ritter, editors, Classifi- cation, Automation, and New Media. Proceedings of Acknowledgements the 24th Annual Conference of the Gesellschaft für Klassifikation e. V., University of Passau, March 15- We are thankful to Xavier Messeguer of the Tech- 17, 2000, pages 445–452. Springer, Berlin, Heidel- nical University of Catalonia who kindly sup- berg and New York. plied us with the source code of ALPHAMALIG. Wilbert Heeringa. 2004. Measuring Dialect Pronunci- We also thank Therese Leinonen and Sebastian ation Differences using Levenshtein Distance. Ph.D. Kürschner of the University of Groningen and Es- thesis, Rijksuniversiteit Groningen. teve Valls i Alecha of the University of Barcelona Lawrence Hubert and Phipps Arabie. 1985. Compar- for their useful feedback on our ideas. ing partitions. Journal of Classification, 2:193–218. Grzegorz Kondrak. 2002. Algorithms for Language References Reconstruction. PhD Thesis, University of Toronto. William M. Rand. 1971. Objective criteria for the Laura Alonso, Irene Castellon, Jordi Escribano, Xavier evaluation of clustering methods. Journal of Amer- Messeguer, and Lluis Padro. 2004. Multiple ican Statistical Association, 66(336):846–850, De- Sequence Alignment for characterizing the linear cember. structure of revision. In Proceedings of the 4th In- ternational Conference on Language Resources and Tandy Warnow, Steven N. Evans, Donald Ringe, and Evaluation. Luay Nakhleh. 2006. A stochastic model of lan- guage evolution that incorporates homoplasy and Quentin Atkinson, Geoff Nicholls, David Welch, and borrowing. In Peter Forster and Colin Renfrew, ed- Russell Gray. 2005. From words to dates: water itors, Phylogenetic Methods and the Prehistory of into wine, mathemagic or phylogenetic inference. Languages. MacDonald Institute for Archaeological Transcriptions of the Philological Society, 103:193– Research, Cambridge. 219.
25 Evaluating the pairwise string alignment of pronunciations
Martijn Wieling Jelena Prokic´ John Nerbonne University of Groningen University of Groningen University of Groningen The Netherlands The Netherlands The Netherlands [email protected] [email protected] [email protected]
Abstract in this paper on the issue of evaluating the quality of these instruments. Pairwise string alignment (PSA) is an im- Pairwise string alignment (PSA) methods, like portant general technique for obtaining a the popular Levenshtein algorithm (Levenshtein, measure of similarity between two strings, 1965) which uses insertions (alignments of a seg- used e.g., in dialectology, historical lin- ment against a gap), deletions (alignments of a gap guistics, transliteration, and in evaluating against a segment) and substitutions (alignments name distinctiveness. The current study of two segments) often form the basis of deter- focuses on evaluating different PSA meth- mining the distance between two strings. Since ods at the alignment level instead of via there are many alignment algorithms and specific the distances it induces. About 3.5 million settings for each algorithm influencing the dis- pairwise alignments of Bulgarian phonetic tance between two strings (Nerbonne and Klei- dialect data are used to compare four al- weg, 2007), evaluation is very important in deter- gorithms with a manually corrected gold mining the effectiveness of the distance methods. standard. The algorithms evaluated in- Determining the distance (or similarity) be- clude three variants of the Levenshtein al- tween two phonetic strings is an important aspect gorithm as well as the Pair Hidden Markov of dialectometry, and alignment quality is impor- Model. Our results show that while all tant in applications in which string alignment is algorithms perform very well and align a goal in itself, for example, determining if two around 95% of all alignments correctly, words are likely to be cognate (Kondrak, 2003), there are specific qualitative differences in detecting confusable drug names (Kondrak and the (mis)alignments of the different algo- Dorr, 2003), or determining whether a string is rithms. the transliteration of the same name from another writing system (Pouliquen, 2008). 1 Introduction In this paper we evaluate string distance mea- Our cultural heritage is not only accessible sures on the basis of data from dialectology. We through museums, libraries, archives and their therefore explain a bit more of the intended use of digital portals, it is alive and well in the varied the pronunciation distance measure. cultural habits practiced today by the various peo- Dialect atlases normally contain a large num- ples of the world. To research and understand this ber of pronunciations of the same word in various cultural heritage we require instruments which are places throughout a language area. All pairs of sensitive to its signals, and, in particular sensitive pronunciations of corresponding words are com- to signals of common provenance. The present pared in order to obtain a measure of the aggre- paper focuses on speech habits which even today gate linguistic distance between dialectal varieties bear signals of common provenance in the vari- (Heeringa, 2004). It is clear that the quality of the ous dialects of the world’s languages, and which measurement is of crucial importance. have also been recorded and preserved in major Almost all evaluation methods in dialectometry archives of folk culture internationally. We present focus on the aggregate results and ignore the in- work in a research line which seeks to develop dividual word-pair distances and individual align- digital instruments capable of detecting common ments on which the distances are based. The fo- provenance among pronunciation habits, focusing cus on the aggregate distance of 100 or so word
Proceedings of the EACL 2009 Workshop on Language Technology and Resources for Cultural Heritage, Social Sciences, Humanities, and Education –LaTeCH – SHELT&R 2009, pages 26–34, Athens, Greece, 30 March 2009. c 2009 Association for Computational Linguistics
26 pairs effectively hides many differences between intonation). The total number of different phonetic methods. For example, Heeringa et al. (2006) find types (or segments) is 98.1 no significant differences in the degrees to which The gold standard pairwise alignment was au- several pairwise string distance measures correlate tomatically generated from a manually corrected with perceptual distances when examined at an ag- gold standard set of N multiple alignments (see gregate level. Wieling et al. (2007) and Wieling Prokic´ et al., 2009 ) in the following way: and Nerbonne (2007) also report almost no differ- ence between different PSA algorithms at the ag- • Every individual string (including gaps) in gregate level. It is important to be able to evaluate the multiple alignment is aligned with ev- the different techniques more sensitively, which is ery other string of the same word. With 152 why this paper examines alignment quality at the words and 197 sites and in some cases more segment level. than one pronunciations per site for a cer- Kondrak (2003) applies a PSA algorithm to tain word, the total number of pairwise align- align words in different languages in order to de- ments is about 3.5 million. tect cognates automatically. Exceptionally, he • If a resulting pairwise alignment contains a does provide an evaluation of the string alignments gap in both strings at the same position (a generated by different algorithms. But he restricts gap-gap alignment), these gaps are removed his examination to a set of only 82 gold standard from the pairwise alignment. We justify this, pairwise alignments and he only distinguishes cor- reasoning that no alignment algorithm may rect and incorrect alignments and does not look at be expected to detect parallel deletions in a misaligned phones. single pair of words. There is no evidence for In the current study we introduce and evaluate this in the single pair. several alignment algorithms more extensively at the alignment level. The algorithms we evaluate To make this clear, consider the multiple align- include the Levenshtein algorithm (with syllabic- ment of three Bulgarian dialectal variants of the ity constraint), which is one of the most popular word ‘I’ (as in ‘I am’): alignment methods and has successfully been used j "A s in determining pronunciation differences in pho- "A z i netic strings (Kessler, 1995; Heeringa, 2004). In j "A addition we look at two adaptations of the Lev- Using the procedure above, the three generated enshtein algorithm. The first adaptation includes pairwise alignments are: the swap-operation (Wagner and Lowrance, 1975), j "A s j "A s "A z i while the second adaptation includes phonetic seg- "A z i j "A j "A ment distances, which are generated by applying an iterative pointwise mutual information (PMI) 3 Algorithms procedure (Church and Hanks, 1990). Finally we include alignments generated with the Pair Hid- Four algorithms are evaluated with respect to the den Markov Model (PHMM) as introduced to lan- quality of their alignments, including three vari- guage studies by Mackay and Kondrak (2005). ants of the Levenshtein algorithm and the Pair They reported that the Pair Hidden Markov Model Hidden Markov Model. outperformed ALINE, the best performing algo- 3.1 The VC-sensitive Levenshtein algorithm rithm at the alignment level in the aforementioned The Levenshtein algorithm is a very efficient dy- study of Kondrak (2003). The PHMM has also namic programming algorithm, which was first in- successfully been used in dialectology by Wieling troduced by Kessler (1995) as a tool for computa- et al. (2007). tionally comparing dialects. The Levenshtein dis- 2 Dataset tance between two strings is determined by count- ing the minimum number of edit operations (i.e. The dataset used in this study consists of 152 insertions, deletions and substitutions) needed to words collected from 197 sites equally distributed transform one string into the other. over Bulgaria. The transcribed word pronuncia- 1The dataset is available online at the website tions include diacritics and suprasegmentals (e.g., http://www.bultreebank.org/BulDialects/
27 For example, the Levenshtein distance between Note that a swap-operation in the alignment is in- [j"As] and ["Azi], two Bulgarian dialectal variants dicated by the symbol ‘><’. The first number fol- of the word ‘I’ (as in ‘I am’), is 3: lowing this symbol indicates the cost of the swap- j"As delete j 1 operation. 2 "As subst. s/z 1 Consider the alignment of [vr"7] and [v"7r], "Az insert i 1 two Bulgarian dialectal variants of the word ‘peak’ "Azi (mountain). The alignment involves a swap and 3 results in a total Levenshtein distance of 1: The corresponding alignment is: v r "7 v "7 r j "A s >< 1 "A z i 1 1 1 However, the alignment of the transcription [vr"7] The Levenshtein distance has been used fre- with another dialectal transcription [v"ar] does not quently and successfully in measuring linguis- allow a swap and yields a total Levenshtein dis- tic distances in several languages, including Irish tance of 2: (Kessler, 1995), Dutch (Heeringa, 2004) and Nor- v r "7 wegian (Heeringa, 2004). Additionally, the Lev- v "a r enshtein distance has been shown to yield aggre- 1 1 gate results that are consistent (Cronbach’s α = Including just the option of swapping identical 0.99) and valid when compared to dialect speak- segments in the implementation of the Leven- ers judgements of similarity (r ≈ 0.7; Heeringa et shtein algorithm is relatively easy. We set the al., 2006). cost of the swap operation to one3 plus twice the Following Heeringa (2004), we have adapted cost of substituting xi with yi+1 plus twice the the Levenshtein algorithm slightly, so that it does cost of substituting yi with xi+1. In this way the not allow alignments of vowels with consonants. swap operation will be preferred when xi = yi+1 We refer to this adapted algorithm as the VC- and yi = xi+1, but not when xi 6= yi+1 and/or sensitive Levenshtein algorithm. yi 6= xi+1. In the first case the cost of the swap operation is 1, which is less than the cost of the 3.2 The Levenshtein algorithm with the swap alternative of two substitutions. In the second case operation the cost is either 3 (if xi 6= yi+1 or yi 6= xi+1) or Because metathesis (i.e. transposition of sounds) 5 (if xi 6= yi+1 and yi 6= xi+1), which is higher occurs relatively frequently in the Bulgarian di- than the cost of using insertions, deletions and/or alect data (in 21 of 152 words), we extend the substitutions. VC-sensitive Levenshtein algorithm as described Just as in the previous section, we do not allow in section 3.1 to include the swap-operation (Wag- vowels to align with consonants (except in the case ner and Lowrance, 1975), which allows two ad- of a swap). jacent characters to be interchanged. The swap- operation is also known as a transposition, which 3.3 The Levenshtein algorithm with was introduced with respect to detecting spelling generated segment distances errors by Damerau (1964). As a consequence the The VC-sensitive Levenshtein algorithm as de- Damerau distance refers to the minimum number scribed in section 3.1 only distinguishes between of insertions, deletions, substitutions and transpo- vowels and consonants. However, more sensi- sitions required to transform one string into the tive segment distances are also possible. Heeringa other. In contrast to Wagner and Lowrance (1975) (2004) experimented with specifying phonetic and in line with Damerau (1964) we restrict the segment distances based on phonetic features and swap operation to be only allowed for string X 2 and Y when x = y and y = x (with x We use transcriptions in which stress is marked on i i+1 i i+1 i stressed vowels instead of before stressed syllables. We fol- being the token at position i in string X): low in this the Bulgarian convention instead of the IPA con- x x vention. i i+1 3Actually the cost is set to 0.999 to prefer an alignment yi yi+1 involving a swap over an alternative alignment involving only >< 1 regular edit operations.
28 also based on acoustic differences derived from 2. The PMI value for every segment pair is cal- spectrograms, but he did not obtain improved re- culated according to (1) and subsequently sults at the aggregate level. transformed to a segment distance by sub- Instead of using segment distances as these are tracting it from zero and adding the maxi- (incompletely) suggested by phonetic or phono- mum PMI value. logical theory, we tried to determine the sound distances automatically based on the available 3. The Levenshtein algorithm using these seg- data. We used pointwise mutual information ment distances is applied to generate a new (PMI; Church and Hanks, 1990) to obtain these set of alignments. distances. It generates segment distances by as- 4. Step 2 and 3 are repeated until the alignments sessing the degree of statistical dependence be- of two consecutive iterations do not differ tween the segments x and y: (i.e. convergence is reached).
p(x, y) The potential merit of using PMI-generated seg- PMI(x, y) = log2 (1) p(x) p(y) ment distances can be made clear by the following Where: example. Consider the strings [v"7n] and [v"7ïk@], Bulgarian dialectal variants of the word ‘outside’. • p(x, y): the number of times x and y occur The VC-sensitive Levenshtein algorithm yields at the same position in two aligned strings the following (correct) alignment: X and Y , divided by the total number of aligned segments (i.e. the relative occurrence v "7 n of the aligned segments x and y in the whole v "7 ï k @ dataset). Note that either x or y can be a gap 1 1 1 in the case of insertion or deletion. But also the alternative (incorrect) alignment: • p(x) and p(y): the number of times x (or y) v "7 n occurs, divided by the total number of seg- v "7 ï k @ ment occurrences (i.e. the relative occurrence 1 1 1 of x or y in the whole dataset). Dividing by The VC-sensitive Levenshtein algorithm gener- this term normalizes the empirical frequency ates the erroneous alignment because it has no way with respect to the frequency expected if x to identify that the consonant [n] is nearer to the and y are statistically independent. consonant [ï] than to the consonant [k]. In con- trast, the Levenshtein algorithm which uses the The greater the PMI value, the more segments tend PMI-generated segment distances only generates to cooccur in correspondences. Negative PMI val- the correct first alignment, because the [n] occurs ues indicate that segments do not tend to cooccur relatively more often aligned with [ï] than with in correspondences, while positive PMI values in- [k] so that the distance between [n] and [ï] will dicate that segments tend to cooccur in correspon- be lower than the distance between [n] and [k]. dences. The segment distances can therefore be The idea behind this procedure is similar to Ris- generated by subtracting the PMI value from 0 and tad’s suggestion to learn segment distances for edit adding the maximum PMI value (i.e. lowest dis- distance using an expectation maximization algo- tance is 0). In that way corresponding segments rithm (Ristad and Yianilos, 1998). Our approach obtain the lowest distance. differs from their approach in that we only learn Based on the PMI value and its conversion to segment distances based on the alignments gener- segment distances, we developed an iterative pro- ated by the VC-sensitive Levenshtein algorithm, cedure to automatically obtain the segment dis- while Ristad and Yianilos (1998) learn segment tances: distances by considering all possible alignments of 1. The string alignments are generated using the two strings. VC-sensitive Levenshtein algorithm (see sec- 3.4 The Pair Hidden Markov Model tion 3.1).4 The Pair Hidden Markov Model (PHMM) also 4We also used the Levenshtein algorithm without the vowel-consonant restriction to generate the PMI values, but generates alignments based on automatically gen- this had a negative effect on the performance. erated segment distances and has been used suc-
29 ery segment) and 9604 emission probabilities for the substitution state. The probability of starting in one of the three states is set equal to the probability of going from the substitution state to that particu- lar state. The Baum-Welch expectation maximiza- tion algorithm (Baum et al., 1970) can be used to iteratively reestimate these probabilities until a lo- cal optimum is found. To prevent order effects in training, every word pair is considered twice (e.g., wa − wb and wb − wa). The resulting insertion and deletion probabil- ities are therefore the same (for each segment), and the probability of substituting x for y is equal to Figure 1: Pair Hidden Markov Model. Image the probability of substituting y for x, effectively courtesy of Mackay and Kondrak (2005). yielding 4802 distinct substitution probabilities. Wieling et al. (2007) showed that using Dutch dialect data for training, sensible segment dis- cessfully in language studies (Mackay and Kon- tances were obtained; acoustic vowel distances drak, 2005; Wieling et al., 2007). on the basis of spectrograms correlated signifi- A Hidden Markov Model (HMM) is a proba- cantly (r = −0.72) with the vowel substitution bilistic finite-state transducer that generates an ob- probabilities of the PHMM. Additionally, proba- servation sequence by starting in an initial state, bilities of substituting a symbol with itself were going from state to state based on transition prob- much higher than the probabilities of substitut- abilities and emitting an output symbol in each ing an arbitrary vowel with another non-identical state based on the emission probabilities in that vowel (mutatis mutandis for consonants), which state for that output symbol (Rabiner, 1989). The were in turn much higher than the probabilities of PHMM was originally proposed by Durbin et al. substituting a vowel for a consonant. (1998) for aligning biological sequences and was After training, the well known Viterbi algorithm first used in linguistics by Mackay and Kondrak can be used to obtain the best alignments (Rabiner, (2005) to identify cognates. The PHMM differs 1989). from the regular HMM in that it outputs two ob- servation streams (i.e. a series of alignments of 4 Evaluation pairs of individual segments) instead of only a se- ries of single symbols. The PHMM displayed in As described in section 2, we use the generated Figure 1 has three emitting states: the substitution pairwise alignments from a gold standard of multi- (‘match’) state (M) which emits two aligned sym- ple alignments for evaluation. In addition, we look bols, the insertion state (Y) which emits a symbol at the performance of a baseline of pairwise align- and a gap, and the deletion state (X) which emits ments, which is constructed by aligning the strings a gap and a symbol. according to the Hamming distance (i.e. only al- The following example shows the state se- lowing substitutions and no insertions or deletions; quence for the pronunciations [j"As] and ["Azi] (En- Hamming, 1950). glish ‘I’): The evaluation procedure consists of comparing the alignments of the previously discussed algo- j "A s rithms including the baseline with the alignments "A z i of the gold standard. For the comparison we use XMMY the standard Levenshtein algorithm without any Before generating the alignments, all probabil- restrictions. The evaluation proceeds as follows: ities of the PHMM have to be estimated. These probabilities consist of the 5 transition probabili- 1. The pairwise alignments of the four algo- ties shown in Figure 1: , λ, δ, τXY and τM . In rithms, the baseline and the gold standard are addition there are 98 emission probabilities for the generated and standardized (see section 4.1). insertion state and the deletion state (one for ev- When multiple equal-scoring alignments are
30 generated by an algorithm, only one (i.e. the 4.1 Standardization final) alignment is selected. The gold standard contains a number of align- 2. In each alignment, we convert each pair of ments which have alternative equivalent align- aligned segments to a single token, so that ev- ments, most notably an alignment containing an ery alignment of two strings is converted to a insertion followed by a deletion (which is equal single string of segment pairs. to the deletion followed by the insertion), or an alignment containing a syllabic consonant such as 3. For every algorithm these transformed strings ["ô], which in fact matches both a vowel and a " are aligned with the transformed strings of neighboring r-like consonant and can therefore be the gold standard using the standard Leven- aligned with either the vowel or the consonant. In shtein algorithm. order to prevent punishing the algorithms which do not match the exact gold standard in these 4. The Levenshtein distances for all these cases, the alignments of the gold standard and all strings are summed up resulting in the total alignment algorithms are transformed to one stan- distance between every alignment algorithm dard form in all relevant cases. and the gold standard. Only if individual For example, consider the correct alignment of segments match completely the segment dis- [v"iA] and [v"ij], two Bulgarian dialectal variations tance is 0, otherwise it is 1. of the English plural pronoun ‘you’: To illustrate this procedure, consider the following v "i A gold standard alignment of [vl"7k] and [v"7lk], two v "i j Bulgarian dialectal variants of the word ‘wolf’: Of course, this alignment is as reasonable as: v l "7 k v "i A v "7 l k v "i j Every aligned segment pair is converted to a single To avoid punishing the first, we transform all in- token by adding the symbol ‘/’ between the seg- sertions followed by deletions to deletions fol- ments and using the symbol ‘-’ to indicate a gap. lowed by insertions, effectively scoring the two This yields the following transformed string: alignments the same. For the syllabic consonants we transform all v/v l/"7 "7/l k/k alignments to a form in which the syllabic con- Suppose another algorithm generates the follow- sonant is followed by a gap and not vice versa. ing alignment (not detecting the swap): For instance, aligning [v"ôx] with [v"Arx] (English: " v l "7 k ‘peak’) yields: v "7 l k v "ô x " The transformed string for this alignment is: v "A r x Which is transformed to the equivalent alignment: v/v l/- "7/"7 -/l k/k v "ô x To evaluate this alignment, we align this string to v "A" r x the transformed string of the gold standard and ob- tain a Levenshtein distance of 3: 5 Results v/v l/"7 "7/l k/k We will report both quantitative results using the v/v l/- "7/"7 -/l k/k evaluation method discussed in the previous sec- 1 1 1 tion, as well as the qualitative results, where we By repeating this procedure for all alignments and focus on characteristic errors of the different align- summing up all distances, we obtain total dis- ment algorithms. tances between the gold standard and every align- ment algorithm. Algorithms which generate high- 5.1 Quantitative results quality alignments will have a low distance from Because there are two algorithms which use gen- the gold standard, while the distance will be higher erated segment distances (or probabilities) in their for algorithms which generate low-quality align- alignments, we first check if these values are sen- ments. sible and comparable to each other.
31 5.1.1 Comparison of segment distances The results are shown in Table 1. We can With respect to the PMI results (convergence clearly see that all algorithms beat the baseline was reached after 7 iterations, taking less than and align about 95% of all string pairs correctly. 5 CPU minutes), we indeed found sensible re- While the Levenshtein PMI algorithm aligns most sults: the average distance between identical sym- strings perfectly, it misaligns slightly more indi- bols was significantly lower than the distance be- vidual segments than the PHMM and the Leven- tween pairs of different vowels and consonants shtein algorithm with the swap operation (i.e. it (t < −13, p < .001). Because we did not allow makes more segment alignment errors per word vowel-consonants alignments in the Levenshtein pair). The VC-sensitive Levenshtein algorithm algorithm, no PMI values were generated for those in general performs slightly worse than the other segment pairs. three algorithms. Just as Wieling et al. (2007), we found sen- sible PHMM substitution probabilities (conver- 5.2 Qualitative results gence was reached after 1675 iterations, taking Let us first note that it is almost impossible for about 7 CPU hours): the probability of matching any algorithm to achieve a perfect overlap with the a symbol with itself was significantly higher than gold standard, because the gold standard was gen- the probability of substituting one vowel for an- erated from multiple alignments and therefore in- other (similarly for consonants), which in turn was corporates other constraints. For example, while a higher than the probability of substituting a vowel certain pairwise alignment could appear correct in with a consonant (all t’s > 9, p < .001). aligning two consonants, the multiple alignment To allow a fair comparison between the PHMM could show contextual support (from pronuncia- probabilities and the PMI distances, we trans- tions in other varieties) for separating the conso- formed the PHMM probabilities to log-odds nants. Consequently, all algorithms discussed be- scores (i.e. dividing the probability by the rela- low make errors of this kind. tive frequency of the segments and subsequently In general, the specific errors of the VC- taking the log). Because the residues after the sensitive Levenshtein algorithm can be separated linear regression between the PHMM similarities into three cases. First, as we illustrated in section and PMI distances were not normally distributed, 3.3, the VC-sensitive Levenshtein algorithm has we used Spearman’s rank correlation coefficient no way to distinguish between aligning a conso- to assess the relationship between the two vari- nant with one of two neighboring consonants and ables. We found a highly significant Spearman’s sometimes chooses the wrong one (this also holds ρ = −.965 (p < .001), which means that the re- for vowels). Second, it does not allow alignments lationship between the PHMM similarities and the of vowels with consonants and therefore cannot PMI distances is very strong. When looking at the detect correct vowel-consonant alignments such as insertions and deletions we also found a significant correspondences of [u] with [v] initially. Third, relationship: Spearman’s ρ = −.736 (p < .001). for the same reason the VC-sensitive Levenshtein 5.1.2 Evaluation against the gold standard algorithm is also not able to detect metathesis of Using the procedure described in section 4, we cal- vowels with consonants. culated the distances between the gold standard The misalignments of the Levenshtein algo- and the alignment algorithms. Besides reporting rithm with the swap-operation can also be split in the total number of misaligned tokens, we also di- three cases. It suffers from the same two prob- vided this number by the total number of aligned lems as the VC-sensitive Levenshtein algorithm in segments in the gold standard (about 16 million) choosing to align a consonant incorrectly with one to get an idea of the error rate. Note that the error of two neighboring consonants and not being able rate is 0 in the perfect case, but might rise to nearly to align a vowel with a consonant. Third, even 2 in the worst case, which is an alignment consist- though it aligns some of the metathesis cases cor- ing of only insertions and deletions and therefore rectly, it also makes some errors by incorrectly ap- up to twice as long as the alignments in the gold plying the swap-operation. For example, consider standard. Finally, we also report the total number the alignment of [s"irjIni] and [s"irjnI], two Bul- of alignments (word pairs) which are not exactly garian dialectal variations of the word ‘cheese’, in equal to the alignments of the gold standard. which the swap-operation is applied:
32 Algorithm Misaligned segments (error rate) Incorrect alignments (%) Baseline (Hamming algorithm) 2510094 (0.1579) 726844 (20.92%) VC-sens. Levenshtein algorithm 490703 (0.0309) 191674 (5.52%) Levenshtein PMI algorithm 399216 (0.0251) 156440 (4.50%) Levenshtein swap algorithm 392345 (0.0247) 161834 (4.66%) Pair Hidden Markov Model 362423 (0.0228) 160896 (4.63%)
Table 1: Comparison to gold standard alignments. All differences are significant (p < 0.01).
s "i rj I n i In addition, we introduced the PMI weight- s "i rj n I ing within the Levenshtein algorithm as a sim- 0 0 0 >< 1 1 ple means of obtaining segment distances, and However, the two I’s are not related and should not showed that it improves on the popular Leven- be swapped, which is reflected in the gold standard shtein algorithm with respect to alignment accu- alignment: racy. j s "i r I n i While the results indicated that the PHMM mis- j s "i r n I aligned the fewest segments, training the PHMM 0 0 0 1 0 1 is a lengthy process lasting several hours. Con- The incorrect alignments of the Levenshtein sidering that the Levenshtein algorithm with the algorithm with the PMI-generated segment dis- swap operation and the Levenshtein algorithm tances are mainly caused by its inability to align with the PMI-generated segment distances are vowels with consonants and therefore, just as the much quicker to (train and) apply, and that they VC-sensitive Levenshtein algorithm, it fails to de- have only slightly lower performance with respect tect metathesis. On the other hand, using seg- to the segment alignments, we actually prefer us- ment distances often solves the problem of select- ing those methods. Another argument in favor of ing which of two plausible neighbors a consonant using one of these Levenshtein algorithms is that should be aligned with. it is a priori clearer what type of alignment errors Because the PHMM employs segment substi- to expect from them, while the PHMM algorithm tution probabilities, it also often solves the prob- is less predictable and harder to comprehend. lem of aligning a consonant to one of two neigh- While our results are an indication of the good bors. In addition, the PHMM often correctly quality of the evaluated algorithms, we only evalu- aligns metathesis involving equal as well as sim- ated the algorithms on a single dataset for which a ilar symbols, even realizing an improvement over gold standard was available. Ideally we would like the Levenshtein swap algorithm. Unfortunately, to verify these results on other datasets, for which many wrong alignments of the PHMM are also gold standards consisting of multiple or pairwise caused by allowing vowel-consonant alignments. alignments are available. Since the PHMM does not take context into ac- count, it also aligns vowels and consonants which often play a role in metathesis when no metathesis is involved. Acknowledgements 6 Discussion We are grateful to Peter Kleiweg for extending the This study provides an alternative evaluation of Levenshtein algorithm in the L04 package with the string distance algorithms by focusing on their ef- swap-operation. We also thank Greg Kondrak for fectiveness in aligning segments. We proposed, providing the original source code of the Pair Hid- implemented, and tested the new procedure on a den Markov Models. Finally, we thank Therese substantial body of data. This provides a new per- Leinonen and Sebastian Kurschner¨ of the Univer- spective on the quality of distance and alignment sity of Groningen and Esteve Valls i Alecha of the algorithms as they have been used in dialectology, University of Barcelona for their useful feedback where aggregate comparisons had been at times on our ideas. frustratingly inconclusive.
33 References Bruno Pouliquen. 2008. Similarity of names across scripts: Edit distance using learned costs of N- Leonard E. Baum, Ted Petrie, George Soules, and Nor- Grams. In Bent Nordstrom¨ and Aarne Ranta, ed- man Weiss. 1970. A maximization technique occur- itors, Proceedings of the 6th international Con- ring in the statistical analysis of probabilistic func- ference on Natural Language Processing (Go- tions of Markov Chains. The Annals of Mathemati- Tal’2008), volume 5221, pages 405–416. cal Statistics, 41(1):164–171. Jelena Prokic,´ Martijn Wieling, and John Nerbonne. Kenneth W. Church and Patrick Hanks. 1990. Word 2009. Multiple sequence alignments in linguistics. association norms, mutual information, and lexicog- In Piroska Lendvai and Lars Borin, editors, Proceed- raphy. Computational Linguistics, 16(1):22–29. ings of the EACL 2009 Workshop on Language Tech- nology and Resources for Cultural Heritage, Social Fred J. Damerau. 1964. A technique for computer de- Sciences, Humanities, and Education. tection and correction of spelling errors. Communi- cations of the ACM, 7:171–176. Lawrence R. Rabiner. 1989. A tutorial on Hidden Markov Models and selected applications in speech Richard Durbin, Sean R. Eddy, Anders Krogh, and recognition. Proceedings of the IEEE, 77(2):257– Biological Sequence Graeme Mitchison. 1998. 286. Analysis: Probabilistic Models of Proteins and Nu- cleic Acids. Cambridge University Press, United Eric Sven Ristad and Peter N. Yianilos. 1998. Learn- Kingdom, July. ing string-edit distance. IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 20:522– Richard Hamming. 1950. Error detecting and error 532. correcting codes. Bell System Technical Journal, 29:147–160. Robert Wagner and Roy Lowrance. 1975. An exten- sion of the string-to-string correction problem. Jour- Wilbert Heeringa, Peter Kleiweg, Charlotte Gooskens, nal of the ACM, 22(2):177–183. and John Nerbonne. 2006. Evaluation of string dis- tance algorithms for dialectology. In John Nerbonne Martijn Wieling and John Nerbonne. 2007. Dialect and Erhard Hinrichs, editors, Linguistic Distances, pronunciation comparison and spoken word recog- pages 51–62, Shroudsburg, PA. ACL. nition. In Petya Osenova, editor, Proceedings of the RANLP Workshop on Computational Phonology, Wilbert Heeringa. 2004. Measuring Dialect Pronunci- pages 71–78. ation Differences using Levenshtein Distance. Ph.D. thesis, Rijksuniversiteit Groningen. Martijn Wieling, Therese Leinonen, and John Ner- bonne. 2007. Inducing sound segment differences Brett Kessler. 1995. Computational dialectology in using Pair Hidden Markov Models. In Mark Ellison Irish Gaelic. In Proceedings of the seventh con- John Nerbonne and Greg Kondrak, editors, Comput- ference on European chapter of the Association for ing and Historical Phonology: 9th Meeting of the Computational Linguistics, pages 60–66, San Fran- ACL Special Interest Group for Computational Mor- cisco, CA, USA. Morgan Kaufmann Publishers Inc. phology and Phonology, pages 48–56. Grzegorz Kondrak and Bonnie Dorr. 2003. Identifica- tion of Confusable Drug Names: A New Approach and Evaluation Methodology. Artificial Intelligence in Medicine, 36:273–291.
Grzegorz Kondrak. 2003. Phonetic Alignment and Similarity. Computers and the Humanities, 37:273– 291.
Vladimir Levenshtein. 1965. Binary codes capable of correcting deletions, insertions and reversals. Dok- lady Akademii Nauk SSSR, 163:845–848.
Wesley Mackay and Grzegorz Kondrak. 2005. Com- puting word similarity and identifying cognates with Pair Hidden Markov Models. In Proceedings of the 9th Conference on Computational Natural Lan- guage Learning (CoNLL), pages 40–47, Morris- town, NJ, USA. Association for Computational Lin- guistics.
John Nerbonne and Peter Kleiweg. 2007. Toward a di- alectological yardstick. Journal of Quantitative Lin- guistics, 14:148–167.
34 A web-enabled and speech-enhanced parallel corpus of Greek - Bulgarian cultural texts
Voula Giouli Nikos Glaros Institute for Language & Speech Institute for Language & Speech Processing Athens, Greece Processing Athens, Greece [email protected] [email protected]
Kiril Simov Petya Osenova Institute for Parallel Processing, Institute for Parallel Processing, BAS, Sofia, Bulgaria BAS, Sofia, Bulgaria [email protected] [email protected]
tion 5 presents the Language Technologies (LT) Abstract deployed in the project elaborating on the Greek and the Bulgarian text processing tools, and dis- This paper reports on completed work carried cusses the LT methods that have been (a) ex- out in the framework of an EU-funded project ploited in the course of the project to facilitate aimed at (a) developing a bilingual collection the web-interface construction and (b) integrated of cultural texts in Greek and Bulgarian, (b) in the search and retrieval mechanisms to im- creating a number of accompanying resources prove the system performance. Finally, Section 6 that will facilitate study of the primary texts across languages, and (c) integrating a system describes the main components of the web inter- which aims to provide web-enabled and face and the way various features are exploited to speech-enhanced access to digitized bilingual facilitate users’ access to the data. In the last sec- Cultural Heritage resources. This simple user tion, we present conclusions and future work. interface, which incorporates advanced search mechanisms, also offers innovative accessibil- 2 Project description ity for visually impaired Greek and Bulgarian users. The rationale behind the work (and the The project aims at highlighting cultural re- relative resource) was to promote the com- sources that, as of yet, remain non-exploited to parative study of the cultural heritage of the their greatest extent, and at creating the neces- two countries. sary infrastructure with the support of LT with a view to promoting the study of cultural heritage 1 Introduction of the eligible neighboring areas and raising awareness about their common cultural identity. The document describes a bilingual Greek (EL) To serve these objectives, the project had a con- and Bulgarian (BG) collection of literary and crete target, that is, the creation of a textual col- folklore texts along with the metadata that were lection and of accompanying material that would deemed necessary for the efficient management be appropriate for the promotion and study of the and retrieval of the textual data. Section 2 out- cultural heritage of the neighboring areas in lines the project aims that guided selection and Greece and Bulgaria (Thrace and the neighboring annotation of the texts, whereas Section 3 pre- Smolyan, Blagoevgrad, Kardjali, Khaskovo ar- sents the primary data that comprise the bilingual eas), the focus being on literature, folklore and textual collection and the methodology adopted language. To this end, the main activities within for collecting them. Section 4 elaborates on the the project life-cycle were to: metadata scheme that has been implemented to • record and roadmap the literary production describe the primary data and the linguistic anno- of the afore mentioned areas spanning from tation tailored to facilitate search and retrieval at th the 19 century till the present days along the document, phrase or word level. This scheme with written records on folk culture and folk- is compliant to widely accepted standards so as tales from the eligible areas. These should to ensure reusability of the resource at hand. Sec- form a pool of candidate texts from which Proceedings of the EACL 2009 Workshop on Language Technology and Resources for Cultural Heritage, Social Sciences, Humanities, and Education –LaTeCH – SHELT&R 2009, pages 35–42, Athens, Greece, 30 March 2009. c 2009 Association for Computational Linguistics
35 the most appropriate for the project objec- areas of Greece and Bulgaria the focus being tives could be selected; on literature, folklore and folktales; • record and roadmap existing translations of • to provide a representative sample of (a) lit- literary works in both languages to serve for erature written by authors from Thrace -that the creation of the parallel corpus; is from the entire area of Thrace- or about • select textual material representative of the Thrace, spanning between the 19th century - two cultures, and thus, suitable for their today, (b) folklore texts about Thrace, that comparative study; would normally reflect cultural as well as • digitize the selected (printed) material to a linguistic elements either shared by the two format suitable for long-term preservation; people or unique to each culture, and (c) • collect meta-texts relevant to the selected folktales and legends from Thrace, the latter literary and folklore texts, that is, texts about being the intermediate between literature and the literary works, biographies of the se- folklore. lected authors, criticism, etc.; these comprise part of the accompanying material In order to gather the candidate texts and au- • document the data with any information thors for such a collection we exploited both deemed necessary for its preservation and printed and digitized sources, i.e., (on-line and exploitation, catering for their interrelation printed) anthologies of Bulgarian, Greek or Bal- so as to highlight their common features and kan literature, digital archives, web resources and allow unified access to the whole set along library material. The outcome of this extensive text types / genres and languages; research was a wealth of literary works including • extract bilingual glossaries from the primary titles by the most prominent authors in Bulgaria collection of literary and folklore texts also and Greece. The selection of the authors, who accounted for as accompanying material; the would finally participate in GBCC, was based on project caters for the extraction of EL and the following criteria: (a) author's impact to BG terms and names of Persons and Loca- Greek or Bulgarian literature respectively; and tions and their translation equivalents in the (b) author's contribution to his county's folk other language; study or other major sectors such as journalism • make the primary resource along with the and education. accompanying material (meta-texts and glos- Additionally, to ensure corpus “representa- saries) publicly available over the internet to tiveness” to some extend, we tried to include the all interested parties, ranging from the re- full range of the literary texts (poetry, fiction, search community to laypersons, school stu- short stories) and in proportion to the literary dents and people interested in finding out production with respect to the parameters of more about the particular areas; place, time and author. To this end, we think we have avoided biases and the corpus models all • facilitate access to the material that wouldn’t language varieties spoken in the areas and at dif- be hampered by users’ computer literacy ferent periods. and/or language barriers. To cater for the lat- Moreover, the "inner" content characteristics ter, the web interface would be as simple as of texts were used as the basic criteria for text possible – yet functional – and the data selection. To this end, we chose texts which should be available in both languages (Greek demonstrate the two people's cultural similarities and Bulgarian) plus in English. and affinity along with each author's most impor- 3 The bilingual Greek – Bulgarian Cul- tant and representative works. Beyond the above, tural Corpus the availability of a translation in the other lan- guage and IPR issues also influenced text selec- Along with the aforementioned lines, the col- tion. lection comprises parallel EL – BG literary and The collection of the primary data currently folklore texts. The main specifications for the comprises of (135) literary works, (70) BG (Bul- Greek - Bulgarian Cultural Corpus (GBCC) crea- garian) and 65 EL (Greek). Moreover, (30) BG tion were: folk texts and 30 EL folk texts along with (25) • to build a bilingual resource that could be BG folktales and 31 EL folktales were added in used as a means to study cultural similarities order to build a corpus as balanced as possible and/or differences between the neighboring and representative of each country's culture. In terms of tokens, the corpus amounts to 700,000
36 in total (circa 350,000 tokens per language): the CES, http://www.cs.vassar.edu/CES/CES1- literature part is about 550,000 tokens, whereas, 0.html), which has been proposed by EAGLES the folklore and legend sub-corpus is about (http://www.ilc.cnr.it/EAGLES96/home.html) 150,000 tokens. and is compliant with the specifications of the Moreover, to cater for the project requirement Text Encoding Initiative (http://www.tei-c.org, that the corpus should be bilingual, available Text Encoding Initiative (TEI Guidelines for translations of the primary EL – BG literary Electronic Text Encoding and Interchange). works were also selected to form the parallel lit- From the total number of elements proposed by erary corpus. Additionally, an extensive transla- these guidelines, the annotation of the parallel tion work was also carried out by specialized corpus at hand has been restricted to the recogni- translators where applicable (folklore texts and tion of structural units at the sentence level, folktales). which is the minimum level required for the The collection covers EL and BG literary pro- alignment and term extraction processes. That duction dating from the 19th century till the pre- means that the requirements of CES Level 1 con- sent day, and also texts (both literary or folklore) formance are met; as regards CES Level 2 the that are written in the dialect(s) used in the eligi- requirements (but not the recommendations) are ble areas. This, in effect, is reflected in the lan- also met, and from CES Level 3 requirements, guage varieties represented in the textual collec- annotation for sentence boundaries is met. tion that range from contemporary to non- Additionally, metadata elements have been contemporary, and from normal to dialectical or deployed which encode information necessary even mixed language. for text indexing with respect to text title, author, Finally, the collection of primary data was publisher, publication date, etc. (bibliographical also coupled with accompanying material (con- information) and for the classification of each tent metadata) for each literary work (literary text according to text type/genre and topic, the criticism) and for each author (biographical in- latter being applicable to folklore texts and folk formation, list of works, etc.). Along with all the tales. Classification of folklore texts is based on above, texts about the common cultural elements the widely accepted Aarne-Thompson classifica- were also included. tion system (Aarne, 1961). To this end, to assure documentation com- 4 Corpus Annotation pleteness, and facilitate the inter-relation among primary data and the accompanying material (bi- After text selection, digitization and extended ographies, criticism, etc) the documentation manual validation (where appropriate) were per- scheme has been extended accordingly. The formed. Normalization of the primary data was aforementioned metadata descriptions are kept kept to a minimum so as to cater, for example, separately from the data in an xml header that is for the conversion from the Greek polytonic to to be deployed by the web interface for search the monotonic encoding system. Furthermore, to and retrieval purposes. ensure efficient content handling and retrieval The external structural annotation (including and also to facilitate access to the resource at text classification) of the corpus also adheres to hand via the platform that has been developed, the IMDI metadata scheme (IMDI, Metadata metadata descriptions and linguistic annotations Elements for Session Descriptions, Version were added across two pillars: (a) indexing and 3.0.4, Sept. 2003). Adaptations proposed specifi- retrieval, and (b) further facilitating the compara- cally concerning Written Language Resources tive study of textual data. To this end, metadata have been taken into account. IMDI metadata descriptions and linguistic annotations compliant elements for catalogue descriptions (IMDI, with internationally accepted standards were Metadata Elements for Catalogue Descriptions, added to the raw material. The metadata scheme Version 2.1, June 2001) were also taken into ac- deployed in this project is compliant with inter- count to render the corpus compatible with exist- nationally accredited standards with certain ing formalisms (ELRA, and LDC). This type of modifications that cater for the peculiarities of metadata descriptions was added manually to the the data. texts. More specifically, the metadata scheme im- To further enhance the capabili- plemented in this project builds on XCES, the ties/functionalities of the final application, ren- XML version of the Corpus Encoding Standard dering, thus the collection a useful resource to (XCES, http://www.cs.vassar.edu/XCES/ and prospective users and researchers, further annota-
37 tions at various levels of linguistic analysis were • Handling and tokenization; following com- integrated across two pillars: (a) efficient index- mon practice, the Greek tokenizer makes use ing and retrieval; and (b) further facilitating the of a set of regular expressions, coupled with comparative study of textual data by means of precompiled lists of abbreviations, and a set bilingual glossaries which were constructed of simple heuristics (Papageorgiou et al., semi-automatically, and via the visualization of 2002) for the recognition of word and sen- aligned parallel texts. tence boundaries, abbreviations, digits, and Text processing at the monolingual level com- simple dates. prises the following procedures: (a) handling and tokenization, (b) Part-of-Speech (POS) tagging • POS-tagging and lemmatization; a tagger and lemmatization, (c) surface syntactic analysis, that is based on Brill's TBL architecture (d) indexing with terms/keywords and (Brill, 1997), modified to address peculiari- phrases/Named Entities (NEs) pertaining to the ties of the Greek language (Papageorgiou et types Location (LOC) and Person (PER). al., 2000) was used in order to assign mor- Annotations at these levels were added semi- phosyntactic information to tokenized words. automatically, by deploying existing generic Furthermore, the tagger uses a PAROLE- Natural Language Processing (NLP) tools that compliant tagset of 584 different part-of- were developed for the languages at hand, speech tags. Following POS tagging, lemmas whereas extensive and intensive validations were are retrieved from a Greek morphological performed via several ways. Indeed, although the lexicon. tools deployed have reported to achieve high ac- • Surface syntactic analysis; the Greek chun- curacy rates in the domains/genres they were ker is based on a grammar of 186 rules intended for, the specific nature of the data led to (Boutsis et al., 2000) developed for the a significant reduction. To this end, half of the automatic recognition of non-recursive annotations were checked manually. After the phrasal categories: adjectives, adverbs, identification of the errors in this part of the cor- prepositional phrases, nouns, verbs (chunks) pus, we have performed a manual check in the (Papageorgiou et al., 2002). second part of the corpus only for these cases • Term extraction; a Greek Term Extractor which were recognized as errors during the vali- was used for spotting terms and idiomatic dation of the first part. For some of the cases words (Georgantopoulos, Piperidis, 2000). relevant constraints in the systems were written, Term Extractor's method proceeds in three which automatically find places where some pipelined stages: (a) morphosyntactic anno- rules were not met. Tools customization was also tation of the domain corpus, (b) corpus pars- performed by adding new rules applicable for the ing based on a pattern grammar endowed language varieties to be handled, and also by ex- with regular expressions and feature- tending/modifying the resources used (word and structure unification, and (c) lemmatization. name lists, etc.). Candidate terms are then statistically evalu- Finally, alignment of parallel texts (primary ated with an aim to skim valid domain terms source documents and their translations) has also and lessen the overgeneration effect caused been performed at both sentence and phrase by pattern grammars (hybrid methodology). level. As expected, poems posited the major dif- Named Entity Recognition was then per- ficulties due the fuzziness in identifying sentence formed using MENER (Maximum Entropy boundaries, and alignments at the phrase level Named Entity Recognizer), a system compatible were favored instead. with the ACE (Automatic Content Extraction) scheme, catering for the recognition and classifi- 5 Language Technologies cation of the following types of NEs: person (PER), organization (ORG), location (LOC) and In what follows the Greek and Bulgarian Text geopolitical entity (GPE) (Giouli et al., 2006). Processing Components will be described. 5.2 Bulgarian Tools 5.1 The Greek pipe-line In the processing of the Bulgarian part of the In the case of the Greek data, text processing corpus we have been using generic language was applied via an existing pipeline of shallow technology tools developed for Bulgarian. Here processing tools for the Greek language. These is the list of tools that we have used. They are include:
38 implemented within the CLaRK System (Simov different text types/genres. Moreover, the data at et al. 2001) via: hand posed another difficulty that is, coping with Tokenization, Morphosyntactic tagging, older/obsolete language usage. In fact, some of Lemmatization; Tokenization is implemented as the literary works were written in the 19th cen- a hierarchy of tokenizers within the CLaRK sys- tury or the beginning of 20th century, and their tem. Morphosyntactic tagging is done on the ba- language reflects the writing standards of the sis a morphological lexicon which covers the corresponding period. grammatical information of about 100 000 lex- Therefore, as it was expected, the overall per- emes (1 600 000 word forms); a gazetteers of formance of the afore-mentioned tools was lower about 25000 names and 1500 abbreviations. We than the one reported for the texts these tools are using the BulTreeBank tagset, which is a were initially trained for. more specialized version of Multext-east tagset. To this end, performance at POS-tagging level The disambiguation is done in two steps. Ini- dropped from 97% to 77% for the Greek data tially, a rule-based module solves the sure cases since no normalization of the primary data was for which manual rules can be written. Then, for performed. On the other hand, the BG morpho- the next step, a neural-network-based disam- logical analyzer coverage, whose benchmark per- biguator is being exploited (Simov and Osenova formance is 96% 2001). Lemmatization is implemented as rules dropped to 92 % on poems and folktales and to which convert each word form in the lemma. The 94% on literary texts and legends. rules are assigned to the word forms in the lexi- The reason was that the language of processed con. This ensures very high level of accuracy. literary texts and legends came normalized from Partial Grammars have also been constructed the sources, while the poems and folktales kept for Sentence splitting, Named-entity recognition, some percentage of archaic or dialect words. and Chunking. Thus, additionally to the guesser, a post POS processing was performed on the unknown 5.3 Alignments words. Moreover, the accuracy of the neural To facilitate the comparative study of parallel network disambiguator and the rule-based one documents, source texts were automatically was 97 %. i.e. the same as for other applications. aligned with their translations. Alignments at the Processing at the levels of chunks and NEs were sentence level were performed semi- even lower. Within the project we had to tune the automatically by means of the ILSP Aligner, tools to the specific language types, such as dia- which is a language independent tool that uses chronically remote texts and domain specific surface linguistic information coupled with in- texts (folklore). Also, some words with higher formation about possible unit delimiters depend- distribution in the target regions appear in some ing on the level at which the alignment is sought. of the works. In order to deal with them we had The resulting translation equivalents were stored to extend the used lexicons, to create a guesser in files conformant to the internationally accred- for the unknown words and add new rules to the ited TMX standard (Translation Memory eX- chunk grammar to handle some specific word change, http://www.lisa.org/tmx/), which is order within the texts. XML-compliant, vendor-neutral open standard Additionally, the deployment of tools that are for storing and exchanging translation memories specific to each language and compatible with created by Computer Aided Translation (CAT) completely distinct annotation standards brought and localization tools. about the issue of metadata harmonization. To Moreover, terms pertaining to the folklore do- this end, although the Greek tools were devel- main as well as names of Persons and Locations oped to confront to the afore-mentioned annota- identified in the EL - BG parallel texts were tion standards, this was not the case for Bulgar- semi-automatically aligned. The outcome of the ian. The first encoding scheme followed the process of text alignment at below the sentence BulTreeBank morphological and chunk level was then validated manually. annotation scheme. Afterwards, the information was transferred into the project 5.4 Tools Customization and metadata scheme in order to be consistent with the Greek harmonization data and applicable for web representation. As a As it has already been stated, the tools that result, the morphosyntactic features of the BG were deployed for the linguistic processing are tagset, which is a more specialized version of the generic ones that were initially developed for
39 Multext-East tagset were mapped onto the rela- glossary of terms and the aligned list of NEs. The tive PAROLE tags. latter is often very interesting, especially with respect to Location entities, since transliteration 6 The web interface is usually non-adequate. The design of the web interface effectively All the data collected (being the primary liter- blends simplicity and advanced functionality so ary or folklore texts or meta-documents, etc.) that to fully support the intended usage scenarios along with their translations, the multi-layered (comparative study of literary and folklore texts annotations, and the resulting glossaries were equally by specialists, laymen or students, lan- integrated in a database platform that was devel- guage and/or literary teaching and learning, lexi- oped to serve as a content management system. cographic projects, etc.). Finally, the web inter- Being the backbone of that platform, the meta- face has been enhanced by integrating last gen- data material facilitates the interlinking of similar eration of synthetic speech technology for both documents, and the access to the primary data Greek and Bulgarian. This speech-enhanced user via the web. To this end, a specially designed interface (S. Raptis et al, 2005), offers innovative web site was developed to satisfy the needs of web accessibility for blind and vision impaired end-users (the general public and the special Greek and Bulgarian users as well as for other groups of researchers and other scientists). The users who use speech as their preferable modal- website features a trilingual interface (Greek, ity to information access. The key-feature of this Bulgarian, English) as well as advanced search web-speech technology is that it lets users to in- and retrieval mechanisms on the entire bilingual teract with the underlying system; so that they content or a user-specified part of it. The users can hear only the portions of a specific web page can perform combined searches by author name, they are interested in, being able at the same time title, genre, etc. Furthermore, they can search for to navigate through the entire web site and visit single keywords/wordforms or for two word- only the web pages of their choice. forms that can be a user-specified number of words apart from each other. Searches by lemma 7 Conclusions and future work and/or by phrase have been also implemented. The latter rely on a matcher, which tries to link We have described work targeted at the promo- the query word(s) with the stored lem- tion and study of the cultural heritage of the mas/wordforms. Additionally, a stemmer for cross-border regions of Greece – Bulgaria, the Greek and Bulgarian has been used for the on- focus been on literature, folklore and language of line stemming of queries, which will then be the two people, by means of modern and techno- matched with the already stemmed corpus. When logically advanced platforms. To this end, a digi- all the above fails, fuzzy matching techniques are tal collection of literary and folklore texts has being employed, facilitating, thus, effective been compiled along with accompanying mate- query expansion functionality. Finally, apart rial selected from various (online and printed from wordforms and lemmas, the collection can sources), which is integrated into a platform with also be queried for morphosyntactic tags or any advanced search and retrieval mechanisms. combination thereof; results, then, come in the However, the cultural value of the bilingual cul- form of concordances and statistics (frequency tural Greek-Bulgarian corpus goes beyond the information), hence the relative document(s) can border areas that it was intended for, because it also be retrieved. Moreover, users can search the shows the similarities and the differences be- whole corpus or define a sub-corpus based on the tween the two neighboring countries. More spe- classification and annotation parameters accom- cifically, it can be used for supporting the acqui- panying each text, thus, creating sub-corpora of a sition of the other language in both countries. specific author, or belonging to a specific genre, Also, it can be explored for comparing the cul- text type, domain, time period, etc. tural and social attitudes in diachronic depth and In addition, the web interface lets the users to genre variety. Apart from the usages from a hu- simultaneously view on screen both Greek and manities point of view, the corpus can become a Bulgarian texts, aligned and in parallel,, so that good base for testing taggers, parsers and align- to become acquainted with the comparative as- ers. It would especially challenge the processing pects of the two languages or perform specific of the regional dialects, the language of poems, linguistic, lexicographic or translation tasks. Al- and the language of non-contemporary works. ternatively, the user can consult the bilingual
40 Future work is being envisaged in the following and Stelios Piperidis. 2002. Multi-level XML-based directions: extending the corpus with more texts, Corpus Annotation. Proceedings of the 3nd Lan- and respectively the glossaries – with more guage and Resources Evaluation Conference. terms, adding more layers of linguistic analysis Harris Papageorgiou, Prokopis Prokopidis, Voula (predicate-argument structure, etc.), and further Giouli, and Stelios Piperidis. 2000. A Unified POS enhance search and retrieval with the construc- Tagging Architecture and its Application to Greek. tion and deployment of an applicable thesaurus. In Proceedings of the 2nd Language and Resources Evaluation Conference, Athens, Greece, pp 1455- Acknowledgments 1462. We would like to thank the anonymous review- Stelios Piperidis. 1995. Interactive corpus based ers for useful suggestions and comments. Most translation drafting tool. In ASLIB Proceedings of the work presented in this paper was done in 47(3), March 1995. the framework of a project that was funded under Spyros Raptis, I. Spais and P. Tsiakoulis. 2005. A the Community Initiative Programme INTER- Tool for Enhancing Web Accessibility: Synthetic REG III A / PHARE CBC Greece – Bulgaria. Speech and Content Restructuring”. In Proc. HCII The project was implemented by the Institute for 2005: 11th International Conference on Human- Language and Speech Processing (ILSP, Computer Interaction, 22-27 July, Las Vegas, Ne- www.ilsp.gr) and a group of academics and re- vada, USA. searchers from the Sofia University St. Kliment Kiril Simov, Z. Peev, M. Kouylekov, A. Simov, M. Ohridski (www.uni-sofia.bg). Dimitrov, and A. Kiryakov. 2001. CLaRK - an XML-based System for Corpora Development. References Corpus Linguistics 2001 Conference. pp 558-560. Antti Aarne. 1961. The Types of the Folktale: A Clas- Kiril Simov, and Petya Osenova. A Hybrid System for sification and Bibliography. Translated and MorphoSyntactic Disambiguation in Bulgarian. In: Enlarged by Stith Thompson. 2nd rev. ed. Helsinki: Proc. of the RANLP 2001 Conference, Tzigov Suomalainen Tiedeakatemia / FF Communications. Chark, Bulgaria, 5-7 September 2001. pages 288- 290. Sotiris Boutsis, Prokopis Prokopidis, Voula Giouli and Stelios Piperidis. 2000. A Robust Parser for René Witte, Thomas Gitzinger, Thomas Kappler, and Unrestricted Greek Tex. In Proceedings of the 2nd Ralf Krestel. 2008. A Semantic Wiki Approach to Language and Resources Evaluation Conference, Cultural Heritage Data Management. Language 467-473, Athens, Greece. Technology for Cultural Heritage Data (LaTeCH 2008), Workshop at LREC 2008, June 1st, 2008, Michel Généreux. 2007. Cultural Heritage Digital Marrakech, Morocco. Resources: From Extraction to Querying, Lan- guage Technology for Cultural Heritage Data (LaTeCH 2007), Workshop at ACL 2007, June 23rd–30th 2007, Prague, Czech Republic. Byron Georgantopoulos and Stelios Piperidis, 2000. Term-based Identification of Sentences for Text Summarization. In Proceedings of LREC2000 Voula Giouli, Alexis Konstandinidis, Elina Desypri, Harris Papageorgiou. 2006. Multi-domain Multi- lingual Named Entity Recognition: Revisiting & Grounding the resources issue. In Proceedings of LREC 2006. IMDI, Metadata Elements for Catalogue Descriptions, Version 2.1, June 2001 IMDI, Metadata Elements for Session Descriptions, Version 3.0.4, Sept. 2003. Harris Papageorgiou, L. Cranias, Stelios Piperidis1994. Automatic alignment in parallel corpora. In Proceedings of ACL 1994. Harris Papageorgiou, Prokopis Prokopidis, Voula Giouli, Iasonas Demiros, Alexis Konstantinidis,
41 The Development of the Index Thomisticus Treebank Valency Lexicon
Barbara McGillivray Marco Passarotti University of Pisa Catholic University of the Sacred Heart Italy Milan, Italy [email protected] [email protected]
necessary to match lexicons with data from (an- Abstract notated) corpora, and viceversa. This requires the scholars to exploit the vast amount of textual We present a valency lexicon for Latin verbs data from classical languages already available in extracted from the Index Thomisticus Tree- digital format,1 and particularly those annotated bank, a syntactically annotated corpus of Me- at the highest levels. The evidence provided by dieval Latin texts by Thomas Aquinas. the texts themselves can be fully represented in In our corpus-based approach, the lexicon re- lexicons induced from these data. Subsequently, flects the empirical evidence of the source data. Verbal arguments are induced directly these lexicons can be used to support the textual from annotated data. annotation itself in a virtuous circle. The lexicon contains 432 Latin verbs with 270 This paper reports on the creation of a valency valency frames. The lexicon is useful for NLP lexicon induced from the Index Thomisticus applications and is able to support annotation. Treebank, a syntactically annotated corpus of Medieval Latin texts by Thomas Aquinas. The paper is organised as follows: section 2 describes the available Latin treebanks, their annotation 1 Introduction guidelines and gives some specific information Over the last decades, annotated corpora and on the Index Thomisticus treebank; section 3 computational lexicons have gained an increas- deals with the notion of valency, while section 4 ing role among language resources in computa- describes the state of the art on valency lexicons; tional linguistics: on the one hand, they are used section 5 illustrates the procedures of acquisition to train Natural Language Processing (NLP) and representation of our valency lexicon; fi- tools such as parsers and PoS taggers; on the nally, section 6 draws some conclusions and de- other hand, they are developed through auto- scribes future work. matic procedures of linguistic annotation and 2 Latin Treebanks lexical acquisition. The relation between annotated corpora and Latin is a richly inflected language, showing: computational lexicons is circular: as a matter of - discontinuous constituents (‘non- fact, if linguistic annotation of textual data is projectivity’): this means that phrasal con- supported and improved by the use of lexicons, stituents may not be continuous, but broken these latter can be induced from annotated data up by words of other constituents. An exam- in a corpus-based fashion. ple is the following sentence by Ovid In the field of cultural heritage and in particu- (Metamorphoses, I.1-2): “In nova fert ani- lar that of classical languages studies, much ef- mus mutatas dicere formas corpora” (“My fort has been devoted throughout the years to the mind leads me to tell of forms changed into digitization of texts, but only recently have some new bodies”). In this sentence, both the projects begun to annotate them above the mor- nominal phrases “nova corpora” and “muta- phological level. tas formas” are discontinuous; Concerning lexicology and lexicography of - moderately free word-order: for instance, the classical languages, a long tradition has produced order of the words in a sentence like “au- and established many dictionaries, thesauri and lexicons, providing examples from real texts. 1 See, for instance, the Perseus Digital Library (Crane et al., Nevertheless, nowadays it is possible and indeed 2001), or data repositories such as LASLA (Denooz, 1996).
Proceedings of the EACL 2009 Workshop on Language Technology and Resources for Cultural Heritage, Social Sciences, Humanities, and Education –LaTeCH – SHELT&R 2009, pages 43–50, Athens, Greece, 30 March 2009. c 2009 Association for Computational Linguistics
42 daces fortuna iuvat” (“fortune favours the available, which is ongoing at the University of bold”) could be changed into “fortuna au- Oslo in the context of the PROIEL project daces iuvat”, or “fortuna iuvat audaces”, (Pragmatic Resources in Old Indo-European without affecting the meaning of the sen- Languages): the aim of PROIEL is the syntactic tence. annotation of the oldest extant versions of the These features of Latin influenced the choice New Testament in Indo-European languages, of Dependency Grammars (DG)2 as the most including Greek, Latin, Gothic, Armenian and suitable grammar framework for building Latin Church Slavonic (Haug and Jøhndal, 2008). annotated corpora like treebanks. While since the 1970s the first treebanks were 2.1 Annotation Guidelines annotated via Phrase Structure Grammar (PSG)- Since LDT and IT-TB were the first projects of based schemata (as in IBM, Lancaster and, later their kind for Latin, no prior established guide- on, Penn treebanks), in the past decade many lines were available to rely on for syntactic anno- projects of dependency treebanks development tation. have started, such as the ALPINO treebank for Therefore, the so-called ‘analytical layer’ of Dutch (Van der Beek et al., 2002), the Turin annotation of the Prague Dependency Treebank University Treebank for Italian (Lesmo et al., (PDT) for Czech (Hajič et al., 1999) was chosen 2002), or the Danish Dependency Treebank and adapted to specific or idiosyncratic construc- (Kromann, 2003). On the one hand, this is due to tions of Latin. These constructions (such as the the fact that the first treebanks were mainly Eng- ablative absolute or the passive periphrastic) lish language corpora. PSG were a suitable could be syntactically annotated in several dif- framework for a poorly inflected language like ferent ways and are common to Latin of all eras. English, showing a fixed word-order and few Rather than have each treebank project decide discontinuous constituents. Later on, the syntac- upon and record each decision for annotating tic annotation of moderately free word-order lan- them, LDT and IT-TB decided to pool their re- guages required the adoption of the DG frame- sources and create a single annotation manual work, which is more appropriate than PSG for that would govern both treebanks (Bamman et such a task. On the other hand, Carroll et al. al., 2007a; Bamman et al., 2007b; Bamman et al., (1998) showed that inter-annotator agreement 2008). was significantly better for dependency tree- As we are dealing with Latin dialects sepa- banks, indicating that phrase structure annotation rated by 13 centuries, sharing a single annotation was requiring too many irrelevant decisions (see manual is very useful for comparison purposes, also Lin, 1995). such as checking annotation consistency or dia- Although much Latin data is nowadays avail- chronically studying specific syntactic construc- able in digital format, the first two projects for tions. In addition, the task of data annotation the development of Latin treebanks have only through these common guidelines allows annota- recently started: namely the Latin Dependency tors to base their decisions on a variety of exam- Treebank (LDT) at the Tufts University in Bos- ples from a wider range of texts and combine the ton (within the Perseus Digital Library) based on two datasets in order to train probabilistic de- texts of the Classical era (Bamman, 2006), and pendency parsers. the Index Thomisticus Treebank (IT-TB) at the Although the PROIEL annotation guidelines Catholic University of the Sacred Heart in Milan, are grounded on the same grammar framework based on the Opera omnia of Thomas Aquinas as the LDT and IT-TB, they differ in a number of (Passarotti, 2007). details, some of which are described in Passarotti Taking into account the above mentioned fea- (forthcoming). tures of Latin, both the treebanks independently chose the DG framework as the most suitable 2.2 The Index Thomisticus Treebank one for data annotation. The same approach was The Index Thomisticus (IT) by Roberto Busa SJ later on followed by a third Latin treebank now (1974-1980) was begun in 1949 and is consid- ered a groundbreaking project in computational 2 With Tesnière (1959) as a common background, there are linguistics. It is a database containing the Opera many different current DG flavours. See for instance the omnia of Thomas Aquinas (118 texts) as well as following: Dependency Unification Grammar (Hellwig, 61 texts by other authors related to Thomas, for a 1986), Functional Generative Description (Sgall, Hajičová and Panevová, 1986), Meaning Text Theory (Mel’čuk, total of around 11 million tokens. The corpus is 1988), Word Grammar (Hudson, 1990). morphologically tagged and lemmatised.
43 Early in the 1970’s Busa started to plan a pro- Arguments can be either obligatory or op- ject aimed at both the morphosyntactic disam- tional, depending on which sense of the verb is biguation of the IT lemmatisation and the syntac- involved. For example, the seem sense of the tic annotation of its sentences. Today, these tasks verb appear requires two obligatory arguments are performed by the IT-TB project, which is in active clauses, as in the following sentence: part of the wider ‘Lessico Tomistico Bicul- “That lawyer appears to love his work”. Here the turale’, a project whose target is the development second argument (“to love his work”) cannot be of a lexicon from the IT texts.3 left out without changing the meaning of the Presently, the size of the IT-TB is 46,456 to- verb. On the other hand, optional arguments are kens, for a total of 2,103 parsed sentences ex- recorded into the verbal argument structure itself, cerpted from the Scriptum super Sententiis Mag- althought they may not appear at the clausal istri Petri Lombardi. level. For instance, in the following sentence the object required by the verb eat is missing, but the 3 Valency sentence is still acceptable: “He eats (some- thing)”. As outlined above, the notion of valency is gen- Optionality can also act at the communicative erally defined as the number of complements level as well as at the structural one. For in- required by a word: these obligatory comple- stance, adjuncts can be necessary for communi- ments are usually named ‘arguments’, while the cative intelligibility in particular contexts, as in non-obligatory ones are referred to as ‘adjuncts’. the following sentence: “I met James at the Mar- Although valency can refer to different parts of quee club”, where the locative adverbial (“at the speech (usually verbs, nouns and adjectives), Marquee club”) is required to answer a question scholars have mainly focused their attention on like “Where did you meet James?”. On the other verbs, so that the notion of valency often coin- hand, structural optionality depends on the fea- cides with verbal valency. tures of the language and applies at the clausal Valency is widely used in DG formalisms, but level. For instance, as a poorly inflected lan- it also figures in PSG-based formalisms like guage, English requires the subject of a predicate HPSG and LFG. to be expressed in declarative and interrogative While Karl Bühler can be considered as the main clauses, so that a sentence like the follow- pioneer of the modern theory of valency,4 Lucien ing is ungrammatical if the subject is missing: Tesnière is widely recognised as its real founder. “[I] slept all morning”. Tesnière views valency as a quantitative quality Given the so-called “syntax-semantics inter- of verbs, since only verbs constrain both the face” (Levin, 1993), arguments are generally quantity and the quality (i.e. nouns and adverbs) associated with a predicate sense rather than a of their obligatory arguments; through a meta- predicate form, and are structured in sequences phor borrowed from drama, Tesnière classifies called ‘subcategorization frames’ (SCFs) or dependents into actants (arguments) and circon- ‘complementation patterns’. For example, there stants (adjuncts): “Le noeud verbal […] exprime is a semantic difference between the bill sense tout un petit drame. Comme un drame en effet, il and the attack sense of the verb charge in Eng- comporte obligatoirement un procès, et le plus lish, as in the following sentences: souvent des acteurs et des circonstances. Trans- - (a) “The hotel charges 80 euros for a night”. posés du plan de la réalité dramatique sur celui - (b) “The army charged the enemy”. de la syntaxe structurale, le procès, les acteurs et In these sentences, the two predicate senses les circonstances deviennent respectivement le show two different SCFs: verbe, les actants et les circonstants” (Tesnière, - (a) [Subj_NP, Pred, Obj_NP, Obj_PP-for] 1959: 102).5 - (b) [Pred, Obj_NP] Arguments are also selected by verbs accord- 3 http://itreebank.marginalia.it. ing to lexical-semantic properties, called ‘selec- 4 In the Sprachtheorie, he writes that “die Wörter einer bes- tional preferences’ (SPs) or ‘selectional restric- timmten Wortklasse eine oder mehrere Leerstellen um sich tions’. For example, a sentence like “*The train eröffnen, die durch Wörter bestimmter anderer Wortklassen ausgefüllt werden müssen” (Bühler, 1934: 173) (“words of a flew to Rome” is ungrammatical, since it violates certain word-class open up around themselves one or sev- eral empty spaces that have to be filled by words of certain and circumstances. Transposed from the dramatic reality to other word-classes”; our translation). structural syntax, the process, the actors and the circum- 5 “The verbal node expresses a whole little drama. As a stances respectively become the verb, the actants and the drama, it implies a process and, most of the times, actors circumstants” (our translation).
44 the SP of the verb fly on its subject and can only the original corpus. Finally, they are less costly be accepted in a metaphorical context. than hand-crafted lexical resources in terms of time, money and human resources. 4 Valency Lexicons While several subcategorization lexicons have been compiled for modern languages, much Over the past years, several valency lexicons work in this field still remains to be done on have been built within different theoretical classical languages such as Greek and Latin. Re- frameworks: these lexicons have an important garding Latin, Happ reports a list of Latin verbs role in the NLP community thanks to their wide along with their valencies (Happ, 1976: 480- applications in NLP components, such as pars- 565). Bamman and Crane (2008) describe a “dy- ing, word sense disambiguation, automatic verb namic lexicon” automatically extracted from the classification and selectional preference acquisi- Perseus Digital Library, using the LDT as a tion. training set. This lexicon displays qualitative and As shown in Urešová (2004), a valency lexi- quantitative information on subcategorization con can also help the task of linguistic annotation patterns and selectional preferences of each word (as in treebank development), providing annota- as it is used in every Latin author of the corpus. tors with essential information about the number Relying on morphological tagging and statistical and types of arguments realized at the syntactic syntactic parsing of such a large corpus, their level for a specific verb, along with semantic approach finds the most common arguments and information on the verb’s lexical preferences. the most common lexical fillers of these argu- In the phase of lexicon creation, both intui- ments, thus reducing the noise caused by the tion-based and corpus-based approaches can be automatic pre-processing of the data. pursued, according to the role played by human intuition and empirical evidence extracted from 5 The Index Thomisticus Treebank annotated corpora such as treebanks. Valency Lexicon For instance, lexicons like PropBank (Kings- bury and Palmer, 2002), FrameNet (Ruppenhofer We propose a corpus-based valency lexicon for et al., 2006) and PDT-Vallex (Hajič et al., 2003) Latin verbs automatically induced from IT-TB have been created in an intuition-based fashion data. The automatic procedure allows both the and then checked and improved with examples extension of this work to the LDT (thanks to the from corpora. common annotation guidelines) and the updating On the other side, research in lexical acqui- of the lexicon as the treebank size increases. sition has recently made available a number of First, we automatically extract the argu- valency lexicons automatically acquired from ments of all the occurrences of verbal lemmata in annotated corpora, such as VALEX (Korhonen, the treebank, along with their morphological fea- et al., 2006) and LexShem (Messiant et al., tures and lexical fillers. 2008). Unlike the fully intuition-based ones, In the IT-TB, verbal arguments are anno- these lexicons aim at systematically reflecting tated using the following tags: Sb (Subject), Obj the evidence provided by data, with very little (Object), OComp (Object Complement) and human intervention. The role of intuition is Pnom (Predicate Nominal); adjuncts are anno- therefore left to the annotation phase (where the tated with the tag Adv (Adverbial). The differ- annotator interprets the corpus data), and not ex- ence between Obj and Adv corresponds to the tended to the development of the lexicon itself. that between direct or indirect arguments (except Corpus-based lexicons show several advan- subjects) and adjuncts. A special kind of Obj is tages if compared with traditional human- the determining complement of the object, which developed dictionaries. Firstly, they systemati- is tagged with OComp, such as senatorem in the cally reflect the evidence of the corpus they were phrase “aliquem senatorem facere” (“to nominate extracted from, while acquiring information spe- someone senator”). Conversely, the determining cific to the domain of the corpus. Secondly, complement of the subject is tagged as Pnom, as unlike manually built lexicons, they are not in “aliquis senator fit” (“someone becomes sena- prone to human errors that are difficult to detect, tor”).6 such as omissions and inconsistencies. In addi- tion, such lexicons usually display statistical in- 6 As in the PDT, all of the syntactic tags can be appended formation in their entries, such as the actual fre- with a suffix in the event that the given node is member of a quency of subcategorization frames as attested in coordinated construction (_Co), an apposition (_Ap) or a parenthetical statement (_Pa).
45 In order to retrieve the arguments realised for each verbal occurrence in the treebank, spe- cific database queries have been created to search for the nodes depending on a verbal head through the functional tags listed above. The head-dependent relation can be either direct or indirect, since intermediate nodes may intervene. These nodes are prepositions (tag AuxP), conjunctions (tag AuxC) and coordinat- ing or apposing elements (respectively, tags Co- ord and Apos). For example, see the following sentences: - [1] “primo determinat formam baptismi;”7 (“at first it determines the form of the bap- tism;”) - [2] “ly aliquid autem, et ly unum non deter- minant aliquam formam vel naturam;”8 (“the ‘something’ and the ‘one’ do not determine any form or nature”) Figure 1 reports the tree of sentence [1], where the Obj relation between the verbal head Figure 2 determinat and the dependent formam is direct. Tree of sentence [2]
In the case of indirect relation, the interme- diate nodes need to be detected and extracted, in order to be inserted into the lexicon as subcate- gorization structures containing the syntactic roles of the verbal arguments. To represent these structures, we distinguished two major types of them: subcategorization frames (SCFs) and sub- categorization classes (SCCs). An SCF contains the sequence of functional labels of verbal arguments as they appear in the sentence order, whereas an SCC reports the sub- categorization elements disregarding their linear order in the sentence. SCFs and SCCs play a dif- ferent role in our lexicon. On the one hand, SCFs are very detailed patterns useful for diachronic Figure 1. and/or comparative studies on linear order. On Tree of sentence [1] the other hand, SCCs are more general and make the data in the lexicon comparable with the sub- Figure 2 shows the tree of sentence [2]. In categorization structures as usually defined in the this tree, two coordinated subjects (aliquid and literature and in other valency lexicons. For each unum) and two coordinated objects (formam and of these structures we then created the following naturam) depend on the common verbal head sub-types, ranging from the most specific to the determinant through two different Coord nodes least specific one. (et and vel)9. SCF1: subcategorization frame marking the full path between the verbal head (referred to as ‘V’) and each of its argument nodes in the tree. 7 Thomas, Super Sententiis Petri Lombardi, IV, Distinctio 3, SCF1 also assigns the same index to those argu- Quaestio 1, Prologus, 41-6, 42-2. The edition of the text ment nodes linked by coordinating or apposing recorded in the IT is Thomas (1856-1858). elements. For instance, the SCF of the verbal 8 Thomas, Super Sententiis Petri Lombardi, III, Distinctio 6, 1 Quaestio 2, Articulus 1, Responsio ad Argumentum 7, 4-5, 6-1. 9 Following PDT-style, the distributed determination ali- and naturam, depends on the coordinating node vel. For quam, which modifies both the coordinated objects formam more details, see Hajic et al. (1999), 236-238.
46 head determino10 in sentence [1] is ‘V + Obj’, depending on the features of the texts. Therefore, while in sentence [2] is ‘(Coord)Sb_Co(1) + (Co- the lexicon reflects the empirical evidence given ord)Sb_Co(1) + V + (Coord)Obj_Co(2) + (Co- by the data it was extracted from, encouraging ord)Obj_Co(2)’. In the latter, the intermediate linguistic studies on the particular language do- nodes Coord are in square brackets and indices 1 main of our corpus. and 2 link the coordinated nodes. These indices In addition to the syntactic information re- have been adopted in order to disambiguate sub- ported in the different types of SCFs and SCCs, categorization structures where more Obj_Co it is possible at each stage to include both the tags can refer to different verbal arguments. For morphological features and the lexical fillers of instance, in a sentence like “I give X and Y to W verbal arguments, helping define verbal selec- and Z”, both the tranferred objects (X and Y) and tional preferences. the receivers (W and Z) are annotated with The lexicon may also be useful for improv- Obj_Co. Using indices, the subcategorization ing the performance of statistical parsers, enrich- structure of the verb give in this sentence appears ing the information acquired by parsers on verbal as follows: ‘Sb + V + (Coord)Obj_Co(1) + (Co- entries. On the other hand, moving from parser ord)Obj_Co(1) + (Coord)Obj_Co(2) + (Co- performance to lexicon development, the lexicon ord)Obj_Co(2)’. The indices cannot be applied a can be induced from automatically parsed texts priori to subsequent arguments, since Latin, al- when an accurate parsing system is available. lowing discontinuous constituents, can show The syntactic and lexical data recorded in the cases where coindexed nodes are separated by lexicon are also important in further semantic other lexical items in the linear order. NLP applications, such as word sense disam- SCC1: the subcategorization class associated biguation, anaphora and ellipsis resolution, and with SCF1. The SCC1 of the verb determino in selectional preference acquisition. Following a [1] is ‘{Obj}’, while in [2] is ‘{(Coord)Sb_Co(1), widespread approach in valency lexicons, a close (Coord)Sb_Co(1), (Coord)Obj_Co(2), (Co- connection between valency frames and word ord)Obj_Co(2)}’. senses will be followed in the description of lexi- SCF2: a subcategorization frame containing con entries: this means that each headword entry only the labels and the indices of the arguments, of our lexicon will consist of one or more SCFs but not the full path. So, the SCF2 of determino and SCCs, one for each sense of the word. in [1] is ‘V + Obj’, while in [2] is ‘Sb_Co(1) + We plan to make the lexicon available on- Sb_Co(1) + V + Obj_Co(2) + Obj_Co(2)’. line through a graphical interface usable also SCC2: the subcategorization class associated during the annotation procedures, as has been with SCF2. For determino, this is ‘{Obj}’ in [1] already done for the PDT via the tree editor and ‘{Sb_Co(1), Sb_Co(1), Obj_Co(2), Obj_Co(2)}’ TrEd.11 In this way, the consistency of the anno- in [2]. tation process can be tested and enforced thanks SCC3: a subcategorization frame containing to the information stored in the lexicon. only the argument labels. The SCC3 of determino In order to test the accuracy of our system, it is ‘{Obj}’ in [1] and ‘{Sb, Obj}’ in [2], showing will be also necessary to evaluate the quality of that in this sentence determino is used as a biar- our valency lexicon against the Perseus “dy- gumental verb, regardless of the number of lexi- namic lexicon”, Happ’s list and other existing cal fillers realised for each of its arguments at the resources for Latin, such as traditional dictionar- surface level. ies and thesauri. A comparison with the lexicon by Perseus is also very interesting in a contras- 6 Conclusion and future work tive diachronic perspective, as it may show im- portant linguistic differences between Classical Presently, the size of the IT-TB valency lexicon and Medieval Latin. is 432 entries (i.e. verbal lemmata, corresponding to 5966 wordforms), with 270 different SCF s. In 1 Acknowledgments the near future, the lexicon will be enriched with valency information for nouns and adjectives. We would like to thank Paolo Ruffolo for his The corpus-based approach we followed in- help in designing the database architecture. duces verbal arguments directly from annotated data, where the arguments may be present or not, References
10 Determino is the lemma of both the wordforms determi- 11 TrEd is freely available at nat (sentence [1]) and determinant (sentence [2]). http://ufal.mff.cuni.cz/~pajas/tred/.
47 David Bamman. 2006. The Design and Use of Latin (LASLA). « Le Médiéviste et l'ordinateur », 33, 14- Dependency Treebank. In Jan Hajič and Joakim 20. Nivre (eds.), TLT 2006. Proceedings of the Fifth Jan Hajič, Jarmila Panevová, Eva Buráňová, Zdeňka Workshop on Treebanks and Linguistic Theories. Urešová and Alla Bémová. 1999. Annotations at December 1-2, 2006, Prague, Czech Republic, In- Analytical Level. Instructions for annotators, Insti- stitute of Formal and Applied Linguistics, Prague, tute of Formal and Applied Linguistics, Prague, Czech Republic, 67-78. Czech Republic. Available at: David Bamman and Gregory Crane. 2008. Building a http://ufal.mff.cuni.cz/pdt2.0/doc/manuals/en/a- Dynamic Lexicon from a Digital Library. In Pro- layer/pdf/a-man-en.pdf. ceedings of the 8th ACM/IEEE-CS Joint Confer- Jan Hajič, Jarmila Panevová, Zdeňka Urešová, Alla ence on Digital Libraries (JCDL 2008), Pittsburgh. Bémová, Veronika Kolárová-Reznícková and Petr David Bamman, Marco Passarotti, Gregory Crane and Pajas. 2003. PDT-VALLEX: Creating a Large Savina Raynaud. 2007a. Guidelines for the Syntac- Coverage Valency Lexicon for Treebank Annota- tic Annotation of Latin Treebanks, «Tufts Univer- tion. In Joakim Nivre and Erhard Hinrichs (eds.), sity Digital Library». Available at: TLT 2003 – Proceedings of the Second Workshop http://dl.tufts.edu/view_pdf.jsp?urn=tufts:facpubs:d on Treebanks and Linguistic Theories, volume 9 of bamma01-2007.00002. Mathematical Modelling in Physics, Engineering David Bamman, Marco Passarotti, Gregory Crane and and Cognitive Sciences, Växjö University Press, Savina Raynaud. 2007b. A Collaborative Model of Växjö, Sweden, 57-68. Treebank Development. In Koenraad De Smedt, Heinz Happ. 1976. Grundfragen einer Dependenz- Jan Hajič and Sandra Kübler (eds.), Proceedings of Grammatik des Lateinischen, Vandenhoeck & Ru- the Sixth International Workshop on Treebanks precht, Goettingen. and Linguistic Theories. December 7-8, 2007, Ber- Dag Haug and Marius Jøhndal. 2008. Creating a Par- gen, Norway, Northern European Association for allel Treebank of the Old Indo-European Bible Language Technology (NEALT) Proceedings Se- Translations. In Proceedings of the Language ries, Vol. 1, 1-6. Technology for Cultural Heritage Data Workshop David Bamman, Marco Passarotti, Roberto Busa and (LaTeCH 2008), Marrakech, Morocco, 1st June Gregory Crane. 2008. The annotation guidelines of 2008, 27-34. the Latin Dependency Treebank and Index Thomis- Peter Hellwig. 1986. Dependency Unification Gram- ticus Treebank. The treatment of some specific mar, In Proceedings of the 11th International Con- syntactic constructions in Latin. In Proceedings of ference on Computational Linguistics, Universität the Sixth International Conference on Language Bonn, Bonn, 195-198. Resources and Evaluation (LREC 2008). May 28- Richard Hudson. 1990. English Word Grammar, 30, 2008, Marrakech, Morocco, European Lan- Blackwell Publishers Ltd, Oxford, UK. guage Resources Association (ELRA), 2008. Paul Kingsbury and Martha Palmer. 2002. From Karl Bühler. 1934. Sprachtheorie: die Darstellungs- Treebank to Propbank. In Proceedings of the Third funktion der Sprache, Jena: Gustav Fischer, Stutt- International Conference on Language Resources gart. and Evaluation (LREC 2002), Las Palmas – Gran Roberto Busa. 1974–1980. Index Thomisticus: sancti Canaria, Spain. Thomae Aquinatis operum omnium indices et con- Anna Korhonen, Yuval Krymolowski and Ted cordantiae, in quibus verborum omnium et singu- Briscoe. 2006. A Large Subcategorization Lexicon lorum formae et lemmata cum suis frequentiis et for Natural Language Processing Applications. In contextibus variis modis referuntur quaeque / con- Proceedings of the Fifth International Conference sociata plurium opera atque electronico IBM on Language Resources and Evaluation (LREC automato usus digessit Robertus Busa SJ, From- 2006), Genoa, Italy. mann-Holzboog, Stuttgart-Bad Cannstatt. Matthias T. Kromann. 2003. The Danish Dependency Gregory R. Crane, Robert F. Chavez, Anne Mahoney, Treebank and the underlying linguistic theory. In Thomas L. Milbank, Jeff A. Rydberg-Cox, David Joakim Nivre and Erhard Hinrichs (eds.), TLT A. Smith and Clifford E. Wulfman. 2001. Drudg- 2003 – Proceedings of the Second Workshop on ery and deep thought: Designing a digital library Treebanks and Linguistic Theories, volume 9 of for the humanities. In Communications of the Mathematical Modelling in Physics, Engineering ACM, 44(5), 34-40. and Cognitive Sciences, Växjö University Press, John Carroll, Ted Briscoe and Antonio Sanfilippo. Växjö, Sweden. 1998. Parser Evaluation: a Survey and a New Pro- Leonardo Lesmo, Vincenzo Lombardo and Cristina posal. In Proceedings of the First International Bosco. 2002. Treebank Development: the TUT Conference on Language Resources and Evalua- Approach. In Rajeev Sangal and Sushma M. tion (LREC 1998). May 28-30, 1998, Granada, Bendre (eds.), Recent Advances in Natural Spain, 447-454. Language Processing. Proceedings of Joseph Denooz. 1996. La banque de données du labo- International Conference on Natural Language ratoire d'analyse statistique des langues anciennes
48 Processing (ICON 2002), Vikas Publ. House, New Dependency Treebank. In Mariet Theune, Anton Delhi, 61-70. Nijholt and Hendri Hondorp (eds.), Proceedings of Beth Levin. 1993. English verb classes and the Twelfth Meeting of Computational Linguistics alternations: a preliminary investigation, in the Netherlands (CLIN 2001), Rodopi, University of Chicago Press, Chicago. Amsterdam, 8-22. Dekang Lin. 1995. A dependency-based method for evaluating broadcoverage parsers. In Proceedings of the IJCAI-95, Montreal, Canada, 1420-1425. Igor Mel’čuk. 1988. Dependency Syntax: Theory and Practice, State University Press of New York, Al- bany/NY. Cedric Messiant, Anna Korhonen and Thierry Poibeau. 2008. LexSchem: A Large Subcategorization Lexicon for French Verbs. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC 2008). May 28-30, 2008, Marrakech, Morocco, European Language Resources Association (ELRA), 2008. Jarmila Panevová. 1974-1975. On Verbal Frames in Functional Generative Description. Part I, «Prague Bulletin of Mathematical Linguistics», 22, 3-40; Part II, «Prague Bulletin of Mathematical Linguis- tics», 23, 17-52. Marco Passarotti. 2007. Verso il Lessico Tomistico Biculturale. La treebank dell’Index Thomisticus. In Raffaella Petrilli and Diego Femia (eds.), Il filo del discorso. Intrecci testuali, articolazioni linguisti- che, composizioni logiche. Atti del XIII Congresso Nazionale della Società di Filosofia del Linguag- gio, Viterbo, 14-16 Settembre 2006, Aracne Editri- ce, Pubblicazioni della Società di Filosofia del Linguaggio, 04, Roma, 187-205. Marco Passarotti. Forthcoming. Theory and Practice of Corpus Annotation in the Index Thomisticus Treebank. In Proceedings of the Conference ‘Trends in Computational and Formal Philology - Venice Padua, May 22-24, 2008’. Josef Ruppenhofer, Michael Ellsworth, Miriam R.L. Petruck, Christopher R. Johnson and Jan Scheffczyk. 2006. FrameNet II. Extendend Theory and Practice. E-book available at http://framenet.icsi.berkeley.edu/index.php?option =com_wrapper&Itemid=126. Petr Sgall, Eva Hajičová and Jarmila Panevová. 1986. The Meaning of the Sentence in its Semantic and Pragmatic Aspects, D. Reidel, Dordrecht, NL. Lucien Tesnière. 1959. Éléments de syntaxe struc- turale, Editions Klincksieck, Paris, France. Thomas Aquinas. 1856-1858. Sancti Thomae Aquinatis, doctoris angelici, Ordinis praedicatorum Commentum in quatuor libros Sententiarum magistri Petri Lombardi, adjectis brevibus adnotationibus, Fiaccadori, Parma. Zdenka Urešová. 2004. The Verbal Valency in the Prague Dependency Treebank from the Annotator's Point of View. Jazykovedný ústav Ľ. Štúra, SAV, Bratislava, Slovakia. Leonoor Van der Beek, Gosse Bouma, Rob Malouf and Gertjan van Noord. 2002. The Alpino
49 Applying NLP Technologies to the Collection and Enrichment of Language Data on the Web to Aid Linguistic Research
Fei Xia William D. Lewis University of Washington Microsoft Research Seattle, WA 98195, USA Redmond, WA 98052, USA [email protected] [email protected]
Abstract guistic typology a well-known implicational uni- versal states that if the demonstrative follows the The field of linguistics has always been noun, then the relative clause also follows the noun reliant on language data, since that is its (Croft, 2003). Although this particular universal principal object of study. One of the major is well-researched and widely accepted, identify- obstacles that linguists encounter is find- ing this tendency anew—as an example of what ing data relevant to their research. In this one must do when researching a new universal— paper, we propose a three-stage approach would require a significant amount of work: in or- to help linguists find relevant data. First, der to be relatively sure that the universal holds, language data embedded in existing lin- the linguist would need to identify a substantial guistic scholarly discourse is collected and number of true positives (those that support the stored in a database. Second, the lan- universal), and ensure that there are not a sufficient guage data is automatically analyzed and number of negatives that would act as a refutation. enriched, and language profiles are created The only way a linguist could be completely sure from the enriched data. Third, a search would be to conduct a thorough literature review facility is provided to allow linguists to on the subject or go through data from a repre- search the original data, the enriched data, sentative and significant sample of data from the and the language profiles in a variety of approximately seven thousand languages that are ways. This work demonstrates the benefits or have been spoken (and for which data exists). of using natural language processing tech- There have been much effort by the linguistic nology to create resources and tools for community to address the issue. For instance, linguistic research, allowing linguists to LinguistList compiles a long list of linguistic re- have easy access not only to language data sources1, making it easier to find electronically embedded in existing linguistic papers, but available resources. Likewise, the Open Language also to automatically generated language Archives Community (OLAC) acts as an online profiles for hundreds of languages. virtual library of language resources, and provides a search tool that searches several dozen online 1 Introduction linguistic resources. Further, the World Atlas of Linguistics is the scientific study of language, and Language Structures (WALS), which was recently the object of study is language, in particular lan- made available online, is a large database of struc- guage data. One of the major obstacles that lin- tural (phonological, grammatical, lexical) proper- guists encounter is finding data relevant to their ties of languages gathered from descriptive mate- 2 research. While the strategy of word of mouth rials (Haspelmath et al., 2005). or consulting resources in a library may work for 1http://www.linguistlist.org/langres/index.html small amounts of data, it does not scale well. Val- 2There are other online resources for searching for lin- idating or reputing key components of a linguistic guistic data, in particular typological data. Two of note in- clude Autotyp (Bickel and Nichols, 2002) and the Typologi- theory realistically requires analyzing data across cal Database System (Dimitriadis et al., forthcoming), among a large sample of languages. For instance, in lin- others. The former has limited online availability (much of
Proceedings of the EACL 2009 Workshop on Language Technology and Resources for Cultural Heritage, Social Sciences, Humanities, and Education –LaTeCH – SHELT&R 2009, pages 51–59, Athens, Greece, 30 March 2009. c 2009 Association for Computational Linguistics
50 We propose a three-stage approach to help lin- data points; each data point is a (language, fea- guists in locating relevant data. First, language ture, feature value) tuple that specifies the value of data embedded in existing linguistic scholarly dis- the feature in a particular language. For instance, course is collected and stored in a database. Sec- (English, canonical word order, SVO) means that ond, the language data is automatically analyzed the canonical word order of English is SVO. and enriched and language profiles are created from the enriched data. Third, a search facility is 2.3 OLAC provided to allow linguists to search the original The Open Languages Archive Community data, the enriched data, and the language profiles. (OLAC), described in (Bird and Simons, 2003), This is an on-going research project. While the is part of the Open Archives Initiative, which first stage is completed, the second and third stages promotes interoperability standards for linguistic are partially completed and still undergoing devel- data.3 The focus of OLAC has been to facilitate opment. In this paper, we will describe each stage the discovery of linguistic resources through a and report results. common metadata structure for describing digital data and by providing a common means for locat- 2 Related work ing these data through search interfaces housed at In this section, we briefly discuss a few projects Linguist List and the Linguistics Data Consortium that are most relevant to our work. (LDC). Our work shares with OLAC the need for resource discovery, and moves beyond OLAC 2.1 Ethnologue by enriching and manipulating the content of The purpose of the Ethnologue is to provide a linguistic resources. comprehensive listing of the known living lan- guages of the world. The most recent version, ver- 3 Building ODIN sion 15, covers more than six thousand languages. The first stage of the three-stage approach is to col- Information in the Ethnologue comes from numer- lect linguistic data and store it in a database. In lin- ous sources and is confirmed by consulting both guistics, the practice of presenting language data reliable published sources and a network of field in interlinear form has a long history, going back correspondents, and has been built to be consistent at least to the time of the structuralists. Interlinear with ISO standard 639-3; the information is com- Glossed Text, or IGT, is often used to present data piled under several specific categories (e.g., coun- and analysis on a language that the reader may tries where a language is spoken and their popula- not know much about, and is frequently included tions) and no effort is made to gather data beyond in scholarly linguistic documents. The canonical those categories (Gordon, 2005). form of an IGT consists of three lines: a lan- guage line for the language in question, a gloss 2.2 WALS line that contains a word-by-word or morpheme- The World Atlas of Language Structures (WALS) by-morpheme gloss, and a translation line, usually is a large database of structural (phonologi- in English. The grammatical markers such as 3sg cal, grammatical, lexical) properties of languages on the gloss line are called grams. Table 1 shows gathered from descriptive materials (such as refer- the beginning of a linguistic document (Baker and ence grammars) by a team of more than 40 lin- Stewart, 1996) which contains two IGTs: one in guists (Haspelmath et al., 2005). WALS con- lines 30-32, and the other in lines 34-36. The line sists of 141 maps with accompanying text on numbers are added for the sake of convenience. diverse features (such as vowel inventory size, ODIN, the Online Database of INterlinear text, noun-genitive order, passive constructions, and is a resource built from data harvested from schol- hand/arm polysemy). Each map corresponds to arly documents (Lewis, 2006). ODIN was built in a feature and the map shows the feature values three main steps: for between 120 and 1370 languages. Altogether there are 2,650 languages and more than 58,000 (1) Crawling: crawling the Web to retrieve docu- the data is not directly accessible through query, but requires ments that may contain IGTs submitting requests to the site owners), however, and the lat- ter is still under development. 3http://www.language-archives.org/
51 1: THE ADJ/VERB DISTINCTION: EDO EVIDENCE evant terms (drawn from the SIL linguistic glos- 2: sary), and by particular words or morphemes 3: Mark C. Baker and Osamuyimen Thompson Stewart found in IGT and their grammatical markup. 4: McGill University .... 27: The following shows a similar minimal pair from Edo, 3.2 IGT detection 28: a Kwa language spoken in Nigeria (Agheyisi 1990). The canonical form of IGT consists of three parts 29: and each part is on a single line. However, many 30: (2) a. Em` er` ´imos` e´. 31: Mary be.beautiful(V) IGT instances, 53.6% of instances in ODIN, do not 32: ‘Mary is beautiful.’ follow the canonical form for various reasons. For 33: instance, some IGTs are missing gloss or trans- 34: b. Em` er` ´i *(ye´) mos` e´. 35: Mary be.beautiful(A) lation lines as they can be recovered from con- 36: ‘Mary is beautiful (A).’ text (e.g., other neighboring examples or the text ... surrounding the instance); some IGTs have multi- ple translations or language lines (e.g., one part in Table 1: A linguistic document that contains IGT: the native script, and another in a latin translitera- words in boldface are language names tion); still others contain additional lines of anno- tation and analysis, such as phonological alterna- (2) IGT detection: extracting IGTs from the re- tions, underlying forms, etc. trieved documents We treat IGT detection as a sequence labeling problem. First, we train a learner and use it to label (3) Language ID: identifying the language code each line in a document with a tag in a pre-defined of the extracted IGTs. tagset. The tagset is an extension of the standard BIO tagging scheme and it has five tags: they are The identified IGTs are then extracted and BL (any blank line), O (outside IGT that is not a stored in a database (the ODIN database), which 4 BL), B (the first line in an IGT), E (the last line in can be easily searched with a GUI interface. In an IGT), and I (inside an IGT that is not a B, E, or this section, we briefly describe the procedure, and BL). After the lines in a document are tagged by more detail about the procedure can be found in the learner, we identify IGT instances by finding (Xia and Lewis, 2008) and (Xia et al., 2009). all the spans in the document that match the “B [I 3.1 Crawling | BL]* E” pattern; that is, the span starts with a B line, ends with an E line, and has zero or more I or In the first step, linguistic documents that may BL lines in between. contain instances of IGT are harvested from the To test the system, we manually annotated 51 Web using metacrawls. Metacrawling involves documents to mark the positions of the IGTs. We throwing queries against an existing search en- trained the system on 41 documents (with 1573 gine, such as Google and Live Search, and crawl- IGT instances) and tested it on 10 documents (with ing only the pages returned by those queries. We 447 instances). The F-score for exact match (i.e., found that the most successful queries were those two spans match iff they are identical) was 88.4%, that used strings contained within IGT itself (e.g. and for partial match (i.e., two spans match iff they grams such as 3sg). In addition, we found pre- overlap), was 95.4%. The detail of the system can cision increased when we included two or more be found in (Xia and Lewis, 2008). search terms per query, with the most successful queries being those which combined grams and 3.3 Language ID language names. The language ID task here is very different from a Other queries we have developed include: typical language ID task. For instance, the num- queries by language names and language codes ber of languages in ODIN is more than a thou- (drawn from the Ethnologue database (Gordon, sand and could potentially reach several thousand 2005), which contains about 40,000 language as more data is added. Furthermore, for most lan- names and their variants), by linguists names and guages in ODIN, our training data contains few the languages they work on (drawn from the Lin- to no instances of IGT. Because of these proper- guist Lists linguist database), by linguistically rel- ties, applying existing language ID algorithms to 4http://odin.linguistlist.org the task does not produce satisfactory results. For
52 instance, Cavnar and Trenkle’s N-gram-based al- Table 2: Language distribution of the IGTs in gorithm produced an accuracy of as high as 99.8% ODIN when tested on newsgroup articles in eight lan- Range of # of # of IGT % of IGT guages (Cavnar and Trenkle, 1994). However, IGT instances languages instances instances > 10000 3 36,691 19.39 when we ran the same algorithm on the IGT data, 1000-9999 37 97,158 51.34 the accuracy fell as low as 2% when the training 100-999 122 40,260 21.27 set was very small. 10-99 326 12,822 6.78 1-9 838 2,313 1.22 Since IGTs are part of a document, there are of- total 1326 189,244 100 ten various cues in the document (e.g., language names) that can help predict the language ID of these instances. We treat the language ID task as ODIN database, there are more than 130,000 ad- a coreference resolution (CoRef) problem: a men- ditional IGT-bearing documents that have been tion is an IGT or a language name appearing in a crawled but have not been fully processed. Once document, an entity is a language code, and find- these additional documents have been processed, ing the language code for an IGT is the same as the database is expected to expand significantly, linking a mention (e.g., an IGT) to an entity (i.e., growing to a million or more IGT instances. a language code).5 Once the language ID task is 4 Analyzing IGT data and creating framed as a CoRef problem, all the existing algo- language profiles rithms on CoRef can be applied to the task. We built two systems: one uses a maximum en- The second stage of the three-stage approach is tropy classifier with beam search, which for each to analyze and enrich IGT data automatically, to (IGT, language code) pair determines whether the extract information from the enriched data, and IGT should be linked to the language code; the to create so-called language profiles for the many other treats the task as a joint inference task and languages in the database. A language profile de- performs the inference by using Markov Logic scribes the main attributes of a language, such Network (Richardson and Domingos, 2006). Both as its word order, case markers, tense/aspect, systems outperform existing, general-purpose lan- number/person, major syntactic phenomena (e.g., guage identification algorithms significantly. The scrambling, clitic climbing), etc.7 detail of the algorithm and experimental results is An example profile is shown below. The pro- described in (Xia et al., 2009). file says that in Yoruba the canonical word or- der is SVO, determiners appear after nouns, and 3.4 The current ODIN database the language has Accusative case, Genitive case, We ran the IGT detection and language ID systems Nominative case, and so on. The concepts such as on three thousand IGT-bearing documents crawled AccusativeCase come from the GOLD Ontology from the Web and the extracted IGTs were stored (Farrar, 2003; Farrar and Langendoen, 2003).
53 (language data, gloss, translation, citation, 2007)). Our task here is slightly different from construction name, etc.) in an IGT. their tasks (e.g., extracting author/title/journal (2) Enriching IGT by processing the translation from citations) in that the fields in IGT could over- line and projecting the information onto the lap9 and corrupted lines need to be re-constructed language line. and re-stored in a particular way (e.g., pasting the (3) Identifying grams in the gloss line and map- second and third lines in Ex (1) back together). ping them to the concepts defined in GOLD Due to the differences, we did not create anno- Ontology or the like. tated data by segmenting IGT into separate fields and labeling each field. Instead, we used a refined (4) Answering questions in the language profile. tagset to indicate what information is available at In this section, we explain each step and report each line of IGT instances. The tagset includes some preliminary results. six main tags (L, G, T, etc.) and nine secondary tags (e.g., -CR for corruption and -SY for syntac- 4.1 Identifying fields in IGT tic information). Each line in each IGT instance is In addition to the language data (L), gloss (G), and labeled with one main tag and zero or more sec- translation (T) parts of IGT, an IGT often contains ondary tags. The labeled lines in Ex (1) are shown other information such as language name (-LN), in (3). citation (-AC), construction names (-CN), and so (3) M-LN-AC: Haitian CF (Lefebvre 1998:165) L-CR: ak on. An example is in (1), in which the first line L-SY-CR: Jani pale lii/j 8 G: John speak with he contains the language name and citation, the third T-DB: (a) 'John speaks with him' (b) 'John line includes coindexes i and i/j, and the last two C: speaks with himself' lines show two possible translations of the sen- The labeling of the data is done semi- tence. Here, the language line is displayed as two automatically. We have created a tool that takes lines due to errors made by the off-the-shelf con- the IGT spans produced by the current IGT detec- verter that converted the crawled pdf documents tor and labels IGT lines by using various cues in into text. an IGT instance, and designed a GUI that allows (1) Haitian CF (Lefebvre 1998:165) annotators to correct the system output easily. The ak Jani pale lii/j annotation speed is about 320 IGT instances per John speak with he (a) 'John speaks with him' (b) 'John hour on average. We are currently experimenting speaks with himself' with different ways of re-training the IGT detector The goal of this step is to separate out differ- with the new data. ent fields in an IGT, fix display errors caused by We have built a rule-based module that identi- the pdf-to-text converter, and store the results in a fies fields in IGT using the enriched tagset (i.e., uniform data structure such as the one in Ex (2) creating Ex (2) from Ex (3)), relying on the knowl- for the example in Ex (1). The task is not trivial edge about the conventions that linguists tend partially because the IGT detector marks only the to follow when specifying citations, construction span of an instance. For instance, the coindex i in names, coindexation and the like. The initial re- Jani and lii/j on the third line of Ex (1) could easily sult of field extraction looks promising. We are be mistaken as being part of the word. also studying whether existing unsupervised sta- (2) Language: Haitian CF tistical systems for information extraction (e.g., Citation: (Lefebvre 1998:165) (Poon and Domingos, 2007)) could be extended L: Jan pale ak li Coindx: (Jan, i), (li, i/j) to handle this task while taking advantage of the G: John speak with he T1: 'John speaks with him' enriched tagset for IGTs. We plan to complete the T2: 'John speaks with himself' study and report the results in the near future. There has been much work on extracting 4.2 Enriching IGT database records from text or semi-structured sources, and the common approach is breaking Since the language line in IGT data typically does the text into multiple segments and labeling each not come with annotations (e.g., POS tags, phrase segment with a field name (e.g., (Wellner et al., 9For instance, in some IGTs, a syntactic structure is added 2004; Grenager et al., 2005; Poon and Domingos, on top of the language line; for instance, the language line in Ex (1) could become something like [IP Jani [VP pale [PP 8CF here stands for French-lexified creole. ak lii/j]]]
54 structures), we developed a method to enrich IGT The base of several hundred manually mapped data and then extract syntactic information (e.g., grams has provided a reasonably reliable “seman- context-free rules) to bootstrap NLP tools such tic search” facility in ODIN, which allows lin- as POS taggers and parsers. The enrichment al- guists to find instances with particular kinds of gorithm first parses the English translation with markup. For example, searching for Perfective an English parser, then aligns the language line Aspect finds instances of data where the data was and the English translation via the gloss line, and marked up with PERF, PFV, etc., but also excludes finally projects syntactic information (e.g., POS instances that map to “Perfect Tense”. While tags and phrase structures) from English to the lan- the manually created mapping table covers many guage line. For instance, given the IGT example in common grams, it is far from complete, especially Ex (4), the enrichment algorithm would produce since linguists can coin new grams all the time. the word alignment in Figure 1 and the phrase We are currently automating the mapping by using structures in Figure 2. The algorithm was tested the grams in the table as labeled data or seeds and on 538 IGTs from seven languages and the word classifying new grams using supervised or semi- alignment accuracy was 94.1% and projection ac- supervised methods. This work, however, is still curacy (i.e., the percentage of correct links in the too preliminary to be included in this paper. projected dependency structures) was 81.5%. De- tails of the algorithm and the experiments are dis- 4.4 Answering questions in language profiles cussed in (Xia and Lewis, 2007). The final step of Stage 2 is answering questions in (4) Rhoddodd yr athro lyfr i'r bachgen ddoe gave-3sg the teacher book to-the boy yesterday language profiles. Some questions are easier to an- ``The teacher gave a book to the boy yesterday'' swer than others. For instance, to determine what (Bailyn, 2001) grammatical or lexical cases are available in a lan-
Target line: Rhoddodd yr athro lyfr i'r bachgen ddoe guage according to the data in ODIN, we simply need to look at the grams in the data that map to the Gloss line: gave-3sg the teacher book to-the boy yesterday case category in GOLD. Other questions are more Translation: The teacher gave a book to the boy yesterday complex; for instance, to determine whether mul- tiple wh-questions are allowed in a language, we Figure 1: Aligning the language line and the En- need to examine the projected syntactic structure glish translation with the help of the gloss line for the language line and look for the positions of any wh-words that were projected relative to one
S S another. A case study is reported next.
VBD NP1 VP NP NP PP NP DT DT NN VBD NP2 PP NP4 NN IN+DT NN rhoddodd NN 4.5 A case study: Answering typological IN NP3 NN gave DT NN (gave) yr The NN i'r bachogen (the) teacher DT NN (to-the) (boy) ddoe questions a book to athro (yesterday) yesterday lyfr (teacher) the boy (book) Two biases are prevalent in IGT data, due to the opportunistic way in which it is harvested and en- Figure 2: Projecting phrase structure from the riched: The first is what we call the IGT-bias, that translation line to the language line is, the bias produced by the fact that IGT examples are used by authors to illustrate a particular fact 4.3 Identifying and mapping grams about a language, causing the collection of IGT for The third step of Stage 2 identifies grams on the the language to suffer from a potential lack of rep- gloss line of an IGT and mapping them to some resentativeness. The second we call the English- common semantic so that they can reliably be bias, an English-centrism resulting from the fact searched. The gloss line of IGT has two types of that most IGT examples provide an English trans- glosses: those representing grammatical informa- lation which is used to enrich the language line: tion (grams) such as NOM, 3sg, PERF, and stan- as discussed in Section 4.2, the enrichment algo- dard glosses such as book or give. Early work in rithm assigns a parse tree to the English transla- ODIN involved significant manual effort to map tion which is then projected onto the langauge line. grams to GOLD concepts.10 Since the original parse is built over English data, the projected parse suffers from a bias caused by 10See (Lewis, 2006) for more background on mapping grams to GOLD concepts, and (Farrar, 2003) and (Farrar and Langendoen, 2003) for more detailed background on GOLD.
55 the English source. Because of these biases and er- than 40 linguists, many of them the leading author- rors introduced at various stages of processing, au- ities in the field. In contrast, the language profiles tomatically generated language profiles and asso- in our work are created automatically from oppor- ciated examples should be treated as preliminary tunistically harvested and enriched linguistic data and unattested, subject to verification by the lin- found on the Web (essentially the IGT in ODIN). guist. The question is how reliable the profiles are. Another difference is that our language profiles To answer the question, we ran a case study in also include highly language-specific information which we evaluated the accuracy of our system in (e.g., lists of language-specific syntactic construc- answering a number of typological questions, such tions, such as bei- and ba- constructions in Man- as the canonical order of constituents (e.g., sen- darin), as discussed in harvested documents. The tential word order, order of constituents in noun information is gathered by checking the construc- phrases) or the existence of particular constituents tion names included in and surrounding IGT. in a language (e.g., determiners). The list of ques- The benefits of our approach are twofold. First, tions and their possible answers are shown in Ta- we can build language profiles for hundreds of ble 3 (the WALS # is a reference number used in languages with little human effort and the lan- WALS (Haspelmath et al., 2005) which uniquely guage profiles can be updated whenever the ODIN identifies each typological parameter). database is expanded or enriched. Second, each In one experiment, we automatically found the entry in the language profile in ODIN is linked to answer to the canonical word order question by the relevant IGT instances that are used to answer looking at the context free rules extracted from the question. For instance, a language profile not enriched IGT data. When tested on about 100 only lists the canonical word order of the language languages, the accuracy was 99% for all the lan- but also IGT instances from which this informa- guages with at least 40 IGT instances.12 Not sur- tion is derived. prisingly, the accuracy decreased for languages with fewer instances (e.g., 65% for languages with 5 Extending the search facility 5-9 IGTs). In another experiment, our system an- The last stage of the three-stage approach is to pro- swered all the 13 typological questions in Table 3 vide a search facility for linguists to search the for 10 languages and the accuracy was 83.1% on original IGTs, the enriched IGTs and the automat- average across the questions. ically created language files. The current search This study shows that, despite potential biases interface for ODIN allows a variety of search op- and errors, we can automatically discover certain tions, including search by language name or code, kinds of linguistic knowledge from IGT with rea- language family, and by grams and their related sonable accuracy and the accuracy increases as concepts (e.g., Accusative case). Once data is dis- more data becomes available. The language pro- covered that fits a particular pattern that a user is files built this way could serve as a complement to interested in, he/she can either display the data manually crafted resources such as WALS. (where sufficient citation information exists and 4.6 Comparison with WALS where the data is not corrupted by the text-to- pdf conversion process) or locate documents from The task is similar to the goal of the WALS which the data is extracted. Additional search fa- project. In fact, the morphological and syntactic cilities allow users to search across linguistically features in WALS form the initial attribute set for 13 salient structures (“constructions”) and return re- our language profiles. The main difference be- sults in the form of language data and language tween WALS and our approach is that the informa- profiles. tion in WALS (including features, feature values, The ODIN database also contains thousands and data points) was gathered by a team of more of tree structures for hundreds of languages, 12Some IGT instances are not sentences and therefore are each linked to the English tree structures from not useful for answering this question. Further, those in- which they were derived. This can provide un- stances marked as ungrammatical (usually with an asterisk “*”) are ignored for this and all typological questions. precedented options for cross-lingual query across 14 13WALS uses the term feature to refer to a property such as “syntactic structures”. canonical word order. Since feature in NLP has a very differ- ent meaning, in this paper we use the term attribute instead 14We fully recognize that the projected structures should to avoid potential confusion. be considered highly experimental, due to noise in the pro-
56 Table 3: Thirteen typlogical questions tested in the case study (ndo=no dominant order, nr=not relevant) Label WALS # Description Possible Values Word Order WOrder 330 Order of Words in a sentence SVO,SOV,VSO,VOS,OVS, OSV,ndo11 V+OBJ 342 Order of the Verb, Object and Oblique Object (e.g., PP) VXO,VOX,OVX,OXV,XVO,XOV,ndo DT+N N/A Order of Nouns and Determiners (a, the) DT-N, N-DT, ndo, nr Dem+N 358 Order of Nouns and Demonstrative Determiners Dem-N, N-Dem, ndo, nr JJ+N 354 Order of Adjectives and Nouns JJ-N, N-JJ, ndo PRP$+N N/A Order of possessive pronouns and nouns PRP$-N, N-PRP$, ndo, nr Poss+N 350 Order of Possessive NPs and nouns NP-Poss, NP-Poss, ndo, nr P+NP 346 Order of Adpositions and Nouns P-NP, NP-P, ndo Morpheme Order N+num 138 Order of Nouns and Number Inflections (Sing, Plur) N-num, num-N, ndo N+case 210 Order of Nouns and Case Inflections N-case, case-N, ndo, nr V+TA 282 Order of Verbs and Tense/Aspect Inflections V-TA, TA-V, ndo, nr Existence Tests Def 154 Do definite determiners exist? Yes, No Indef 158 Do indefinite determiners exist? Yes, No
We plan to extend the current query facility in existing linguistic scholarly discourse is collected three steps to allow these structure-based queries. and stored in the ODIN database. Second, the The first step is to do a user study and identify the language data is automatically analyzed and en- types of queries that linguists would be interested riched, and language profiles are created from the in. We have already consulted with a number of enriched data. Our case study shows that knowl- syntacticians and other linguists, and have com- edge discovery (for the targeted attributes) works piled a list of “constructions” that would be of the reasonably well with even a small amount of IGT most interest, and plan to consult with more lin- data. Third, a search facility is provided that al- guists to extend this list.15 Some of the initial con- lows linguists to search the original data, the en- struction queries have already been implemented riched data, and the language profiles by language in ODIN as “prototypes” for testing purposes. The name, language family, and construction names. second step is to identify tools that would facili- There are several directions for future research. tate implementing these queries. One such tool is We will improve and thoroughly evaluate the mod- tgrep2,16 which is widely used to search treebank ule that extracts various fields from IGT. We will style phrase structures. Since the tool is robust and also build more complete language profiles for a widely used and supported, we plan to extend it dozen or so languages for which we have suffi- to handle the rich data structures found in the en- cient IGT data and linguistic knowledge to ade- riched IGT data. The third step is to write a large quately evaluate the results. Finally, we are ex- set of queries in tgrep2 (or other query languages) ploring ways of extending the query facility (e.g., that “pre-package” the most desirable queries into using tgrep2) to allow sophisticated search on the a form that can be easily executed as a Web ser- original and enriched IGT data, and plan to pro- vice, and design a Web GUI that provides the most vide a GUI with pre-packaged queries which will accessibility to these queries. be easy for linguists to use.
6 Conclusion Acknowledgements This work has been sup- ported, in part, by NSF grants BCS-0748919 and One of the major obstacles that linguists encounter BCS-0720670 and a RRF grant from the Univer- is finding data relevant to their research. In this sity of Washington. We would also like to thank paper, we outline a three-stage procedure to allevi- four anonymous reviewers for their valuable com- ate the problem. First, language data embedded in ments. jection algorithms, and the resulting structures still need to be reviewed by the linguist throwing the query. However, our case study demonstrates the reasonably high accuracy of an- References swering typological questions with even very limited supplies of data. This supports their utility in spite of noise and error. John Frederick Bailyn. 2001. Inversion, Dislocation 15A similar study was discussed in (Soehn et al., 2008). and Optionality in Russian. In Gerhild Zybatow, ed- 16http://tedlab.mit.edu/˜ dr/TGrep2/ itor, Current Issues in Formal Slavic Linguistics.
57 Mark C. Baker and Osamuyimen Thompson Stewart. Hoifung Poon and Pedro Domingos. 2007. Joint in- 1996. Unaccusativity and the adjective/verb distinc- ference in information extraction. In Proceedings tion: Edo evidence. In Proceedings of the Fifth An- of the Twenty-Second National Conference on Artifi- nual Conference on Document Analysis and Infor- cial Intelligence (AAAI), pages 913–918, Vancouver, mation Retrieval (SDAIR), Amherst, Mass. Canada. AAAI Press.
Balthasar Bickel and Johanna Nichols. 2002. Autoty- M. Richardson and P. Domingos. 2006. Markov logic pologizing databases and their use in fieldwork. In networks. Machine Learning, pages 107–136. Proceedings of the LREC Workshop on Resources and Tools in Field Linguistics, Las Palmas, Spain, Jan-Philipp Soehn, Heike Zinsmeister, and Georg Jun. Rehm. 2008. Requirements of a user-friendly, general-purpose corpus query interface. In Pro- Steven Bird and Gary Simons. 2003. Extending dublin ceedings of the LREC 2008 Workshop Sustainability core metadata to support the description and discov- of Language Resources and Tools for Natural Lan- ery of language resources. Computers and the Hu- guage Processing, Marrakech, Morocco, May 31. manities, 17(4):375–388. B. Wellner, A. McCallum, F. Peng, and M. Hay. 2004. William B. Cavnar and John M. Trenkle. 1994. N- An integrated, conditional model of information ex- gram-based text categorization. In Proceedings of traction and coreference with application to citation SDAIR-94, 3rd Annual Symposium on Document matching. In Proc. of the 20th Conference on Un- Analysis and Information Retrieval, pages 161–175, certainty in AI (UAI 2004). Las Vegas, US. Fei Xia and William Lewis. 2007. Multilingual struc- William Croft. 2003. Typology and Universals. Cam- tural projection across interlinear text. In Proc. of bridge University Press, Cambridge, England. the Conference on Human Language Technologies (HLT/NAACL 2007), pages 452–459, Rochester, Alexis Dimitriadis, Menzo Windhouwer, Adam New York. Saulwick, Rob Goedemans, and Tams Br. forth- coming. How to integrate databases without start- Fei Xia and William Lewis. 2008. Repurposing ing a typology war: the typological database sys- Theoretical Linguistic Data for Tool Development tem. In Simon Musgrave Martin Everaertand Alexis and Search. In Proc. of the Third International Dimitriadis, editors, The Use of Databases in Cross- Joint Conference on Natural Language Processing Linguistic Studies. Mouton de Gruyter, Berlin. (IJCNLP-2008), Hyderabad, India. Fei Xia, William D. Lewis, and Hoifung Poon. 2009. Scott Farrar and D. Terence Langendoen. 2003. A lin- Language ID in the Context of Harvesting Language guistic ontology for the semantic web. GLOT Inter- Data off the Web. In Proceedings of The 12th Con- national, 7(3):97–100. ference of the European Chapter of the Association Scott Farrar and William D. Lewis. 2006. The of Computational Linguistics (EACL 2009), Athens, GOLD Community of Practice: An infras- Greece, April. tructure for linguistic data on the Web. Lan- guage Resources and Evaluation. Available at http://faculty.washington.edu/wlewis2/papers/FarLew- 06.pdf.
Scott Farrar. 2003. An ontology for linguistics on the Semantic Web. Ph.d., University of Arizona, May.
Raymond G. Gordon, editor. 2005. Ethnologue: Lan- guages of the World. SIL International, Dallas, 15 edition.
T. Grenager, D. Klein, and D. Manning. 2005. Unsu- pervised learning of field segmentation models for information extraction. In In Proc. ACL-05.
Martin Haspelmath, Matthew Dryer David Gil, and Bernard Comrie, editors. 2005. World Atlas of Lan- guage Structures. Oxford University Press, Oxford.
William Lewis. 2006. ODIN: A Model for Adapting and Enriching Legacy Infrastructure. In Proc. of the e-Humanities Workshop, held in cooperation with e- Science 2006: 2nd IEEE International Conference on e-Science and Grid Computing, Amsterdam.
58 Instance-driven Discovery of Ontological Relation Labels
Marieke van Erp, Antal van den Bosch, Sander Wubben, Steve Hunt ILK Research Group Tilburg centre for Creative Computing Tilburg University The Netherlands {M.G.J.vanErp,Antal.vdnBosch,S.Wubben,S.J.Hunt}@uvt.nl
Abstract bootstrapped from such a database by first assum- ing that the database columns can be mapped onto An approach is presented to the auto- the domain’s ontological classes. The next step matic discovery of labels of relations be- is then to determine which classes are related to tween pairs of ontological classes. Using each other, and by which relation. In this paper a hyperlinked encyclopaedic resource, we we present a method that partially automates this gather evidence for likely predicative la- process. bels by searching for sentences that de- To gather evidence for a relation to exist be- scribe relations between terms. The terms tween two ontological classes, it is not possible to are instances of the pair of ontological simply look up the classes in text. Rather, classes classes under consideration, drawn from are realised typically as a multitude of terms or a populated knowledge base. Verbs or phrases. For example, the natural history class verb phrases are automatically extracted, “species” is realised as many different instances of yielding a ranked list of candidate rela- species names in text. The automatic discovery of tions. Human judges rate the extracted relations between ontological classes thus requires relations. The extracted relations provide at least a two-step approach: first, the identifica- a basis for automatic ontology discovery tion of instances of ontological classes in text and from a non-relational database. The ap- their particular relations, and second, the aggrega- proach is demonstrated on a database from tion of these analyses in order to find evidence for the natural history domain. a most likely relation. 1 Introduction It is common in ontology construction to use predicative labels for relations. Although no regu- The rapid growth in the digitisation of data has lations for label names exist, often a verb or verb caused many curators, researchers, and data man- phrase head is taken, optionally combined with a agers of cultural heritage institutions (libraries, prepositional head of the subsequent verb-attached archives, museums) to turn to knowledge man- phrase (e. g., “occurs in”, or “donated by”). In agement systems. Using these systems typically this study, we make the assumption that good can- causes them to think about the ontological struc- didate labels are frequent verbs or verb phrases ture of their domain, involving the identification found between instances from a particular pair of of key classes in object data and metadata fea- classes, and that this may sometimes involve a tures, and importantly, their relations. The start- verb-attached prepositional phrase containing one ing point of this process is often a more classi- of the two terms. In this paper we explore this cal “flat” database matrix model of size n × m, route, and present a case study on the discovery where n is the number of collection items, and m of predicative labels on relations in an ontology is a fixed number of database columns, typically for animal specimen collections. The first step, denoting object metadata features, as cultural her- identifying instances of ontological classes, is per- itage institutions are generally well accustomed to formed by selecting pairs of instances from a flat using databases of that type. An ontology can be n × m specimen database, in which the instances
Proceedings of the EACL 2009 Workshop on Language Technology and Resources for Cultural Heritage, Social Sciences, Humanities, and Education –LaTeCH – SHELT&R 2009, pages 60–68, Athens, Greece, 30 March 2009. c 2009 Association for Computational Linguistics
59 are organised by the database columns, and there by the system in Section 4 and an explanation is a one-to-one relationship between the database of how we evaluated the possible relations our columns and the classes in our ontology. system discovered is presented in Section 5. We Any approach that bases itself on text to dis- report on the results of our study in Section 6, and cover relations, is dependent on the quality of formulate our conclusions and points for further that text. In this study we opt for Wikipedia research in Section 7. as a resource from which to extract relations be- tween terms. Although the status of Wikipedia 2 Related Work as a dependable resource is debated, in part be- A key property of Wikipedia is that it is for the cause of its dynamic nature, there is some evi- greater part unstructured. On the one hand, ed- dence that Wikipedia can be as reliable a source itors are encouraged to supply their articles with as one that is maintained solely by experts (Giles, categories. These categories can be subsumed by 2005). Wikipedia is also an attractive resource due broader categories, thus creating a taxonomy-like to its size (currently nearly 12 million articles in structure. On the other hand, editors can link to over 250 languages). Additionally, Wikipedia’s any other page in Wikipedia, no matter if it is part strongly hyperlinked structure closely resembles a of the same category, or any category at all. An semantic net, with its untyped (but directed) re- article can be assigned multiple categories, but the lations between the concepts represented by the number of hyperlinks provided in an average arti- article topics. Since the hyperlinks in Wikipedia cle typically exceeds the number of categories as- indicate a relations between two encyclopaedia ar- signed to it. ticles, we aim at discovering the type of relation The free associative hyperlink structure of such a link denotes through the use of syntactic Wikipedia is intrinsically different from the hier- parsing of the text in which the link occurs. archical top down architecture as seen in Word- The idea of using Wikipedia for relation ex- Net, as a hyperlink has a direction, but not a traction is not new (Auer and Lehmann, 2007; type. A Wikipedia article can contain any num- Nakayama et al., 2008; Nguyen et al., 2007; ber of links, pointing to any other Wikipedia arti- Suchanek et al., 2006; Syed et al., 2008). How- cle. Wikipedia guidelines state however that wik- ever, most studies so far focus on the structured ilinks (hyperlinks referring to another Wikipedia information already explicit in Wikipedia, such as page) should only be added when relevant to the its infoboxes and categories. The main contribu- topic of the article. Due to the fact that most users tions of our work are that we focus on the in- tend to adhere to guidelines for editing Wikipedia formation need emerging from a specific domain, pages and the fact that articles are under constant and that we test a method of pre-selection of sen- scrutiny of their viewers, most links in Wikipedia tences to extract relations from. The selection is are indeed relevant (Blohm and Cimiano, 2007; based on the assumption that the strongest and Kamps and Koolen, 2008). most reliable lexical relations are those expressed The structure and breadth of Wikipedia is a po- by hyperlinks in Wikipedia pages that relate an ar- tentially powerful resource for information extrac- ticle topic to another page (Kamps and Koolen, tion which has not gone unnoticed in the natu- 2008). The selection procedure retains only sen- ral language processing (NLP) community. Pre- tences in which the topic of the article, identified processing of Wikipedia content in order to ex- by matching words in the article title, links to an- tract non-trivial relations has been addressed in a other Wikipedia article. The benefit of the pre- number of studies. (Syed et al., 2008) for instance selection of sentences is that it reduces the work- utilise the category structure in Wikipedia as an load for the syntactic parser. upper ontology to predict concepts common to a Since the system is intentionally kept set of documents. In (Suchanek et al., 2006) an lightweight, the extraction of relations from ontology is constructed by combining entities and Wikipedia is sufficiently fast, and we observe that relations between these extracted from Wikipedia the results are sufficient to build a basic ontology through Wikipedia’s category structure and Word- from the data. This paper is organised as follows. Net. This results in a large “is-a” hierarchy, draw- In Section 2 we review related work. In Section 3 ing on the basis of WordNet, while further rela- the data used in this work is described, followed tion enrichments come from Wikipedia’s category
60 structure. (Chernov et al., 2006) also exploit the 3 Data Preparation Wikipedia category structure to which concepts in 3.1 Data the articles are linked to extract relations. The data used in this work comes from a manu- (Auer and Lehmann, 2007) take a different ap- ally created, non-relational research database of a proach in that they focus on utilising the structure collection of reptiles and amphibians at a natural present in infoboxes. Infoboxes are consistently history museum. The information contained in the formatted tables in articles that provide summary cells describes when a specimen entered the col- information, such as information about area, pop- lection, under what circumstances it was collected, ulation and language for countries, and birth dates its current location, registration number, etc. We and places for persons. Although infoboxes pro- argue that the act of retrieving information from vide rich structured information, their templates this flat database could be enhanced by providing are not yet standardised, and their use has not per- a meta-structure that describes relations between meated throughout the whole of Wikipedia. the different database columns. If for instance a relation of the type ”is part of” can be defined be- Although the category and infobox structures in tween the database columns province and country, Wikipedia already provide a larger coverage at the then queries for specimens found at a particular concept or term level than for instance WordNet, location can be expanded accordingly. they do not express all possibly relevant seman- Even though the main language of the database tic relations. Especially in specific domains, re- is Dutch, we still chose to use the English lations occur that would make the Wikipedia data Wikipedia as the resource for retrieval of rela- structure unnecessarily dense if added, thus an ap- tion label candidates. Explicitly choosing the En- proach that exploits more of the linguistic content glish Wikipedia has as a consequence that the of Wikipedia is desirable. relation labels we are bound to discover will be English phrases. Furthermore, articles in Such approaches can be found in (Nakayama et the English Wikipedia on animal taxonomy have al., 2008) and (Nguyen et al., 2007). In both works a broader coverage and are far more elaborate full sections of Wikipedia articles are parsed, en- than those contained in the Dutch Wikipedia. tities are identified, and the verb between the enti- Since these database values use a Latin-based ties is taken as the relation. They also extract re- nomenclature, using the wider-coverage English lations that are not backed by a link in Wikipedia, Wikipedia yields a much higher recall than the resulting in common-sense factoids such as ‘Bres- Dutch Wikipedia. The values of the other columns cia is a city’. For a domain specific application mainly contain proper names, such as person this approach lacks precision. In our approach, we names and geographic locations and dates, which care more for high precision in finding relations are often the same; moreover, English and Dutch than for recall; hence, we carefully pre-select on- are closely related languages. Different names ex- tological classes among which relations need to be ist for different countries in each language, but found, and use these as filters on our search. here the inconsistency of the database aids us, as it in fact contains many database entries partially The usefulness of the link structure in or fully in English, as well as some in German and Wikipedia has been remarked upon by (Volkel¨ et Portuguese. al., 2006). They acknowledge that the link struc- The database contains 16,870 records in 39 ture in Wikipedia denotes a potentially meaning- columns. In this work we focus on 20 columns; ful relation between two articles, though the re- the rest are discarded as they are either extrinsic lation type is unknown. They propose an exten- features not directly pertaining to the object they sion to the editing software of Wikipedia to enable describe, e.g., a unique database key, or elaborate users to define the type of relation when they add textual information that would require a separate a link in Wikipedia. Potentially this can enrich processing approach. The columns we focus on Wikipedia tremendously, but the work involved describe the position of the specimen in the zo- would be tremendous as well. We believe some of ological taxonomy (6 columns), the geographical the type information is already available through location in which it was found (4 columns), some the linguistic content of Wikipedia. of its physical properties (3 columns), its collector
61 Column Name Value index was built of a subset of the link structure Taxonomic Class Reptilia present in Wikipedia. The subset of links included Taxonomic Order Crocodylia in the index is constrained to those links occur- Amphisbaenia ring in sentences from each article in which the Taxonomic Genus Acanthophis main topic of the Wikipedia article (as taken from Xenobatrachus the title name) occurs. For example, from the Country Indonesia Wikipedia article on Anura the following sentence Suriname would be included in the experiments1: Location city walls near Lake Mahalona The frog is an [[amphibian]] in the order Anura Collection Date 01.02.1888 (meaning “tail-less”, from Greek an-, without + 02.01.1995 oura, tail), formerly referred to as Salientia (Latin Type holotype saltare, to jump) paralectotype Determinator A. Dubois whereas we would exclude the sentence: M. S. Hoogmoed Species defined by (Linnaeus, 1758) An exception is the [[fire-bellied toad]] (Bombina (LeSueur, 1827) bombina): while its skin is slightly warty, it prefers a watery habitat. Table 1: Example classes from test data This approach limits the link paths to only and/or determiner, donator and associated date (4 those between pages that are probably semanti- columns), and other information (3 columns). The cally strongly connected to each other. In the values in most columns are short, often consist- following section the computation of the link ing of a single word. Table 1 lists some example paths indicating semantic relatedness between two database values. Wikipedia pages is explained. 3.3 Computing Semantic Relatedness 3.2 Preprocessing Relation discovery between terms (instantiations As the database was created manually, it was nec- of different ontological classes) that have a page essary to normalise spelling errors, as well as in Wikipedia is best performed after establishing variations on diacritics, names and date formats. if a sufficiently strong relation between the two The database values were also stripped of all non- terms under consideration actually exists. To do alphanumeric characters. this, the semantic relatedness of those two terms In order to find meaningful relations between or concepts needs to be computed first. Seman- two database columns, query pairs are generated tic relatedness can denote every possible relation by combining two values occurring together in a between two concepts, unlike semantic similarity, record. This approach already limits the number which typically denotes only certain hierarchical of queries applied to Wikipedia, as no relations are relations (like hypernymy and synonymy) and is attempted to be found between values that would often computed using hierarchical networks like not normally occur together. This approach yields WordNet (Budanitsky and Hirst, 2006). a query pair such as Reptilia Crocodylia from the A simple and effective way of computing se- taxonomic class and order columns, but not Am- mantic relatedness between two concepts c and c phibia Crocodylia. Because not every database 1 2 is measuring their distance in a semantic network. field is filled, and some combinations occur more This results in a semantic distance metric, which often, this procedure results in 186,141 query can be inversed to yield a semantic relatedness pairs. metric. Computing the path-length between terms For this study we use a database snapshot of the c and c can be done using Formula 1 where P is English Wikipedia of July 27, 2008. This dump 1 2 the set of paths connecting c to c and Np is the contains about 2.5 million articles, including a vast 1 2 number of nodes in path p. amount of domain-specific articles that one would typically not find in general encyclopaedias. An 1The double brackets indicate Wikilinks
62 Step 2 The system finds the shortest link path be- 1 tween the two selected Wikipedia articles. If the relpath(c1, c2) = argmaxp∈P (1) Np path distance is 1, this means that the two con- cepts are linked directly to each other via their We search for shortest paths in a semantic net- Wikipedia articles. This is for instance the case work that is constructed by mapping the concepts for Megophrys from the genus column, and Anura in Wikipedia to nodes, and the links between the from the order column. In the Wikipedia article on concepts to edges. This generates a very large Megophrys, a link is found to the Wikipedia arti- network (millions of nodes and tens of millions cle on Anura. There is no reverse link from Anura of edges), but due to the fact that Wikipedia is to Megophrys; hierarchical relationships in the zo- scale-free (Barabasi and Albert, 1999) (its con- ological taxonomy such as this one are often uni- nectedness degree distribution follows a power- directional in Wikipedia as to not overcrowd the law), paths stay relatively short. By indexing parent article with links to its children. both incoming and outgoing links, a bidirectional breadth-first search can be used to find shortest Step 3 The sentence containing both target con- paths between concepts. This means that the cepts as links is selected from the articles. search is divided in two chains: a forward chain From the Megophrys article this is for instance “Megophrys is a genus of frogs, order [[Anura]], from c1 and a backward chain to c2. As soon as the two chains are connected, a shortest path is found. in the [[Megophryidae]] family.” Step 4 If the shortest path length between two 4 Extracting Relations from Wikipedia Wikipedia articles is 2, the two concepts are linked via one intermediate article. In that case the sys- Each query pair containing two values from two tem checks whether the title of the intermediate ar- database columns are sent to the system. The sys- ticle occurs as a value in a database column other tem processes each term pair in four steps. A than the two database columns in focus for the schematic overview of the system is given in Fig- query. If this is indeed the case, the two addi- ure 1. tional relations between the first term and the in- termediate article are also investigated, as well as Term 1 Term 2 the second term and that of the intermediate ar- ticle. Such a bridging relation pair is found for Interm. Term 1 Term Indexed instance for the query pair Hylidae from the tax- Wikipedia Interm. Term 2 Step 1 corpus Term onomic order column, and Brazil from the coun- try column. Here, the initial path we find is Hyl- Step 4 idae ↔ Sphaenorhynchys → Brazil. We find that if found Art. Art. find path length Step 2 1 2 the article-in-the-middle value (Sphaenorhynchys)
if path length == 2 indeed occurs in our database, in the taxonomic find intermediate value in database if path length == 1 genus column. We assume this link is evi- dence for co-occurrence. Thus, the relevant sen- Step 3 ...
63 between the column pairs, where the subject of the 6 Results and Evaluation sentence is a value from one of the two columns, are presented to the human annotators. The cut-off As expected, between certain columns there are of five was chosen to prevent the annotators from more relations than between others. In total 140 having to evaluate too many relations and to only relation candidates were retrieved directly, and present those that occur more often, and are hence 303 relation label candidates were retrieved via an less likely to be misses. Misses can for instance intermediate Wikipedia article. We work with the be induced by ambiguous person names that also assumption that these columns have a stronger on- accidentally match location names (e.g., Dakota). tological relation than others. For some database In Section 7 we discuss methods to remedy this in columns we could not retrieve any relations, such future work. as the “collection date” field. This is not sur- prising, as even though Wikipedia contains pages about dates (‘what happened on this day’), it is 5 Evaluating Relations from Wikipedia unlikely that it would link to such a domain spe- cific event such as an animal specimen collec- Four human judges evaluated the relations be- tion. Relations between instances denoting per- tween the ontological class pairs that were ex- sons and other concepts in our domain are also not tracted from Wikipedia. Evaluating semantic rela- discovered through this approach. This is due to tions automatically is hard, if not impossible, since the fact that many of the biologists named in the the same relation can be expressed in many ways, database do not have a Wikipedia page dedicated and would require a gold standard of some sort, to them, indicating the boundaries of Wikipedia’s which for this domain (as well as for many cul- domain specific content. Although not ideal, a tural heritage domains) is not available. named-entity recognition filter could be applied to the database after which person names can be re- The judges were presented with the five highest- trieved from other resources. ranked candidate labels per column pair, as well a Occasionally we retrieve a Wikipedia article for longer snippet of text containing the candidate la- a value from a person name column, but in most bel, to resolve possible ambiguity. The items in cases this mistakenly matches with a Wikipedia each list were scored according to the total recip- article on a location, as last names in Dutch are rocal rank (TRR) (Radev et al., 2002). For every often derived from place names. Another problem correct answer 1/n points are given, where n de- induced by incorrect data is the incorrect match notes the position of the answer in the ranked list. of Wikipedia pages on certain values from the If there is more than 1 correct answer the points “Town” and “Province” columns. Incorrect rela- will be added up. For example, if in a list of five, tion candidates are retrieved because for instance two correct answers occur on positions 2 and 4, the the value ‘China’ occurs in both the “Town” and TRR would be calculated as (1/2 + 1/4) = .75. the “Province” columns. A data cleaning step The TRR scores were normalised for the number would solve these two problems. of relation candidates that were retrieved, as for From each column pair the highest rated rela- some column pairs less than five relation candi- tion was selected with which we constructed the dates were retrieved. ontology displayed in Figure 2. As the figure As an example, for the column pair “Province” shows, the relations that are discovered are not and “Genus”, the judges were presented with the only ‘is a’-relations one would find in strictly hier- relations shown in Table 2. The direction arrow archical resources such as a zoological taxonomy in the first column denotes that the “Genus” value or geographical resource. occurred before the “Province” value. The numbers in the relation labels in Figure 2 The human judges were sufficiently familiar denote the average TRR scores given by the four with the domain to evaluate the relations, and had judges on all relation label candidates that the the possibility to gain extra knowledge about the judges were presented with for that column pair. class pairs through access to the full Wikipedia The scores for the relations between the taxo- articles from which the relations were extracted. nomic classes in our domain were particularly Inter-annotator agreement was measured using high, meaning that in many cases all relation can- Fleiss’s Kappa coefficient (Fleiss, 1971). didates presented to the judges were assessed as
64 Direction Label Snippet → is found in is a genus of venomous pitvipers found in Asia from Pakistan, through India, → is endemic to Cross Frogs) is a genus of microhylid frogs endemic to Southern Philippine, → are native to are native to only two countries: the United States and → is known as is a genus of pond turtles also known as Cooter Turtles, especially in the state of
Table 2: Relation candidates for Province and Genus column pair correct. The inter-annotator agreement was κ = we intend to incorporate more domain specific re- 0.63, which is not perfect, but reasonable. Most sources, such as research papers from the domain disagreement is due to vague relation labels such that may mention persons from our database. We as ‘may refer to’ as found between “Province” and are also looking into sending queries to the web, “Country”. If a relation that occurred fewer than 5 whilst keeping the constraint of hyperlink pres- times was judged incorrect by the majority of the ence. judges the relation was not included in Figure 2. Another factor that may help back up the rela- Manual fine-tuning and post-processing of the tions already discovered is more evidence for ev- results could filter out synonyms such as those ery relation. Currently we only include sentences found for relations between “Town” and other in our Wikipedia corpus that contain the literal classes in the domain. This would for instance de- words from the title of the article, to ensure we fine one particular relation label for the relations have content that is actually about the article and ‘is a town in’ and ‘is a municipality in’ that the sys- not a related topic. This causes many sentences tem discovered between “Town” and “Province” in which the topic is referred to via anaphoric ex- and “Town” and “Country”, respectively. pressions to be missed. (Nguyen et al., 2007) take the most frequently used pronoun in the article as 7 Conclusion and Future Work referring to the topic. This still leaves the prob- lem of cases in which a person is first mentioned In this work we have shown that it is possible by his/her full name and subsequently only by last to extract ontological relation labels for domain- name. Coreference resolution may help to solve specific data from Wikipedia. The main contri- this, although accuracies of current systems for en- bution that makes our work different from other cyclopaedic text are often not much higher than work on relation extraction from Wikipedia is that baselines such as those adopted by (Nguyen et al., the link structure is used as a strong indication of 2007). the presence of a meaningful relation. The pres- Errors in the database lead to some noise in ence of a link is incorporated in our system by only the selection of the correct Wikipedia article. The using sentences from Wikipedia articles that con- queries we used are mostly single-word and two- tain links to other Wikipedia articles. Only those word terms, which makes disambiguation hard. sentences are parsed that contain the two terms we Fortunately, we have access to the class label (i.e., aim to find a relation between, after which the verb the database column name) which may be added phrase and possibly the article or preposition fol- to the query to prevent retrieval of an article about lowing it are selected for evaluation by four human a country when a value from a person name col- judges. umn is queried. We would also like to inves- The advantage of the pre-selection of content tigate whether querying terms from a particular that may contain a meaningful relation makes our database column to Wikipedia can identify incon- approach fast, as it is not necessary to parse the sistencies in the database and hence perform a whole corpus. By adding the constraint that at database cleanup. Potentially, extraction of re- least one of the query terms should be the sub- lation labels from Wikipedia articles can also be ject of a sentence, and by ranking results by fre- used to assign types to links in Wikipedia. quency, our system succeeds in extracting correct and informative relations labels. However, there is Acknowledgments clearly some room for improvement, for instance in the coverage of more general types of infor- The authors would like to thank the anonymous mation such as dates and person names. For this reviewers for their comments. This research
65 Species Order Town Type
is a (1.000) is a town in on the island of is a (0.794) (0.500) (0.854) is a is a is found in (0.833) (1.000) (0.566) Location is a municipality in Family (0.891) is a (1.000) is a town in is in is a (0.759) Genus (0.500) (0.750) may refer to is found in (0.560) Type Name occur in (0.635) Class (0.750) Country is found in (0.573) occur in may refer to (0.333) (0.482)
Province
Figure 2: Graph of relations between columns, with TRR scores in parentheses was funded as part of the Continuous Access to Walter Daelemans, Sabine Buchholz, and Jorn Veen- Cultural Heritage (CATCH) programme of the stra. 1999. Memory-based shallow parsing. In Pro- Netherlands Organisation for Scientific Research ceedings of CoNLL’99, pages 53–60, Bergen, Nor- way, June 12. (NWO). Walter Daelemans, Jakub Zavrel, Ko Van der Sloot, and Antal Van den Bosch. 2004. Timbl: Tilburg References memory based learner, version 5.1, reference guide. Technical Report 04-02, ILK/Tilburg University. Soren¨ Auer and Jens Lehmann. 2007. What have inns- bruck and leipzig in common? extracting seman- J. L. Fleiss. 1971. Measuring nominal scale agree- tics from wiki content. In Franconi et al., editor, ment among many raters. Psychological Bulletin, Proceedings of European Semantic Web Conference 76(5):378–382. (ESWC’07), volume 4519 of Lecture Notes in Com- Jim Giles. 2005. Internet encyclopaedias go head to puter Science, pages 503–517, Innsbruck, Austria, head. Nature, 438:900–901. June 3 - 7. Springer. Jaap Kamps and Marijn Koolen. 2008. The impor- A. L. Barabasi and R. Albert. 1999. Emer- tance of link evidence in wikipedia. In Craig Mac- gence of scaling in random networks. Science, donald, Iadh Ounis, Vassilis Plachouras, Ian Rutven, 286(5439):509–512, October. and Ryen W. White, editors, Advances in Infor- mation Retrieval: 30th European Conference on Sebastian Blohm and Philipp Cimiano. 2007. Using IR Research (ECIR 2008), volume 4956 of Lecture the web to reduce data sparseness in pattern-based Notes in Computer Science, pages 270–282, Glas- information extraction. In Proceedings of the 11th gow, Scotland, March 30 - April 3. Springer Verlag. European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD), War- Kotaro Nakayama, Takahiro Hara, and Shojiro Nishio. saw, Poland, September. Springer. 2008. Wikipedia link structure and text mining for semantic relation extraction towards a huge scale A. Budanitsky and G. Hirst. 2006. Evaluating global web ontology. In Proceedings of Sem- WordNet-based measures of lexical semantic relat- Search 2008 CEUR Workshop, pages 59–73, Tener- edness. Computational Linguistics, 32(1):13–47. ife, Spain, June 2. Sergey Chernov, Tereza Iofciu, Wolfgang Nejdl, and Dat P. T. Nguyen, Yutaka Matsuo, and Mitsuru Xuan Zhou. 2006. Extracting semantic relation- Ishizuka. 2007. Exploiting syntactic and semantic ships between wikipedia categories. In Proceedings information for relation extraction from wikipedia. of the First Workshop on Semantic Wikis - From Wiki In Proceedings of Workshop on Text-Mining & Link- to Semantics [SemWiki2006] - at ESWC 2006, pages Analysis (TextLink 2007) at IJCAI 2007, pages 153 – 163, Karlsruhe, Germany, May 15. 1414–1420, Hyderabad, India, January 7.
66 Dragomir R. Radev, Hong Q, Harris Wu, and Weiguo Fan. 2002. Evaluating web-based question answer- ing systems. In Demo section, LREC 2002, Las Pal- mas, Spain, June. F. M. Suchanek, G. Ifrim, and G. Wiekum. 2006. Leila: Learning to extract information by linguistic analysis. In Proceedings of the ACL-06 Workshop on Ontology Learning and Population, pages 18–25, Sydney, Australia, July. Zareen Saba Syed, Tim Finin, and Anupam Joshi. 2008. Wikitology: Using wikipedia as an ontology. Technical report, University of Maryland, Baltimore County. Max Volkel,¨ Markus Krotzsch,¨ Denny Vrandecic, Heiko Haller, and Rudi Studer. 2006. Semantic wikipedia. In WWW 2006, pages 585–594, Edin- burgh, Scotland.
67 The Role of Metadata in the Longevity of Cultural Heritage Resources
Milena Dobreva Nikola Ikonomov Centre for Digital Library Research, Institute of Mathematics and University of Strathclyde Informatics 26 Richmond Street Glasgow, G1 bl. 8 Acad. G. Bonchev St. 1XH, Scotland, United Kingdom 1113 Sofia, Bulgaria [email protected] [email protected]
tised and born-digital material…” It also pro- Abstract vides a good practice example from the UK, the Digital Preservation Coalition. Although this Digital preservation is an integral part of British experience is given as an example of a the management of information and the successful approach, the implementation of pres- institutions in the cultural heritage sector ervation in the current digital repositories in the are seeking for ways to incorporate it into UK is far from being satisfactory. The recent their everyday practice. While there are final report of the DRAI project (see Daisy Ab- generic approaches to long-term preser- bott 2008) on UK-based digital repositories iden- vation, further research and development tified that “Of the 3,707collections aggregated, work is needed to address any specifics only 737 (20%) had evidence of a preservation of the digital objects in the cultural heri- policy, and of these the vast majority were from tage domain. In this paper, we will take within the Arts and Humanities Data Service two case studies of recent projects and (AHDS) Collections Repository which, since analyse to what extent the metadata ac- March 2008, is no longer funded, leaving only companying digital objects contribute to 6% of resources with ongoing policies for long guarantee longevity. We summarize this term preservation.” analysis in two scenarios for sustainabil- Why the digital preservation which is recog- ity of resources produced by small pro- nised to be of high importance, is still not im- jects because compared to big institutions plemented widely? The Blue Ribbon Task Force their digital assets are facing a higher risk Report (2008) summarized five major obstacles not to be preserved properly. We also in building sustainable digital preservation pro- identify processes where natural lan- grams: inadequacy of funding models to address guage technologies could be of help to long-term access and preservation needs; confu- make the preservation more efficient. sion and/or lack of alignment between stake- holders, roles, and responsibilities with respect to 1 Introduction digital access and preservation; inadequate insti- tutional, enterprise, and/or community incentives An evaluation made in August 2008 in a recent 1 to support the collaboration needed to reinforce Communication of the EC states that “The ab- sustainable economic models; complacency that sence of clear and comprehensive policies in current practices are good enough; and fear that many Member States was identified in the Rec- digital access and preservation is too big to take ommendation as a threat to the survival of digi- on. While these address mostly stakeholder- related issues, a recent report (DPE: 2007) re- viewed the progress of digital preservation re- 1 Communication from the Commission to the Council, the Euro- pean Parliament, the European Economic and Social Committee search world-wide. It concludes that “The analy- and the Committee of the Regions Europe’s cultural heritage at the sis of the last 16 years of effort in the area of click of a mouse. Progress on the digitisation and online accessibil- ity of cultural material and digital preservation across the EU digital preservation… support our claim that [SEC(08) 2372] of 11 August 2008. Available: while much work has been done on the periph- http://ec.europa.eu/information_society/activities/digital_libraries/d ery, the central problems of digital preservation oc/communications/progress/communication_en.pdf
Proceedings of the EACL 2009 Workshop on Language Technology and Resources for Cultural Heritage, Social Sciences, Humanities, and Education –LaTeCH – SHELT&R 2009, pages 69–76, Athens, Greece, 30 March 2009. c 2009 Association for Computational Linguistics
68 have yet to be resolved.” Automation in digital standard ISO 14721:2003, see (OAIS: 2003). It preservation is one of the ten areas named by this provides a functional approach which is helpful study as being in a need of accelerated research. to understand and describe the processes in a These two reports identify two key areas preservation system. However, the specification which need attention: the organisational aspects of the minimum information which should ac- and the research. Any institution which currently company a digital object in order to guarantee its faces the preservation challenge has to make a accessibility, interpretability and usability in the difficult choice under the circumstances of the future, is not addressed in OAIS. rapidly changing technological environment and There are different ongoing attempts to build a the lack of a generally recommended and effec- theory of preservation. Giorgos Flouris and Carlo tive solution. Meghini in 2007 again suggested basing such a However, the deeper reason why after two theory on firm mathematical foundations and decades of efforts digital preservation still has such an effort will be made within the CASPAR 2 not reached maturity, is the lack of consistent project. Paul Watry in 2007 presented the direc- theoretical foundation. In 2001, James Cheney et tions of future research of the SHAMAN 3 project al. articulated the need to develop a mathematical which aims to study the context which guaran- theory for the area of information preservation. tees that the essential properties of digital objects Having such a theory should help to understand such as authenticity and integrity will be main- better the object models and the basic transfor- tained over time. Reagan Moore and MacKenzie mations which need to be supported in a preser- Smith suggested in 2007 a practical approach, vation system; it will also be of great benefit to which demonstrates how various requirements of automation because it would be much more clear the preservation systems can be implemented as what processes can be automated. In addition, a sets of rules. coherent theory would help to identify any gaps Attempts to find an approach to building a co- in procedures or data flows and thus would con- herent theory of preservation should also reflect tribute to the implementation of systems which the specific features of various subject domains. behaviour could be consistently traced. However, In Section 2 of this paper, we introduce the since this first suggestion that a theory of preser- basic concepts of the ISO standard in the digital vation is needed, there has not been much pro- preservation domain OAIS 4. Then in Section 3 gress in developing one. Even the identification we discuss issues related to the use of metadata of the basic elements of a theory of preservation for preservation in the cultural heritage domain. is not a trivial task. We take two examples of resources under prepa- In addition to this methodological difficulty, ration in the TEXTE project 5 and the KT- the variety of domains where digital preservation DigiCult-BG 6 project and suggest two scenarios is to be applied brings even more difficulties. It for preservation of the products of small projects. is still not clear whether a universal ‘one-size- In Section 4 we summarise the issues which need fits-all’ solution could be used in all different further development in the cultural heritage sec- domains. For example, in the cultural heritage tor in order to address better the issue of longev- domain which is considered in this paper there ity of the digital resources. are various curatorial situations with regard to digital objects. Resources prepared by small pro- jects are in more danger with regard to their lon- gevity compared to resources of large institutions 2 CASPAR – Cultural, Artistic and Scientific knowledge if we consider that they are created but the life- for Preservation, Access and Retrieval – an Integrated Pro- cycle for digital objects curation is not applied in ject co-financed by the EU within FP6. http://www.casparpreserves.eu/caspar-project full. In this paper it is not our aim to investigate 3 SHAMAN – Sustaining Heritage Access through Multiva- what is the level of awareness and readiness of lent ArchiviNg an Integrated Project co-financed by the EU the whole range of cultural heritage institutions within FP7. http://shaman-ip.eu/ but we try to highlight what is the current state in 4 Open archival information system, see OAIS 2003. 5 the digital preservation field and what issues TEXTE: Transfer of Expertise in Technologies of Editing, funded under the Marie Curie programme of FP6, need to be addressed by projects in the cultural http://www.nuigalway.ie/mooreinstitute/projects.php?projec heritage domain. t=15 In 2002, the reference model for an open ar- 6 Knowledge Transfer in Digitisation of Cultural and Scien- chival information system (OAIS) was published tific Heritage to Bulgaria, funded under the Marie Curie and one year later adopted in the international programme of FP6, http://www.math.bas.bg/digi/indexen.html
69 2 The Basic Standard in the Digital ground knowledge of the designated community Preservation Domain: OAIS is not trivial in this domain, especially in the cases of creating scholarly resources. In response to the increased need for the intro- This in fact opens a new essential issue related duction of common practices in assuring long to the dynamics of resources: while the resources term digital preservation (DP) of digital objects, created by memory institutions such as libraries, the International Standards Organisation (ISO) in archives and museums are ‘static’ because they the last decade developed a number of concep- represent the resources of the institutions, the tual DP standards and also some technical guide- resources created by projects tend to be dynamic, lines. The most popular standard in the area is because they could be enriched and extended. ISO 14721:2003 (Space data and information These resources usually do not cover a fixed set transfer systems – Open archival information of holdings but are oriented towards a specific system – Reference model), widely known as theme and after a core set of resources is avail- 7 OAIS . able, more materials could be added at later It is a conceptual framework which presents stages. Being familiar with the model will help the main functional components and identifies cultural heritage professionals to organize better the basic data flows within a digital preservation the digital objects’ lifecycle and take informed 8 system. The development of OAIS arose from decisions on the preservation aspects of their re- the need for a model which would specify the sources. basic components of a system for long-term In addition to the definition of these functional preservation of digital objects and their relation- entities, an OAIS information model explains the ships with the ‘external world’. data flows between the environment and the ar- In Fig. 1, an OAIS is represented as a box chive, and also within the functional components which is connected to other entities from the en- of the archive. vironment. These are Producers , Consumers , Every act of submission of information to an and Management ( OAIS is not suggesting what OAIS by a Producer, as well as the dissemination roles could be defined within the archive). A of information to a Consumer, occurs either as a special class of Consumers is the Designated single discrete transmission, or as a series of Community . The Designated Community is the transmissions. To describe the exchange of data, set of Consumers who should be able to under- OAIS defines the concept of an Information 9 stand the preserved information.” Package as a “container” of two types of infor- mation: Content Information (CI) and Preser- vation Description Information (PDI) . The Content Information and PDI are viewed as be- ing encapsulated and identifiable by the Packag- ing Information (PI) . OAIS defines three specialised types of infor- mation packages (IP), namely: • Archival Information Package (AIP): An Information Package, consisting of the Content Information and the associated Figure 1: The OAIS functional entities and PDI, which is preserved within an OAIS. environment 10 • Dissemination Information Package (DIP): Within the cultural heritage domain, a special The Information Package, derived from attention also should be paid to the concept of one or more AIPs, received by the Con- designated community which is essential for the sumer in response to a request to the understanding of resources in the future. Under- OAIS. standing the actual needs and the level of back- • Submission Information Package (SIP): An Information Package that is delivered 7 A webpage which provides information on the basic stages by the Producer to the OAIS for use in the of its development is http://nost.gsfc.nasa.gov/isoas/. construction of one or more AIPs. 8 For a record of the activities see http://nssdc.gsfc.nasa.gov/nost/isoas/. PDI is divided into four types of preservation 9 Source: p.2.2-2.3, OAIS (2003). information called Provenance, Context, Refer- 10 Source: Figure 4-1, page 4-1 (OAIS, 2003).
70 ence, and Fixity . Provenance describes the source they should help the designated communities of the Content Information; Context describes to understand the resources), and how the CI relates to other information outside 2. How to preserve the metadata accompanying the Information Package. Reference provides existing digital objects. Since it is often the identifiers, by which the CI may be uniquely case that cultural heritage resources have ex- identified. Fixity provides a wrapper, which pro- tensive metadata and there are multiple tects the CI from undocumented alteration. A key schemes used, this issue also touches upon point to note is that OAIS does not suggest any understanding and use of present metadata specific metadata as instantiations of PDI; so that schemes in the future; this issue is very similar there is no guidance on what constitutes a mini- to interoperability of metadata schemes but mum technical requirement in respect of repre- considered as interoperability between now senting and encoding PDI information within and the future. corresponding PDI data bitstreams. The defini- In the area of preservation metadata, the basic tion of a minimum required set of data should be concern, as stated by Brian Lavoie and Richard based on a study of what is required to assure a Gartner, still seems to be the development of reliable, consistent, and measurable specification preservation elements’ schemes (2005). How- and implementation of a preservation system. ever, presentation and interoperability issues are The consequences for the cultural heritage not the only concern: preservation metadata, as domain is that currently there is no single and all other types of metadata are affected by the uniformly accepted set of elements which guar- metadata bottleneck 11 , a metaphor which indi- antee the longevity of resources; but every single cates that the human efforts needed to create institution and project should consider how to metadata can not cope with the pace of creation structure the different types of archival packages of new digital resources. if it establishes and digital archive and also de- Another disquieting example comes from a re- cide what metadata need to be stored for preser- cent evaluation of a German national digitisation vation purposes. programme which reveals “insufficient metadata In OAIS the closest analogue to the high-level practice, endangering the usage of the digital notion of metadata can be found in the idea of documents, not to speak of their preservation: representation information while on the imple- 33% of the objects had no metadata at all, 33% mentation level PDI provides the closest notion bibliographic metadata only, 10% had both bib- to a particular instantiation of a set of metadata liographic and subject metadata (rest: no infor- elements. mation). Less than a third of the metadata was Having a reference framework for long-term digital.” 12 digital preservation systems had an essential im- To suggest an OAIS-compliant preservation pact on the development of a common profes- metadata solution, in 2002 OCLC (Online Com- sional understanding and vocabulary. It has had a puter Library Center, Inc.) and RLG (Research significant impact on the subsequent develop- Libraries Group) created a working group to ex- ment of standards and as a guide to the setting up plore how a consensus between different projects of preservation systems. Yet, the complexity of and initiatives in the field of preservation meta- the area allows for multiple interpretations of data could be achieved. A body of recommended how the framework should be implemented in metadata for digital preservation was developed real life applications. This arguably gives rise to in the next years under the name PREMIS: a need to develop implementation guidelines and PREservation Metadata Implementation Strate- also to adjust any practical solution to policy gies (see PREMIS (2008)). The effort of OCLC frameworks. and RLG to define preservation metadata is a major development in the digital preservation 3 Preservation-Related Metadata field. However, it analyses only preservation metadata related to the digital object itself. A 3.1 Metadata for Preservation vs. Preser- number of issues likely to appear in real imple- vation of Existing Metadata
There are two key issues which need to be con- 11 The term metadata bottleneck was coined by E. Liddy in sidered vis-à-vis metadata and preservation. 2002. 1. What metadata are needed for preservation 12 DELOS brainstorming on Metadata topics, Juan les Pins, purposes in the cultural heritage domain (be- 05/12/2005, sides assuring a reliable preservation process, http://www.ukoln.ac.uk/ukoln/staff/t.koch/pres/Brainst2005 12-MDc.html
71 mentation scenarios are not considered, for ex- 3.2 Two Examples ample the preservation and re-use of descriptive and other metadata which might have been sup- 3.2.1. The TEXTE Project Digital Resources plied with the digital object which is of special TEXTE is an ongoing project coordinated by importance in the cultural heritage domain. Such The Moore Institute in the National University of metadata could be of help in generating preserva- Ireland, Galway. It combines four tasks which tion metadata, or could be later used for search. aim to illustrate how the current computer- Having all these in mind, preservation meta- enhanced editing tools can be used to produce data area provides many challenges. What pres- various types of high-quality scholarly editions ervation metadata to use? What minimum set (mediaeval manuscripts; correspondence; a set of needs to be supplied in order to guarantee a reli- journal issues and ephemera). The materials are able preservation process? How to automate the digitised and then transcribed and encoded using creation of preservation metadata? How to guar- the Guidelines of the Text Encoding Initiative 16 . antee that the digital resources developed within The resulting resources are highly annotated a particular project are accompanied by sufficient scholarly editions, a combination of the original preservation quality metadata? And how to guar- texts with the scholarly findings. antee interoperability between multiple existing All these resources are built using the same schemes? It is not easy for any organisation or guidelines for text encoding. This common project to make decisions regarding the metadata framework facilitates the development of the in this situation. digital objects especially with regard to their on- In the last years there were several metadata line publication and visualisation and to the an- schemes developed dedicated to long-term pres- notation of resources from the same historical ervation. In the domain of cultural heritage insti- and cultural background. tutions these new kind of metadata have to be From digital preservation point of view, there combined with existing descriptive metadata are several issues which need to be taken into based on the traditionally used catalogue sys- account: tems. Amongst those we are The Metadata En- The TEI P5: Guidelines for Electronic Text coding and Transmission Standard 13 (METS), a Encoding and Interchange 17 are not suggesting container format for metadata and content files metadata elements to be used especially for pres- maintained by the Library of Congress (USA); ervation purposes. It is essential to have well- and LMER (Long-term preservation Metadata formed
13 http://www.loc.gov/standards/mets/ 16 Metadata Object Description Schema, MODS, 14 LMER description and LMER schema: http://www.d- http://www.loc.gov/standards/mods/ nb.de/eng/standards/lmer/lmer.htm 17 http://www.tei-c.org/release/doc/tei-p5- 15 http://www.tei-c.org/index.xml doc/en/html/index-toc.html
72 On the other metadata-related issue, the lon- Table 2. Digitised Resources in 2005-08 in IMI-BAS gevity of specific metadata schemes applied in the cultural heritage domain, the decision to use Type of material Size a widespread encoding framework might be seen Old printed books (National Li- 17,000 pages as a guarantee of the future usability of re- brary Ivan Vazov, Plovdiv) sources. This is based on the expectation that the State Archives 8,000 pages TEI will be continuously supported in the future. 24 archival Research Archive of the Acad- units, emy of Sciences 3.2.2. The KT-DigiCult-BG Project Digital 1,000 pages Resources Old periodicals (newspapers) 1,200 pages The KT-DigiCult-BG project funded by the Musical periodicals 1,000 pages Marie Curie programme of FP6 was recently fin- Archive of the Higher Attesta- 8,500 pages tion Commission ished (in 2008) and one of its outcomes was the Archive of IMI-BAS 18,000 pages creation of the Digitisation Centre in the Institute Mathematical heritage (the heri- 2,000 pages of Mathematics and Mathematics at the Bulgar- tage of Prof. Obreshkov) ian Academy of Sciences (IMI-BAS). This cen- Audio archive of Bulgarian dia- 7 hours tre digitised a substantial number of various lects types of objects belonging to the Bulgarian cul- Mathematical periodicals (1905 - 29,000 pages tural and scientific heritage (see Table 2). One of 1991) the issues the project faced was the organisation Old photographs 200 photographs and the long-term preservation of these objects. TOTAL 85,900 units These resources were not deeply annotated and are intended for the most general type of users – 3.3 Two Emerging Scenarios for Digital the citizens. In this case the metadata used in the Preservation of Small Projects’ Outputs mix of archival materials, old printed books, Two scenarios emerge from the analysis of the photographs and periodicals followed various two case studies. encodings and were not encoded like the meta- Scenario 1. Preparation of stand-alone re- data of the TEXTE project using the same sources. Under this scenario, the set of digital Guidelines. resources is prepared as a stand-alone collection This heterogeneity of approaches leads to a and does not necessarily become a part of a digi- different situation in the produced resources. tal repository. Currently the digitisation centre team is planning 1. Such projects need to pay special attention to to place all scientific resources into a repository proper documentation of the decisions taken using DSpace 18 which will enable the use of its on the encoding applied within the project – resources within the DRIVER 19 project reposi- this is not only substantial for sustainability, tory. but also for re-use of resources in the future Preservation-wise, these resources will be as demonstrated by Claire Warwick et al. transformed according to this larger initiative (2009). Documentation which refers to a requirements and the relevant SIP will be built. general standard like TEI or XML is not suf- In this scenario the preservation task with regard ficient because it does not guarantee future to the digitised documents prepared for access in understanding and interoperability. PDF format stays with the repository while the 2. A set of preservation metadata needs to be digitisation centre will take care for the physical supplied for every single digital object. This copying of the master files which are stored in is currently not a trivial requirement because TIFF format. This is a mixed approach where there is no common view on the structure of various bodies take care for the preservation of preservation metadata. Again, the internal clusters of digital resources. project decisions need to be well- documented. 3. The overall responsibility to document sig- nificant properties of the digital objects which will allow checking authenticity and chain of custody stays with the project; it is
18 unlikely to expect that anyone in the future http://www.dspace.org/ will be able to fill in such gaps if they are 19 DRIVER: Networking European Scientific Repositories, http://www.driver-repository.eu/. present.
73 Scenario 2. Preparation of resources which will needs to plan for make sure what be integrated into a larger digital repository. these otherwise are the digital The following key considerations should be the danger is that repositories poli- taken into account: the collection as cies on physical 1. It is essential to be familiar with the proc- a whole (or sepa- copying and for- rate objects) will mat migration. esses of ingest of digital objects into the re- be lost because of pository. media decay. 2. The structure of the SIPs for ingest should be OAIS im- It is unlikely that The repository discussed in advance. plementa- small projects hosting the pro- 3. The significant properties of digital objects tion will implement ject outputs may need to be discussed with the digital reposi- the complete set implement a spe- tory; it should guarantee to retain authentic- of OAIS func- cific subset of ity and chain of custody related to the in- tional entities. OAIS functional gested objects. entities; this The following Analysis Matrix (see Table 3) should be dis- summarizes what issues needs to be taken into cussed in advance as a sustainability account in these scenarios. guarantee. Possible NLP can be ap- An area of active Table 3. Summary of digital-preservation issues re- application plied for auto- research is the flected in the two suggested scenarios of natural mated generation application of Issue Scenario 1. Scenario 2. (re- language of metadata, in- NLP for auto- (stand-alone re- sources to be processing cluding preserva- mated extraction sources) integrated into a (NLP) tech- tion metadata; we of metadata from digital repository) nology do not have evi- digital objects Digital ob- The digital object The digital object dence of such during ingest. ject is prepared com- is likely to be attempts so far. This will help to pletely and solely enriched in order overcome the within the pro- to be ingested metadata bottle- ject. into a repository. neck. This enrichment We hope that this succinct analysis will help can be done by various parties projects to understand better the preservation- according to the related options they have. adopted proce- dures for ingest 4 Conclusions into the reposi- Our presentation of digital preservation issues tory. Significant Small projects The values of the and the specifics of cultural heritage domain pin- properties usually do not significant prop- points several issues which are essential for fu- of the digi- consider signifi- erties which ture implementations: tal object cant properties guarantee authen- 1. The understanding of digital preservation but they are es- ticity, integrity standards and models is essential to organise a sential as future and chain of cus- proper lifecycle management for the cultural evidence of au- tody need to be heritage resources; there are examples of solu- thenticity, integ- supplied jointly tions applied in the large memory institutions 20 rity and chain of with the digital but smaller projects and initiatives need to ad- custody. object when it is dress these issues when they develop their sus- being ingested into the digital tainability plans. repository. 2. Widely used metadata standards in the cultural Preservation The preservation The preservation heritage domain, such as TEI, could offer bet- actions actions e.g. copy- actions are ap- ter guidance on metadata elements subset de- ing of media or plied within the signed especially to support long-term preser- migration to repository. The vation. TEI provides mechanisms for follow- newer file for- creators of the mats will be ap- collection do not plied within the have to plan for 20 The German National Library and the Royal Library in host institution; it this but need to the Netherlands are examples of institutions which apply successfully digital preservation solutions.
74 ing any changes in the digital object; it also http://www.dpconline.org/docs/reports/dpctw05- could be used to encode information related to 01.pdf other copies made such as master or preserva- Claire Warwick, Isabel Galina, Jon Rimmer, Melissa tion copies and access or use copies; and pos- Terras, Ann Blandford, Jeremy Gow, George Bu- sibly to record any processes applied to a re- chanan (2009) Documentation and the Users of source, but this is not sufficient because it re- Digital Resources in the Humanities. Journal of flects the process of creation of an object but Documentation, Vol. 65 (1), pp. 33-57. this is not sufficient to claim authenticity of Daisy Abbott. 2008. JISC Final Report – Digital Re- this object. Currently there is no mechanism to positories and Archives Inventory Project, HATII, check the integrity and the chain of custody of University of Glasgow, July 2008. digital objects. However, TEI guidelines DPE. 2007. Digital Preservation Europe (DPE) should suggest how correctly to represent (2007). Access to and preservation of cultural and these changes. For example Linda Cantara in scientific resources: research roadmap. 2005 suggested to form an AIP for the re- http://www.digitalpreservationeurope.eu/publicatio sources of the Tibet Oral History Archive Pro- ns/reports/dpe_research_roadmap_D72.pdf ject especially for digital preservation pur- Giorgos Flouris and Carlo Meghini. 2007. Some Pre- poses; more examples are needed in order to liminary Ideas Towards a Theory of Digital Preser- establish a good practices which could be in- vation, 2007, Proceedings of the 1st International corporated in other cases. Workshop on Digital Libraries Foundations 3. OAIS does not suggest any specific metadata (DLF1-07). Available: as instantiations of preservation description in- http://www.ite.gr/ics/isl/publications/paperlink/DL formation and in particular in the cultural heri- F107.pdf tage domain there is no single and uniformly James Cheney, Carl Lagoze and Peter Botticelli. accepted set of elements which guarantee the 2001. Towards a Theory of Information Preserva- longevity of resources; therefore it is neces- tion . In: Constantopoulos & I.T. Sølvberg (Eds.): sary to define what metadata need to be stored ECDL 2001, LNCS 2163: 340–351, Springer- for preservation purposes. We can not offer a Verlag Berlin Heidelberg. general solution to this problem because it re- Linda Cantara. 2005. The Tibet Oral History Archive quires consensus of the communities of pro- Project and Digital Preservation. ACH-ALLC fessionals. 2005 conference. Available: We also have presented two scenarios on digi- http://mustard.tapor.uvic.ca/cocoon/ach_abstracts/x tal preservation derived from case studies on two q/xhtml.xq?id=141 EC-funded projects. This work shows that there OAIS. 2003. ISO 14721:2003 (Space data and infor- is definitely a current need to suggest good prac- mation transfer systems – Open archival informa- tices for such endeavours, because while the pro- tion system – Reference model). Available: jects tackle the preservation issue, it is very diffi- http://public.ccsds.org/publications/archive/650x0b cult to find individual good models to guarantee 1.pdf the longevity of resources. Paul Watry. (2007). Digital Preservation Theory and Application: Transcontinental Persistent Archives Reference Testbed Activity. International Journal of Digital Angela Dappert and Markus Enders. 2008. Using Curation 2(2). Available: METS, PREMIS and MODS for Archiving eJour- http://www.ijdc.net/ijdc/article/view/43/0 nals. In: D-Lib Magazine September/October 2008, PREMIS. (2008) Data Dictionary for Preservation Vol. 14, Number 9/10, ISSN 1082-9873 Metadata: PREMIS version 2.0, Available: Available: http://www.loc.gov/standards/premis/v2/premis-2- http://www.dlib.org/dlib/september08/dappert/09da 0.pdf. ppert.html Reagan Moore and MacKenzie Smith. (2007). Auto- Blue Ribbon. 2008. The Blue Ribbon Task Force on mated Validation of Trusted Digital Repository Sustainable Digital Preservation and Access Assessment Criteria. Journal of Digital Information (2008). Interim Report. Available: 8(2). Available: http://brtf.sdsc.edu/biblio/BRTF_Interim_Report.p http://journals.tdl.org/jodi/article/view/198/181. df Brian Lavoie and Richard Gartner. 2005. Preservation metadata, DPC Technology Watch Series Report 05-01. Available:
75
Author Index
Bilidas, Dimitris, 10
Dobreva, Milena, 68
Giouli, Voula, 35 Glaros, Nikos, 35 Goerz, Guenther, 1
Hunt, Steve, 59
Ikonomov, Nikola, 68
Karkaletsis, Vangelis, 10 Konstantopoulos, Stasinos, 10
Lewis, William, 50
McGillivray, Barbara, 42
Nerbonne, John, 18, 26
Osenova, Petya, 35
Passarotti, Marco, 42 Prokic,´ Jelena, 18, 26
Scholz, Martin, 1 Simov, Kiril, 35 van den Bosch, Antal, 59 van Erp, Marieke, 59
Wieling, Martijn, 18, 26 Wubben, Sander, 59
Xia, Fei, 50
77