DigHum 2011
Proceedings of the Workshop on Language Technologies for Digital Humanities and Cultural Heritage
associated with The 8th International Conference on Recent Advances in Natural Language Processing (RANLP 2011)
Edited by Cristina Vertan, Milena Slavcheva, Petya Osenova and Stelios Piperidis
16 September, 2011 Hissar, Bulgaria INTERNATIONAL WORKSHOP LANGUAGE TECHNOLOGIES FOR DIGITAL HUMANITIES AND CULTURAL HERITAGE
PROCEEDINGS
Hissar, Bulgaria 16 September 2011 ISBN 978-954-452-019-9 Designed and Printed by INCOMA Ltd. Shoumen, BULGARIA
ii Foreword
Following several digitization campaigns during the last years, a large number of printed books, manuscripts and archaeological digital objects have become available through web portals and associated infrastructures to a broader public. These infrastructures enable not only virtual research and easier access to materials independent of their physical place, but also play a major role in the long term preservation and exploration.
However, the access to digital materials opens new possibilities of textual research like: synchronous browsing of several materials, extraction of relevant passages for a certain event from different sources, rapid search through thousand pages, categorisation of sources, multilingual retrieval and support, etc.
Methods from Language Technology are therefore highly required in order to ensure extraction of content related semantic metadata, and analysis of textual materials. There are several initiatives in Europe aiming to foster the application of language technology in the humanities (CLARIN, DARIAH). Through initiatives like those, as well as many other research projects, the awareness of such methods for the humanities has risen considerably. However, there is still enough potential on both sides:
on one hand, there are still research tracks in the humanities which do not sufficiently and effectively • exploit language technology solutions;
on the other hand, there are many languages, especially historical variants of languages, for which • the available tools and resources still have to be developed or adapted to serve successfully humanities applications.
The current workshop brings together researchers from the Humanities, as well as from Language and Information Technologies, and thus fosters the above mentioned directions.
As a confirmation of the generated interest in the topic of our workshop, we received a large number of very good submissions. This fact allowed us to provide a programme covering the most important aspects within the area of digital humanities and cultural heritage. Following the workshop programme, the Proceedings of the workshop are thematically structured as follows: Electronic Archives, Language Technology and Resources, Computational Methods for Literary Analysis, Multimodal Aspects in Digital Humanities.
The workshop papers address a multitude of problems and suggest a wealth of developments and solutions related to the digital humanities and the preservation of cultural heritage. The papers represent a whole spectrum of relevant topics: utilizing interlinked semantic technologies for managing and accessing museum data; exploiting topic models in a query classification system for an art image archive; metadata and content-oriented search methods for a multilingual audio-and-video archive; maintaining a digital library of Polish and Poland-related old ephemeral prints; normalization of historical wordforms in German; developing a Bulgarian-Polish on-line dictionary as a technological tool for applications in the digital humanities; semantic annotation models based on ontological representation of knowledge concerning Bulgarian iconography; preparation of an electronic edition of the largest Old Church Slavonic manuscript, the Codex Suprasliensis; literary research support by creating and visualizing profiles of sentimental content in texts; profiling of literary characters in 19th century Swedish prose fiction by interpersonal relation extraction; investigation of diachronic stylistic changes in British and American varieties of 20th century written English language; speeding up the process of creating annotations of audio-visual data for humanities research; automatic transcription of ancient handwritten documents; OCR processing of Gothic-script documents.
We would like to thank the Organisers of the RANLP events, especially Galia Angelova and Kiril Simov, for their unceasing help in the organisation of the workshop.
iii We are indebted to the Programme Committee members who provided very detailed reviews in extremely short time.
Special thanks are addressed to Gabor´ Prosz´ eky,´ who accepted to be our keynote speaker and additionally raised the interest in our workshop.
September 2011
Cristina Vertan, Milena Slavcheva, Petya Osenova and Stelios Piperidis
iv Workshop Organizers:
Cristina Vertan (University of Hamburg, Germany) Milena Slavcheva (Bulgarian Academy of Sciences, Bulgaria) Petya Osenova (Sofia University and Bulgarian Academy of Sciences, Bulgaria) Stelios Piperidis (ILSP, Greece)
Programme Committee: Galia Angelova (Bulgarian Academy of Sciences, Bulgaria) David Baumann (Perseus, Tufts University, USA) Nuria´ Bel (University of Barcelona, Spain) Antonio´ Branco (University of Lisbon, Portugal) Nicoletta Calzolari (University of Pisa, Italy) Gunther¨ Gorz¨ (University of Erlangen, Germany) Walther v. Hahn (University of Hamburg, Germany) Fotis Jannidis (University of Wurzburg,¨ Germany) Steven Krauwer (University of Utrecht, the Netherlands) Eric´ Laporte (Universite´ Paris-Est, Marne-la-Vallee,´ France) Anke Ludeling¨ (Humboldt University, Berlin, Germany) Adam Przepiorkowski´ (Polish Academy of Sciences, Poland) Gabor´ Prosz´ eky´ (MorphoLogic, Hungary) Laurent Romary (LORIA-INRIA, Nancy, France) Manfred Thaler (Cologne University, Germany) Tamas´ Varadi´ (Hungarian Academy of Sciences, Hungary) Martin Wynne (University of Oxford, UK)
Invited Speaker: Gabor´ Prosz´ eky´ (MorphoLogic & Pazm´ any´ University, Budapest, Hungary) Endangered Uralic Languages and Language Technologies
v
Table of Contents
Invited Talk
Endangered Uralic Languages and Language Technologies Gabor´ Prosz´ eky...... ´ 1
Electronic Archives
A Framework for Improved Access to Museum Databases in the Semantic Web Dana Dannells,´ Mariana Damova, Ramona Enache and Milen Chechev...... 3
Query classification via Topic Models for an art image archive Dieu-Thu Le, Raffaella Bernardi and Ed Vald...... 11
Unlocking Language Archives Using Search Herman Stehouwer and Eric Auer ...... 19
Digital Library of Poland-related Old Ephemeral Prints: Preserving Multilingual Cultural Heritage Maciej Ogrodniczuk and Włodzimierz Gruszczynski...... ´ 27
Language Technology and Resources
Rule-Based Normalization of Historical Texts Marcel Bollmann, Florian Petran and Stefanie Dipper ...... 34
Survey on Current State of Bulgarian-Polish Online Dictionary Ludmila Dimitrova, Ralitsa Dutsova and Rumiana Panova ...... 43
Language Technology Support for Semantic Annotation of Icono-graphic Descriptions Kamenka Staykova, Gennady Agre, Kiril Simov and Petya Osenova ...... 51
The Tenth-Century Cyrillic Manuscript Codex Suprasliensis: the creation of an electronic corpus. UNESCO project (2010–2011) Hanne Martine Eckhoff, David Birnbaum, Anissava Miltenova and Tsvetana Dimitrova...... 57
Computational Methods in Literary Analysis
SentiProfiler: Creating Comparable Visual Profiles of Sentimental Content in Texts Tuomo Kakkonen and Gordana Galic Kakkonen ...... 62
Character Profiling in 19th Century Fiction Dimitrios Kokkinakis and Mats Malm ...... 70
vii Diachronic Stylistic Changes in British and American Varieties of 20th Century Written English Lan- guage Sanja Stajnerˇ and Ruslan Mitkov ...... 78
Multimodal Aspects in Digital Humanites
AVATecH: Audio/Video Technology for Humanities Research Sebastian Tschopel,¨ Daniel Schneider, Rolf Bardeli, Oliver Schreer, Stefano Masneri, Peter Witten- burg, Han Sloetjes, Przemek Lenkiewicz and Eric Auer ...... 86
Handwritten Text Recognition for Historical Documents Veronica Romero, Nicolas Serrano, Alejandro H. Toselli, Joan Andreu Sanchez and Enrique Vidal ...... 90
Reducing OCR Errors in Gothic-Script Documents Lenz Furrer and Martin Volk...... 97
viii Workshop Programme
Friday, 16 September 2011
9:00–9:15 Opening
9:15–10:15 Endangered Uralic Languages and Language Technologies Gabor´ Prosz´ eky´
10:15–10:45 Coffee Break
Electronic Archives
10:45–11:10 A Framework for Improved Access to Museum Databases in the Semantic Web Dana Dannells,´ Mariana Damova, Ramona Enache and Milen Chechev
11:10–11:35 Query classification via Topic Models for an art image archive Dieu-Thu Le, Raffaella Bernardi and Ed Vald
11:35–12:00 Unlocking Language Archives Using Search Herman Stehouwer and Eric Auer
12:00–12:25 Digital Library of Poland-related Old Ephemeral Prints: Preserving Multilingual Cultural Heritage Maciej Ogrodniczuk and Włodzimierz Gruszczynski´
12:25–14:00 Lunch
Language Technology and Resources
14:00–14:25 Rule-Based Normalization of Historical Texts Marcel Bollmann, Florian Petran and Stefanie Dipper
14:25–14:50 Survey on Current State of Bulgarian-Polish Online Dictionary Ludmila Dimitrova, Ralitsa Dutsova and Rumiana Panova
14:50–15:15 Language Technology Support for Semantic Annotation of Icono-graphic Descrip- tions Kamenka Staykova, Gennady Agre, Kiril Simov and Petya Osenova
ix Friday, 16 September 2011 (continued)
15:15–15:40 The Tenth-Century Cyrillic Manuscript Codex Suprasliensis: the creation of an electronic corpus. UNESCO project (2010–2011) Hanne Martine Eckhoff, David Birnbaum, Anissava Miltenova and Tsvetana Dimitrova
15:40–16:10 Coffee Break
Digital Methods in Literary Analysis
16:10–16:35 SentiProfiler: Creating Comparable Visual Profiles of Sentimental Content in Texts Tuomo Kakkonen and Gordana Galic Kakkonen
16:35–17:00 Character Profiling in 19th Century Fiction Dimitrios Kokkinakis and Mats Malm
17:00–17:25 Diachronic Stylistic Changes in British and American Varieties of 20th Century Written English Language Sanja Stajnerˇ and Ruslan Mitkov
17:25–17:40 Short break
Multimodal Aspects in Digital Humanites
17:40–18:05 AVATecH: Audio/Video Technology for Humanities Research Sebastian Tschopel,¨ Daniel Schneider, Rolf Bardeli, Oliver Schreer, Stefano Masneri, Pe- ter Wittenburg, Han Sloetjes, Przemek Lenkiewicz and Eric Auer
18:05–18:30 Handwritten Text Recognition for Historical Documents Veronica Romero, Nicolas Serrano, Alejandro H. Toselli, Joan Andreu Sanchez and En- rique Vidal
18:30–18:55 Reducing OCR Errors in Gothic-Script Documents Lenz Furrer and Martin Volk
18:55–19:15 Conclusions and closing
x Endangered Uralic Languages and Language Technologies
Gábor Prószéky MorphoLogic & Pázmány University, Budapest, Hungary [email protected]
Language tools and resources for analysis of cialized in Finno-Ugric linguistics and a Hun- less-elaborated languages are in the focus of our garian language technology company (Morpho- workshop. There are still research tracks which Logic) have initiated a project with the goal of still do not sufficiently and effectively exploit producing annotated electronic corpora and com- language technology solutions, and there are putational morphological tools for small Uralic many languages for which the available tools and languages, like Mordvin, Udmurt (Votyak), resources still have to be developed to serve as a Komi (Zyryan), Mansi (Vogul), Khanty basis of further applications. (Ostyak), Nenets (Yurak) and Nganasan (Tavgi). The presentation introduces a set of morpholo- Altogether around a dozen Uralic languages gical tools for small and endangered Uralic lan- totaling some 3.3 million live as scattered minor- guages. Various Hungarian research groups spe- ities in Russia, as shown by the map below:
The morphologies of these languages are com- speed Unification MORphology') developed at plex enough, thus the implementation of the mor- MorphoLogic, which was first successfully ap- phological tools was a real challenge. The sub- plied to another Finno-Ugric language, Hungari- projects concerning the individual languages an. We supplemented the analyzer with two addi- slightly differed, depending on the special prob- tional tools: a lemmatizer and a morphological lems these languages raise (how precisely the generator. Creating analyzers for the Samoyed languages have been described so far, whether languages involved in the project turned out to be there is a standard dialect, what kinds of texts are a great challenge. Nganasan from the Northern available, etc.). In the project, we used the mor- Samoyed branch is a language on the verge of phological analyzer engine called Humor ('High extinction (the number of native speakers is be-
1 Proceedings of Language Technologies for Digital Humanities and Cultural Heritage Workshop, pages 1–2, Hissar, Bulgaria, 16 September 2011. low 500 by now, most of them are middle-aged or old), so its documentation is an urgent sci- entific task. Nganasan morphology and espe- cially its phonology is very complex and the available linguistic data and their linguistic de- scriptions proved to be incomplete and partly contradictory. Thus, using the Humor formalism, which we successfully applied to other languages involved in the project, was not to be feasible in the case of one of the chosen languages, Nganas- an. The Humor formalism uses an 'item-and-ar- rangement' model of morphology where feature- based allomorph adjacency restrictions are the primary device for constraining word structure. Gradation in Nganasan is difficult to formalize as a set of allomorph adjacency restrictions because the segments involved in determining the out- come of the process may belong to non-adjacent morphemes. For Nganasan, we used therefore another tool (xfst of Xerox), mainly because gradation is just a small part of the complicated system of dozens of interacting productive and lexicalized morpho-phonological and phonolo- gical alternations. Besides the annotated corpora and the mor- phological analyzers, a website was also de- veloped where all of the tools described above are available for a wider public.
Acknowledgments
The projects have been funded by the Hungarian Scientific Research Fund (OTKA) and the Na- tional Research and Development Programme (NKFP): A Complex Uralic linguistic database (NKFP-5/135/01), Linguistic databases for Permic languages (OTKA T 048309), Develop- ment of a morphological analyzer for Nganasan (OTKA K 60807), Ob Ugric morphological ana- lyzers and corpora (OTKA 71707).
2 A Framework for Improved Access to Museum Databases in the Semantic Web Dana Dannells´ Mariana Damova Department of Swedish Language Ontotext University of Gothenburg Sofia 1784, Bulgaria SE-405 30 Gothenburg, Sweden [email protected] [email protected]
Ramona Enache Milen Chechev Department of Computer Science and Engineering Ontotext GU and Chalmers University of Technology Sofia 1784, Bulgaria SE-412 96 Gothenburg, Sweden [email protected] [email protected]
Abstract stored in internal museum databases requires ad- vanced mapping and vocabulary integration for it Digital museum databases have extremely to be accessible for Semantic Web applications. heterogeneous data structures which re- In addition to establishing ways for managing quire advanced mapping and vocabulary various vocabularies, and for exploiting semantic integration for them to benefit from the alignments across them automatically (van der interoperability enabled by semantic tech- Meij et al., 2010), computer engineers also need nologies. In addition to establishing ways to investigate automatic methods to make this in- of extracting and manipulating digitally formation available to computer users in different encoded cultural material, there exists a forms and languages that are available to them. need to make this material available and Our work is a step towards this direction. It is accessible to human users in different about an automatic workflow of sharing data in- forms and languages that are available to frastructures that is explicitly targeted towards the them. In this paper we describe a method Semantic Web. We have developed a method to to manage and access museum data by in- manage and access museum data by integrating tegrating it within a series of interlinked it within a series of interlinked ontological mod- ontological models. The method allows els. The method allows querying and generation of querying and generation of query results in query results in natural language using the Gram- natural language. We report on the results matical Framework (GF). We have been experi- of applying this method from experiments menting with data collections from the Gothen- we have been pursuing. burg City Museum that we made available for querying in the Museum Reason-able View loaded 1 Introduction in the triple store OWLIM. In the remainder of this paper we present the During the past few years several projects have ontologies that were merged including CIDOC- been undertaken to digitize cultural heritage ma- CRM,2 PROTON,3, the Painting ontology and the terials (Clough et al., 2008; Dekkers et al., 2009) data that we have been experimenting with (Sec- through the use of Semantic Technologies such tion 2). We describe the creation of the Museum as RDF (Brickley and Guha, 2004) and OWL Reason-able View with structured query examples (Berners-Lee, 2004). Today there exist large num- (Section 3). In Section 4, we introduce the Gram- ber of digital collections and applications provid- matical Framework and demonstrate the mecha- ing direct access to cultural heritage content.1 nisms of interfacing between the structured data However, digitization is a labour intensive pro- and natural language. We provide an overview of cess and is long from being complete. Because related work (Section 5) and end with conclusions. of the heterogeneous data structures different mu- seums have, digitally encoded cultural material 2The Conceptual Reference Model (CRM): http:// cidoc.ics.forth.gr/ 1http://www.europeana.eu/portal/ 3http://proton.semanticweb.org/
3 Proceedings of Language Technologies for Digital Humanities and Cultural Heritage Workshop, pages 3–10, Hissar, Bulgaria, 16 September 2011. 2 The Ontologies and Museum Data mentation of the CRM.5 The additional models that are correctly integrated in the ontology are: 2.1 The CIDOC-CRM SOCH, Time Ontology,6 SUMO and Mid-Level- The International Committee for Documentation Ontology.7 The painting ontology was constructed Conceptual Reference Model (CIDOC CRM) that manually using the Proteg´ e´ editing tool.8 was accepted by ISO in 2006 as ISO21127 (Crofts Integration of the ontology concepts are accom- et al., 2008), is one of a widely used standards plished by using the OWL construct: intersec- that has been developed to facilitate the integra- tionOf as specified below: tion, mediation and interchange of heterogeneous
4 the data integration model to provide access to the describes more than 3.5 million things and cov- Linked Open Data (LOD) for Cultural Heritage ers 97 languages. 1.67 million of DBpedia things (Damova and Dannells,´ 2011). are classified in a consistent ontology, including 364,000 persons, 462,000 places, and 99,000 mu- 2.5 The Gothenburg City Museum Database sic albums. The DBpedia knowledge base has over The Gothenburg City Museum (GCM) preserves 672 million RDF triples out of which 286 million 8900 museum objects described in two of the mu- extracted from the English edition of Wikipedia seum database tables. These two tables corre- and 386 million extracted from other language edi- spond to two of the museum collections, i.e. GSM tions. and GIM. Each of these tables contains 39 prop- DBpedia is used as an additional source of erties for describing museum objects. Table 1 data, which can enrich the information about the shows 20 of these properties, including the object Gothenburg museum data. For example, their lo- type, its material, measurements, location, etc. All cation identified with the DBpedia resource refer- properties and object values stored in the database ring to the city of Gothenburg. are given in Swedish. 3 Integrating and Accessing Museum Field name Value Data Field nr. 4063 Prefix GIM Object nr. 8364 3.1 Integration for flexible computing Search word painting Class 1 353532 Class 2 Gothenburg portrait Integrating datasets into linked data in RDF usu- Amount 1 Producer E.Glud ally takes place by indicating that two instances Produced year 1984 Length cm 106 from two datasets are the same by using the built Width cm 78 owl:sameAs 12 Description oilpainting represents a studio indoors in OWL predicate: . However, History Up to 1986 belonged to Datema AB, Flojelbergsg¨ 8, Gbg recent research (Damova, 2011; Damova et al., Material oil colour Current keeper 2 2011; Jain et al., 2011) has shown that interlinking Location Polstjarnegatan¨ 4 Package nr. 299 the models according to which the datasets are de- Registration date 19930831 Signature BI scribed is a more powerful mechanism of dealing Search field BO:BU Bilder:TAVLOR PICT:GIM with large amounts of data in RDF, as it exploits Table 1: A painting object representation in the inference and class assignment. GCM database. We have adopted this approach when creating the infrastructure for the museum linked data, in- The Gothenburg City Museum’s data that is cluding several layers of upper-level ontologies. used as our experimental data follows the struc- They provide a connection to different sets of ture of the CIDOC-CRM but it contains many con- linked data, for example PROTON for the LOD cepts that are not available in CIDOC-CRM. So, in cloud. They also provide an extended pool of con- order to be able to fully integrate the Gothenburg cepts that can be referred to in museum linked data City Museum data into a semantic view it was nec- that do not directly pertain to the expert descrip- essary to make use of concepts and relationships tions of the museum objects, and the strictly ex- from the remaining ontologies. pert museum knowledge is left to CIDOC-CRM. Figure 1 shows how elements from the This model of interlinked ontologies offers a flex- Goethenburg city museum are represented with ible access to the data with different conceptual elements from different schemata, e.g. CIDOC- access points. This approach is implemented as a CRM, PROTON, SOCH and the Painting ontol- Reason-able View of the web of data (Kiryakov et ogy. al., 2009).
2.6 DBpedia 3.2 The Museum Reason-able View DBpedia (Auer et al., 2007) is the RDF-ized ver- Using linked data techniques (Berners-Lee, 2006) sion of Wikipedia, comprising the information for data management is considered to have great from Wikipedia infoboxes, designed and devel- potential in view of the transformation of the web oped to provide as full as possible coverage of of data into a giant global graph. Still there are the factual knowledge that can be extracted from challenges related to them that have to be handled Wikipedia with a high level of precision. DBpedia 12http://www.w3.org/TR/owl-ref/
5 Figure 1: Dataset interconnectedness in the Museum Reason-able View. to make this possible. Kiryakov et al. (2009) dis- Reason-able View, which includes interconnected cuss these challenges and present an approach for schemata and links to external to the Gothenburg reasoning with and management of linked data. In museum data, such as DBpedia. The knowledge summary, a Reason-able View is an assembly of base contains close to 10K museum artifacts from independent datasets, which can be used as a sin- the Gothenburg city museum, and the entire DB- gle body of knowledge with respect to reasoning pedia. and query evaluation. Each Reason-able View is aiming at lowering the cost and the risks of us- 3.3 Accessing Museum Linked Data ing specific linked datasets for specific purposes. The Museum Reason-able View is loaded in We followed this approach when constructing the OWLIM (Bishop et al., 2011) and its data are ac- Museum Reason-able View with the data from the cessible via a SPARQL (Eric and Andy, 2008) end Gothenburg City Museum, DBpedia, Geonames point and keywords.14 The queries can be for- and the ontologies listed in Section 2.13 mulated by combining predicates from different datasets and ontologies in a single SPARQL query, retrieving results from all different datasets that are part of the Reason-able View. A query example about the location, address, description and time of paintings by Carl Larsson is given below.
PREFIX crm:
6 Figure 3: The results from a SPARQL query.
type theory (Martin-Lof,¨ 1982). Its key feature is The above query returns the results that are de- the division of a grammar in the abstract syntax- picted in Figure 3. Note that the returned location which acts as a semantic interlingua and the con- is the DBpedia resource about the city of Gothen- crete syntaxes-representing verbalizations in vari- burg. The results also show that museum items ous target languages (natural or formal). from the two collections – GIM and GSM – are GF comes with a resource library (Ranta, 2009), harvested, which means that the data from the where the abstract syntax describes the most com- collections are integrated together and accessible mon grammatical constructions allowing text gen- from a single query point. eration, which are further mapped to concrete syn- 15 Other queries can be asked about the types of art taxes corresponding to 18 languages. The re- work preserved in the museum, their material, or source library aids the development of new gram- about artwork from a certain period of time, etc. mars for specific domains by providing the op- Below follows another query example about the erations for basic grammatical constructions, and address, the time of paintings and the collection thus making it possible for users without linguistic they are coming from. background to generate syntactically correct natu- ral language. select ?museumObject ?location ?collection ?address ?description ?time where To verbalize the data that is stored in the Mu- { ?museumObject crm:P55_has_current_location ?location ; seum Reason-able View, we utilize GF. The ad- ptop:partOf [ rdfs:label ?collection ] ; painting:hasCategory [ rdfs:label "teckning"@sv ] ; vantages of using GF for verbalization is three crm:P62_depicts ?description . OPTIONAL {?museumObject pext:hasAddress fold: it provides mechanisms for type checking, by [ rdfs:label ?address ] .} OPTIONAL {?museumObject crm:P92_brought_into_existence validating coercions between the basic class of an [ crm:P4_has_time-span [ rdfs:label ?time ] ] .} instance and the class required by the definition of } the relation that uses it; the framework offers sup- The Reason-able View is accessible with port of direct verbalization which makes it easier SPARQL queries, which require intimate knowl- to generate text from the ontology and so to create edge of the schemata describing the data, and tech- natural language applications using it without the nical expertise in SPARQL. Moreover, the results aid of external tools; GF has a resource library that from SPARQL are not always easy to understand, cover the syntax for 18 languages. in particular if the retrieved information is given 4.2 Translation of the Museum Reason-able in a language other than English. This is why the View to GF results are send forward to the NLP component to verbalize the ontology links. The capabilities of GF as a host-language for ontologies were already investigated in Enache 4 Ontologies Verbalization and Angelov (2010), where SUMO, the largest open-source ontology was translated to GF. It 4.1 The Grammatical Framework (GF) was shown that the type system provides a robust The Grammatical Framework GF (Ranta, 2004) is a grammar formalism, based on Martin-Lof’s¨ 15www.grammaticalframework.com
7 framework for encoding classes, instances and re- Below follow a few English sentence examples lations. The same basic implementation design that we are able to generate: that was used for encoding SUMO in GF is applied Big Garden is a painting in this work for representing the Museum Reason- • able View. Big Garden is painted on canvas The classes form a hierarchy modelled by • an inheritance relation, which is the reflexive- Big Garden is painted by Carl Larsson • transitive closure of the subclass relation Big Garden was created in 1937 rdfs:subClassOf from the ontology, are • encoded as functions in the GF grammar. Other Below we provide examples for ontology rela- information stated in the ontology, is encoded in tions in the shape of O1 is painted by O2 and feed GF as axioms, external to the grammar. These are these to the GF parser which will build an abstract used for verbalization as in the following example syntax tree, from which we abstract over the place- from the OWL entry corresponding to the painting holders O1 and O2, replacing them with function Big Garden: arguments.
8 To our knowledge, we made the first attempt Equiv_TimePeriod = Equivalent TimePeriod of using CIDOC-CRM to produce museum linked (both E52_TimeSpan Sumo.YearDuration) ; data with connections to external sources like DB- pedia. Our attempt to generate natural language Equivalent Class Class is a dependent type that sentences from ontologies, and more precisely encodes type equivalence. from the structured results of SPARQL queries are the novelty of the work presented in this paper. 5 Related Work
Museum Data Integration with semantic technolo- 6 Conclusions gies as proposed in this paper is intended to enable We presented a framework for integrating and ac- efficient sharing of museum and cultural heritage cessing museum linked data, and a method to information. Initiatives for developing such shar- present this data using natural language generation ing museum data infrastructures have emerged in technology. the recent years. Only a few of them rely on se- A series of upper-level and domain specific mantic technologies. ontologies have been used to transform Gothen- The Museum Data Exchange 2010 project has burg museum data from a relational database into developed a metadata publishing tool to extract RDF and build a Museum Reason-able View. We 16 data in XML. Brugman et al. (2008) have devel- showed how federated results to SPARQL queries oped an Annotation Meta Model providing a way using predicates from multiple ontologies can be of defining annotation values and anchors in an an- obtained. Consequently, we demonstrated how notation for multimedia resources. The difference templates are automatically obtained in GF to gen- between these approaches and our approach is that erate the query results in natural language. we chose to reuse many of the concepts and the re- Future work includes extending the museum lationships that are already defined in the standard data in the Museum Reason-able View, running model CIDOC-CRM. several queries, and increasing the coverage of the Other related initiatives in the Web of structured GF grammar. We intend to have a grammatical data is the Amsterdam Museum Linked Open coverage for at least five languages. Other direc- 17 Data project, aiming at producing Linked Data tions for future work, also include fluent discourse within the Europeana data model (Dekkers et al., generation from the ontology axioms, as well as 2009; Haslhofer and Isaac, 2011), and the Na- paraphrasing of the existing patterns for verbaliza- tional Database Project of Norwegian University tion. Museums (Ore, 2001) who developed a unified in- 18 terface for digitalizing cultural material. Acknowledgments In Sweden, as well as other countries, seman- tic technologies enter the cultural heritage field in- This work is supported by MOLTO Euro- creasingly and there have been some suggestions pean Union Seventh Framework Programme describing the tools and techniques that should (FP7/2007-2013) under grant agreement FP7-ICT- be applied to digitalize the Swedish Union Cat- 247914. alogue (Malmsten, 2008). Following these ideas and experiences from experimenting with museum data (Bryne, 2009) who have shown that con- References version of museum databases is best approached Soren¨ Auer, Christian Bizer, Georgi Kobilarov, Jens through integration of existing models, we decided Lehmann, Richard Cyganiak, and Zachary Ives. to invest in a manual design step to built a frame- 2007. DBpedia: A nucleus for a web of open data. work that captures specific characteristics of mu- In Lecture Notes in Computer Science (LNCS), vol- ume 4825, pages 722–735. seum databases.
16http://www.oclc.org/research/ Tim Berners-Lee. 2004. OWL Web Ontology Lan- activities/museumdata/default.htm guage reference, February. W3C Recommendation. 17http://www.europeana.eu/portal/ thoughtlab_linkedopendata.html T. Berners-Lee. 2006. Design is- 18http://www.muspro.uio.no/engelsk-omM. sues: Linked data. Retrieved from shtml http://www.w3.org/DesignIssues/LinkedData.html.
9 B. Bishop, A. Kiryakov, D. Ognyanoff, I. Peikov, Z. Ta- Prateek Jain, Peter Z. Yeh, Kunal Verma, Reymonrod shev, and R. Velkov. 2011. OWLIM: A family G., Mariana Damova, Vasquez Pascal Hitzler, and of scalable semantic repositories. Semantic Web Amit P. Sheth. 2011. Contextual ontology align- Journal, Special Issue: Real-World Applications of ment of lod with an upper ontology: A case study OWL. with proton. In Proceedings of 8th ESWC, Extended Semantic Web Conference, Heraklion, Greece, May. Dan Brickley and R.V. Guha, 2004. RDF Vocabulary Description Language 1.0: RDF Schema. W3C. A. Kiryakov, D. Ognyanoff, R. Velkov, Z. Tashev, and http://www.w3.org/TR/rdf-schema/. I. Peikov. 2009. LDSR: Materialized reason-able view to the web of linked data. In Proceedings of Hennie Brugman, Veronique´ Malaise,´ and Laura OWL: Experiences and Directions (OWLED) 2009, Hollink. 2008. A common multimedia annotation Chantilly, USA. framework for cross linking cultural heritage digi- tal collections. In International Conference on Lan- Martin Malmsten. 2008. Making a library catalogue guage Resources and Evaluation (LREC). part of the semantic web. In Proceedings of the Intl. Conf. on Dublin Core and Metadata Applications, Kate Bryne. 2009. Putting hybrid cultural data on Berlin, German. the semantic web. Journal of Digital Information (JoDI), 10(6). Eds. Martha Larson, Kate Fernie, Per Martin-Lof.¨ 1982. Constructive mathematics John Oomen. and computer programming. In Cohen, Los, Pfeif- fer, and Podewski, editors, Logic, Methodology and Paul Clough, Jennifer Marlow, and Neil Ireson. 2008. Philosophy of Science VI, pages 153–175. North- Enabling semantic access to cultural heritage: A Holland, Amsterdam. case study of tate online. In Proceedings of the ECDL. Workshop on Information Access to Cultural Christian-Emil Smith Ore. 2001. The norwegian mu- Heritage, ISBN 978-90-813489-1-1, Aarhus, Den- seum project, access to and interconnection between mark, September. various resources of cultural and natural history. In Nick Crofts, Martin Doerr, Tony Gill, Stephen Stead, European Conference on Research and Advanced and Matthew Stiff, 2008. Definition of the CIDOC Technology for Digital Libraries ECDL. Conceptual Reference Model. Aarne Ranta. 2004. Grammatical framework, a type- Mariana Damova and Dana Dannells.´ 2011. Reason- theoretical grammar formalism. Journal of Func- able view of linked data for cultural heritage. In tional Programming, 14(2):145–189. Proceedings of the third International Conference Aarne Ranta. 2009. The GF resource grammar library. on Software, Services and Semantic Technologies Linguistic Issues in Language Technology, 2(2). (S3T). I. Terziev, A. Kiryakov, , and D. Manov, 2005. D.1.8.1 Mariana Damova, Atanas Kiryakov, Maurice Grinberg, Base upper-level ontology (BULO) Guidance. De- Michael K. Bergman, Frederik Giasson, and Kiril liverable of EU-IST Project IST. Simov. 2011. Creation and integration of reference ontologies for efficient lod management. In Semi- Lourens van der Meij, Antoine Isaac, and Claus Zinn. Automatic Ontology Development: Processes and 2010. A web-based repository service for vocab- Resources, IGI Global, Hershey PA, USA. ularies and alignments in the cultural heritage do- Mariana Damova, 2011. Data Models and Alignment, main. In Proceedings of the 7th Extended Semantic May. Deliverable 4.2. MOLTO FP7-ICT-247914. Web Conference, (ESWC 2010). Makx Dekkers, Stefan Gradmann, and Carlo Meghini, 2009. Europeana Outline Functional Specification For development of an operational European Digital Library. Europeana Thematic Network Deliverable 2.5. Ramona Enache and Krasimir Angelov. 2010. Type- ful ontologies with direct multilingual verbalization. Workshop on Controlled Natural Languages (CNL) 2010. Prud’hommeaux Eric and Seaborne Andy. 2008. SPARQL. the query language for RDF, January. W3C Recommendation. Bernhard Haslhofer and Antoine Isaac. 2011. data.europeana.eu the europeana linked open data pilot. In Proceedings of the Intl. Conf. on Dublin Core and Metadata Applications.
10 Query classification via Topic Models for an art image archive
Dieu-Thu Le Raffaella Bernardi Edwin Vald DISI, University of Trento DISI, University of Trento Bridgeman Art Library [email protected] [email protected] [email protected]
Abstract case study. Learning to classify queries in this do- main is particularly challenging due to the specific In recent years, there has been an increas- vocabulary and the small amount of textual infor- ing amount of literature on query classi- mation associated with the images. Examples of fication. Click-through information has user queries taken from BAL logs and categories been shown to be a useful source for im- from BAL taxonomy are: proving this task. However, far too little attention has been paid to queries in very Queries monster woman; messe; ribera crucifix- specific domains such as art, culture and ion; woman perfume, etc. history. We propose an approach that ex- Categories Religion and Belief; People and Sci- ploits topic models built from a domain ence; etc. specific corpus as a mean to enrich both the query and the categories against which Clearly, classifying these queries against these the query need to be classified. We take domain specific categories is a hard challenge and an Art Library as the case study and show standard text classification techniques need to be that topic model enrichment improves over tailored for the specific problem in hands. the enrichment via click-through consider- Following the literature on web search classifi- ably. cation (Cao et al., 2009), we enrich the queries by exploiting the click-through information, which 1 Introduction provides us with titles of the images as well as key- In Information Science, transaction log analyses words assigned by domain experts to the clicked have been carried out since its early stages (Pe- images. Furthermore, we employ unsupervised ters, 1993). Within this tradition, lately, Query Topic Models (Blei et al., 2003) to detect the top- Classification (QC) to detect user web search in- ics of the queries as well as to enrich the target tent has obtained interesting results (Shen et al., taxonomy. The novelty of our work is on the use 2006a; Shen et al., 2006b; Li et al., 2008; Cao of Topic Models for a domain specific application et al., 2009). A QC system is required to auto- and in particular the proposal of using the meta- matically label a large propotion of user queries data itself as a source to train the model. to a given target taxonomy. Successfully mapping We confirm the impact of the click-through in- a user query to target categories brings improve- formation, which increased the number of cor- ments in general web search and online advertis- rect categories found by 120%, and show that for ing. Recently, most studies have focused on the closed domain image archive, Topic Models (TM) mapping of user queries to a general target tax- bring a valuable contribution when built out of a onomy. However, little has been discussed about very domain specific data-set. In particular, we learning to classify queries in a specific domain. In compare the results obtained by TM enrichment this paper, we focus on QC for queries in transac- when the model is built out of (a) Wikipedia pages tion logs of an image archive; we take the Bridge- and (b) the Bridgeman Catalogue itself. The latter man Art Library (BAL)1, one of the world’s top increased the number of correct categories found image libraries for art, culture and history, as our by 117% and resulted in a raise of 18% in F1- measure with respect to the classifier based on 1http://www.bridgemanart.com/ click-through information.
11 Proceedings of Language Technologies for Digital Humanities and Cultural Heritage Workshop, pages 11–18, Hissar, Bulgaria, 16 September 2011. 2 Bridgeman Art Library (BAL)
Taxonomy In Bridgeman Art Library, images are classified with sub-categories from a two- level taxonomy. We use “top-category” and “sub- category” to refer to the first and second level, respectively. The taxonomy contains 289 top- categories and 1,148 sub-categories, with an aver- age of 4 sub-categories for top-category. The ≈ top-categories can be divided into three main groups “topic”, “object” and “material”, we will come back to this with more details in Section 4. A sample of the taxonomy is given in Figure 1. Figure 2: Distribution of images in the Bridgeman metadata among the three groups: topic, material, object
and the corresponding clicked images (i.e., the image that the user clicked after submitting the query). Via this clicked image, queries can be mapped to the information about the image pro- vided in the metadata.
3 Data enrichment via Topic Models Since a query can express different information needs and hence can be associated to different cat- egories, we choose multiple classification and aim to classify a query by assigning to it three top- categories out of the BAL taxonomy. To overcome the distance in the vocabulary be- tween the queries and the categories, we enrich the query with the words from the title, description and keywords associated with the corresponding clicked image, and enrich the top-category with its Figure 1: Taxonomy sub-categories. We represent the enriched queries and enriched categories as vectors built using oc- currence counts as values for these words. Still Catalogue The Catalogue contains 324,232 im- this enrichment does not cover the gap between ages. Their distribution by group of category is the query and the top-categories, hence we ex- as following: “Topic” 79%, “Material” 18% and ploit topic models (TMs) to reduce the distance “Object” 3% (Figure 2). For each image the meta- and capture their semantic similarity. The full en- data contains the title, a description, keywords and richment process is sketched in Figure 3. a sub-category from the taxonomy above, besides other information we are not going to consider in Hidden Topic Models A topic model (Blei et this paper. The keyword field is for free-text terms al., 2003; Griffiths and Steyvers, 2004; Blei and (no controlled vocabulary is used), the terms pro- Lafferty, 2007) is a semantic representation of text vides physical description, aspects of the image, that discovers the abstract topics occurring in a like the color, shape or the object described, dates, collection of documents. Latent Dirichlet Alloca- conceptual terms, etc. An example is given in Ta- tion (LDA), first introduced by (Blei et al., 2003), ble 1. is a type of topic model that performs the so-called latent semantic analysis (LSA). By analyzing sim- Query Logs Query logs contain information ilar patterns of documents and word use, LDA al- about the queries (usually 1 to max. 5 words each) lows representing text on a latent semantic level,
12 Title A Section of the Passaic Class Single-Turret Ironclad Monitor (engraving) Keywords design, battleship, weapon, armoured, boat, submarine, warship, naval, cannon, ship; Description Transverse section of pilot-house and turret; The Passaic class, single- turret monitors of the U.S. Navy were enlarged versions of the original Monitor ships; the first Passaic was commissioned 5 November 1863; Sub-category Sea Battles
Table 1: Meta-data: An example
M: number of documents • Nd: number of words in document d • z: latent topic • w: observed word • θ: distribution of topic in documents • φ: distribution of words generated from topic z • Conversely, given a set of documents, we can discover a set of topics that are responsible for Figure 3: Enriching queries and categories: (1) generating a document, and the distribution of Learning a TM from the universal data-set; (2) En- words that belong to a topic. Estimating these pa- riching queries and categories with their topics rameters for LDA is intractable. Different solu- tions for approximating estimation such as Varia- rather than by lexical occurrence. It has been tional Methods (Blei et al., 2003) and Gibbs Sam- used in many applications such as dimensionality pling (Griffiths and Steyvers, 2004) can be used. reduction (Blei et al., 2003), text categorization, Gibbs Sampling is an example of a Markov chain clustering, collaborative filtering and other tasks Monte Carlo with relatively simple algorithm for for textual documents as well as other kinds of dis- approximate inference in high dimensional mod- crete data. els, with the first use for LDA reported in (Griffiths The underlying idea of LDA is based upon a and Steyvers, 2004). probabilistic procedure of generating new docu- In our experiment, we have estimated the ments: First, each document d in the corpus is multinomial observations by unsupervised learn- Introductiongenerated by samplingLatent Dirichlet a distribution Allocation θ over top-Gibbs Samplinging with GibbsLDA++Short Text Enrichment toolkit. with2 TopicFollowing Models the ics from a Dirichlet distribution (Dir(α)). After data enrichment approach in (Phan et al., 2010), that, the topic assignment for each observed word we have enriched the query and category with hid- w is performed by sampling a topic z from a multi- den topic. In particular, given a probability ϑm,k nomial distribution (Mult(θ)). This process is re- of document m over topic k, the corresponding peated until all T topics have been generated for LDA: Graphical Model weight wtopick,m was determined by discretizing the whole corpus. ϑm,k using two parameters cut-off and scale: α, β:Dirichletpriorround (scale ϑ ) , if × m,k Mwtopic: number,m = ϑ of doccut-off k m,k ≥ 0, if ϑ < cut-off Nd : number of wordsm,k in d (1) Wez: chose latentcut-off topic = 0.01, scale = 20 as to ensure that the number of topics assigned to a w: observed word query/category does not exceed the number of original θ: distribution terms of that query/category, of topic in i.e., to keep a balancedoc weight between topics enriched and orig- inal terms. Toφ discover: distribution the set of topicsof words and the distribution Figure 4: Latent Dirichlet Allocation of wordsgenerated per topic, from we need topic to choosez a universal Using plate notation: data set. Since we are interested in topics within Sampling of distribution over topics2 for each document d α, β: Dirichlet prior http://gibbslda.sourceforge.net/ • Sampling of word distributions for each topic z until T topics have been generated 13 13 / 43 a rather specific domain, we need to choose a data set that provides an appropriate vocabulary. We have tried two options, a Topic Model built out of selected pages of Wikipedia and a Topic Model built out of BAL Catalogue. Wikipedia Topic Model Wikipedia is a rich source of data that has been widely exploited to extract knowledge in many different domains. We have used a version of it, viz. WaCKypedia (Ba- Figure 6: Hidden topics derived from the Bridge- roni et al., 2009),3 that contains around 3 million man catalogue articles from Wikipedia segmented, normalized, POS-tagged and parsed. In order to extract those pages that could provide a better model for our 4 Data Sets specific domain, we selected those pages that con- Categories From a BAL six month log, we tain at least one content word of the BAL browse extracted all the top category connected to the categories listed below. queries via the click-through information and ob- The Arts and Entertainment, Ancient and World tained the list of 55 categories given by group in Cultures, Architecture, Business and Industry, Table 2. Crafts and Design, Places, Science and Medicine History, Religion and Belief, Sport, People and Society, Travel and Transport, Plants and Animals Queries From the six month log we have ex- Land and Sea, Emotions and Ideas tracted a sample of 1,049 queries by preserving the For our vocabulary, we considered only words distribution of queries per top-category obtained in the selected WaCKypedia pages that are either via the click-through information and the taxon- Nouns (N.*) or Verbs (VV.*) or Adjectives (J.*) omy. We selected only queries with at least one after being lemmatized. We obtain 14K docu- clicked image. Not all image metadata contains ≈ ments, with a vocabulary of 200K words, out title, keywords and a description: for around 60% ≈ of which we computed 100 topics. Examples of of images the meta-data provides only the title and random topics are illustrated in Figure 5. sub-category. For each query, we kept only one clicked image randomly selected. We leave for fu- ture study the impact the full set of clicked images per query could have on our query classifier.
Gold-standard: annotation by domain experts The 1,049 queries have been annotated by a do- main expert who was asked to assign up to three categories per query out of the 55 categories in Table 2 and to mark the query as “unknown” if no category in the list was considered to be ap- propriate. The domain expert looked at the click- Figure 5: Hidden topics derived from WaCKype- through information and the corresponding image dia to assign the categories to the query. The distri- Bridgeman catalogue Topic Model The most bution of queries per group of categories obtained straightforward way of choosing a close domain by this manual annotation is as following: 1395, corpus is to use the Bridgeman catalogue itself. 268, 87 queries have been annotated with a cat- We group together images that share the same egory out of the “topic”, “object” and “material” sub-categories and consider each group of sub- group, respectively. category as a document. We have 732 documents Out of this sample, 100 queries have been anno- and 136K words, out of which we computed tated by three annotators, BAL cataloguers, twice: ≈ 100 topics. Examples of topics estimated from this (a) by looking at the click-through information dataset are given in Figure 6. and the image, and (b) by looking only at the 3WaCkypedia (http://wacky.sslmit.unibo. query. The agreement between the annotators in it/doku.php) both cases is moderate (kappa in average 0.60 for
14 Topics Land and Sea; Places; Religion and Belief; Ancient and World Cultures; Mythology Mythological Myth; Allegory/Allegorical; People and Society; Sports and Leisure; History; Travel and Transport; Personalities; Business and Industry; Costume & Fashion; Plants and Animals; Botanical; Animals; The Arts and Entertainment; Emotions and Ideas; Science and Medicine; Science; Medicine; Architecture; Photography. Materials Metalwork; Silver, Gold & Silver Gilt; Lacquer & Japanning; Enamels; Semi-precious Stones; Bone, Ivory & Shellwork; Glass; Stained Glass; Textiles; Ceramics. Objects Crafts and Design; Manuscripts; Maps; Ephemera; Posters; Magazines; Choir Books; Cards & Postcards; Sculpture; Clocks, Watches, Barometers & Sundials; Oriental Miniatures; Furniture; Arms, Armour & Miltaria; Objects de Vertu; Trade Emblems, City Crests, Coats of Arms; Coins & Medals; Icons; Mosaics; Inventions; Jewellery; Juvenilia/Children’s Toys & Games; Lighting;
Table 2: Categories used by the annotators
the annotation without click-through information sifying in either cases: when evaluated against the and 0.64 for the annotation done using the click- manual or the automatic gold-standard) and anal- through information), the agreement is higher for yse them in further detail. the categories within the “topic” group. For each annotator, using the click-through information and 5 Experiments the image has not had a significant impact on the Let Q = q1, q2, . . . , qN be a set of N queries and annotation of categories from the “topic” group { } C = c1, c2, . . . , cM a set of M categories. We (kappa in average 0.80), whereas it has increased { } represent each query qi and category cj as the vec- and changed the annotation of categories from the tors −→qi = wtqi t V and −→cj = wtcj t V where { } ∈ { } ∈ other two groups, “object” (kappa 0.57) and “ma- V is the vocabulary that contains all terms in the terial” (kappa 0.62). corpus and wtqi , wtci are the frequency in qi and cj, respectively, of each term t in the vocabulary. Gold-standard: automatic extraction from We use the cosine similarity measure to assign the meta-data of the clicked image The top- categories to the queries. For each query q , the category associated in the taxonomy with the sub- i cosine similarity between every pair q , c categories of the image clicked after querying can h i jij=1..M is computed as: be extracted automatically exploiting the click- through information. Hence, we created a sec- ond gold-standard using such automatic extrac- −→qi .−→cj cosin sim(qi, cj) = = tion. Though our extraction is automatic, the as- −→qi . −→cj signment of the categories to the images is the re- | | | | t V wtqi .wtcj sult of the manual annotation by BAL cataloguers = ∈ 2 2 tPV wtqi . t V wtcj through the years. This annotation was done, of ∈ ∈ course, by looking only at the images, differently qP qP from the previous one for which the domain ex- For each query, the top 3 categories with highest perts was given both the query and the clicked im- cosine similarities are returned. age. This second gold-standard differs from the The different query and category enrichment one created by domain experts. For instance, the methods are spelled out in Table 3. To evaluate query “mountain lake near piedmont” is classified the effect of click-through information in query to the category “Places” by the expert, while using classification, we set up two different configura- the automatic mapping method, we obtain the cat- tions: QR, where besides the terms contained in egory “Emotions & Ideas: Peace & Relaxation”. the top and sub-categories, V consists of terms ap- The kappa agreement between the manual annota- pearing in the queries; QR-CT for which V con- tion and the automatic extraction is 0.52, 0.53, 0.6 sists also of terms in the title, keywords, descrip- for categories within the “material”, “object” and tion fields of the clicked images’ meta-data. In “topic” group, respectively. the case of the classifiers exploiting topic models, both vocabulary is extended with the hidden topics In our experiment, we will evaluate the classi- too and both queries and categories are enriched fier against the “manual” gold-standard and use with them as explained in section 3. In particular, the second one only to select the most challeng- TMwiki is the classifier based on the model built ing queries (those queries the classifiers fail clas- with the hidden topics extracted from WaCKpe-
15 dia, and TMBAL is the one based on the model we provide the total number of correct categories built out of Bridgeman metadata. found in all position 1, 2 and 3 ( T op 3). Setting Query enrichment Category enrichment QR q CAT + sCAT HitsP Setting QR-CT q + ct CAT + sCAT # 1 # 2 # 3 T op 3 TMwiki q + ct HTwiki CAT + sCAT HTwiki QR 92 38 26 156 ⊕ ⊕ P TMBAL q + ct HTBAL CAT + sCAT HTBAL QR-CT 183 97 62 342 ⊕ ⊕ q: query TMwiki 145 112 88 345 • ct: click-through information: title, keywords and TMBAL 340 257 144 741 description• - if available CAT: top category • sCAT: all sub categories of the corresponding CAT Table 5: Results of query classification: number • HTwiki: hidden topics from WaCKpedia of correct categories found (for 1,049 queries) • HTBAL: hidden topics from Bridgeman Metadata • As can be seen in Table 5, the performance Table 3: Experimental Setting of query classification using only terms in the queries (QR) is very poor. Already enriching 5.1 Results the query with the words from the title, keywords and description (QR-CT ) increases the To evaluate the classifiers, first of all we com- T op 3 by nearly 120%. pute Precision, Recall and F-measure as defined P Topics derived from the TM estimated from for KDD Cup competition and reported below.4 Wikipedia (TM ) did not help much in find- The results obtained are given in Table 4. wiki ing the right categories for a query. In comparison to QR-CT classifier, they decreased the number # queries correctly tagged as ci of correct categories in position 1 and they only P = i (2) # queries tagged as c slightly raised the number of correct categories P i i when considering the three positions. P On the other hand, the TM built from the # queries correctly tagged as ci R = i Bridgeman catalogue (TMBAL) increased the re- # queries manually labeled as c Pi i sults considerably for each of the three positions. (3) P Compared with QR-CT , 399 other correct cate- 2 P R gories were further found by using topics extracted F measure = × × (4) − P + R from the catalogue, giving a raise of 117%. The F-measure average at KDD Cup competi- tion was 0.24, with the best performing system reaching the result of 0.44 F-measure. Differ- ently from our scenario, the KDD Cup task was for web search query classification against 67 gen- eral domain categories (like shopping, companies, cars etc.) and classifiers could assign max. 5 cate- gories. In the following we report further studies of our results by considering the number of queries that are assigned the correct category in each of Figure 7: Matching QR-CT and TMBAL correct the three positions (Hits # 1, 2, 3). Furthermore, categories against the manual and automatic gold- standards 4http://www.sigkdd.org/kddcup/index. php?section=2005\&method=task Figure 7 reports the number of hits in each po- sition 1, 2, 3 for the two settings QR-CT and Precision Recall F-measure TMBAL. It clearly shows that TMBAL outper- QR-CT 0.11 0.17 0.13 forms QR-CT and matches more correct cate- TMBAL 0.26 0.40 0.31 gories both when considering either of the two gold-standards. It is interesting to note that this Table 4: P, R and F measures – Evaluation holds in particular for categories in the first posi-
16 tion of the ranked list (Hits #1): it results in a raise “Land & Sea” and its sub-categories, hence it was of 92% in the first position (from 224 correct cat- not possible for the QR-CT classifier to spot their egories to 431). similarity. However, the enrichment with the hid- den topics discovered the similarity between the 5.2 Analysis of wrong classification query and the top-category: they share topic To better understand the results obtained, we 14 with high probability. looked into the wrong classification. Figure 8 re- ports the number of queries for which QR-CT and TMBAL have not selected in the top three posi- tions any correct category using either the manual gold-standard and the automatic classification.
Figure 9: Effects of TM on the classification task
Figure 8: Queries incorrectly classified In total, the categories assigned to these 270 queries, were considered to be corrected in 123 We found that there were 692 queries cases for the TMBAL classifier and in 45 cases (422+270) for which QR-CT had not found for the QR-CT (Table 6). any correct category in the top three positions; Hits whereas 326 queries incorrectly classified by Setting # 1 # 2 # 3 T op 3 TM , of which 270 queries were in common QR-CT 31 7 7 45 BAL P with those wrongly classified by QR-CT . TMBAL 59 43 21 123 We further analyzed the set of 270 queries of Figure 8 which we take to be the most difficult Table 6: Correct categories checked by the expert queries to classify since neither of the two clas- for the 270 queries (using the click-through infor- sifiers have succeed with them considering ei- mation) ther the manual or the automatic gold-standard. These queries and the categories assigned to them Finally, the numbers of queries with at least one correct label out of these 270 queries are 39 (14%) by the QR-CT and TMBAL classifier have been QR CT checked and evaluated again by the domain expert. for the - method and 115 (43%) for the TM Figure 9 gives an example out of the 270 and the BAL method. result of the second run evaluation by the domain 6 Related Work expert. The top categories assigned to the query “mountain lake near piedmont” by the classifier (Cao et al., 2009) shows that context information QR-CT and TMBAL are “Ancient & World Cul- is crucial for web search query classification. They tures” and “Land & Sea”, respectively. The two consider the context to be both previous queries categories do not match either the correct category within the same session and pages of the clicked assigned by the expert (“Places”) or the category urls. In this paper, we focus on information simi- assigned by the automatic method (“Emotions & lar to the latter and postpone the analysis of query Ideas”). However, after being checked by the ex- session to further studies. (Cao et al., 2009) also pert, it was decided that the category proposed by shows that the taxonomy-based association be- the TMBAL classifier (“Land & Sea”) was also tween adjacent labels is useful for our task. Sim- correct whereas the one assigned by QR-CT was ilarly, we exploit Bridgeman taxonomy to enrich not. This query and click-through information do the categories target of the classifier. not share any common words with the category Finally, the use of a gold-standard automatically
17 created via click-through information is inspired Huanhuan Cao, Derek Hao Hu, Dou Shen, Daxi Jiang, by (Hofmann et al., 2010) where it has been shown Jian-Tao Sun, Enhong Chen, and Qiang Yang. 2009. that system rankings based on clicks are very close Context-aware query classification. In SIGIR’09, The 32nd Annual ACM SIGIR Conference. to those based on purchase decisions. There is strong evidence in favor of the relevance of click- T. L. Griffiths and M. Steyvers. 2004. Finding scien- through data to detect user’s intention. tific topics. Proceedings of the National Academy of Sciences, 101(Suppl. 1):5228–5235, April. 7 Conclusions Katja Hofmann, Bouke Huurnink, Marc Bron, and Maarten de Rijke. 2010. Comparing click-through This paper shows the effect of the click-through data to purchase decisions for retrieval evaluation. information and the use of topic models in query In SIGIR’10. classification in the art, history and culture closed Xiao Li, Ye-Yi Wang, and Alex Acero. 2008. Learn- domain. The main contribution of this study is the ing query intent from regularized click graphs. In proposal of using the metadata as a source to train SIGIR’08. topic models for the query and category enrich- Thomas Andrew Peters. 1993. The history and devel- ment. In particular, we first enriched the queries opment of transaction log analysis. Library Hi Tech, with the click-through information including in- 11(2):41–66. formation associated with the image clicked by Xuan-Hieu Phan, Cam-Tu Nguyen, Dieu-Thu Le, Le- the user. Then, we used topic models built out Minh Nguyen, Susumu Horiguchi, and Quang-Thuy of Wikipedia and the Bridgeman catalogue to an- Ha. 2010. A hidden topic-based framework to- alyze topics for both of the queries and the target wards building applications with short web docu- categories. Experiments from the real dataset ex- ments. IEEE Transactions on Knowledge and Data Engineering, 99(PrePrints). tracted from the query logs have shown the impact of the click-through information and topic models Dou Shen, Rong Pan, Jian-Tao Sun, Jeffrey Junfeng built from the catalogue in helping to find the cor- Pan, Kangheng Wu, Jie Yin, and Giang Yang. 2006a. Query enrichment for web-query classifi- rect categories for a given query. cation. ACM Transactions on Information Systems, In this paper, we have not considered more than 24(3):320–352. one click-through image for each query. How- Dou Shen, Jian-Tao Sun, Qiang Yang, and Zheng Chen. ever, we expect that more click-through images 2006b. Building bridges for web query classifica- can give a better understanding of user intent. Fur- tion. In SIGIR’06. ther research regarding this issue might be studied in more detail in future.
Acknowledgments
This work has been partially supported by the GALATEAS project (http://www.galateas.eu/ – CIP-ICT PSP-2009-3-25430) funded by the Euro- pean Union under the ICT PSP program.
References Marco Baroni, Silvia Bernardini, Adriano Ferraresi, and Eros Zanchetta. 2009. The WaCky wide web: a collection of very large linguistically pro- cessed web-crawled corpora. Language Resources and Evaluation.
David M. Blei and John D. Lafferty. 2007. A corre- lated topic model of science. AAS, 1(1):17–35.
David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent dirichlet allocation. J. Mach. Learn. Res., 3:993–1022, March.
18 Unlocking Language Archives Using Search
Herman Stehouwer Eric Auer MPI for Psycholinguistics MPI for Psycholinguistics [email protected] [email protected]
Abstract Skiba, 2009; Wittenburg and Trilsbeek, 2010; Wit- tenburg et al., 2010; Johnson, 2002) The Language Archive manages one of the In terms of preservation and access to cultural largest and most varied sets of natural lan- heritage data the MPI occupies a unique position. guage data. This data consists of video The uniqueness is captured by four aspects of the and audio enriched with annotations. It situation, (1) the cultural heritage data concerns is available for more than 250 languages, mainly currently spoken linguistic data, (2) often many of which are endangered. the linguistic data is accompanied by video to cap- Researchers have a need to access this ture non-verbal communication and cultural back- data conveniently and efficiently. We pro- ground, (3) the archive now hosts data from many vide several browse and search methods to linguistic preservation projects, linguistic studies cover this need, which have been devel- and psycholinguistic experiments, and (4) there oped and expanded over the years. Meta- are a variety of methods offered to browse, search, data and content-oriented search meth- and leverage the large archive of data. ods can be connected for a more focused This article is structured as follows. In Section 2 search. we provide an overview of the archive, in the form This article aims to provide a complete of a brief history and an overview of available overview of the available search mecha- data. In Section 3 we describe the current access nisms, with a focus on annotation content methods, including our powerful combined meta- search, including a benchmark. data and content search. We will present some per- formance statistics for the content search in Sec- 1 Introduction tion 4. We end this article in Section 5 with a sum- Digital preservation of cultural data has been an mary. important topic in much recent research. Large amounts of cultural heritage data is still only avail- 2 The Archive able in paper form. Digitization and digital preser- The archive stores a large variety of material vation is relatively affordable and provides a num- in more than 250 different languages. It con- ber of significant benefits. Digital data does not tains circa 160, 000 annotation files for more than degrade in quality with use. In addition, dissemi- 200, 000 audio or video recordings. The record- nation, access, and analysis techniques are easier ings include more than 4, 300 hours of SD quality1 with digital data. Furthermore, digital data enables video and more than 3, 500 hours of CIF quality2 researchers to answer questions which were unfea- video. The archived content is supported by al- sible to answer before. (Ordelman et al., 2009; most 200, 000 metadata files and 50, 000 auxiliary Reynaert, 2010) information files. Some of these digitization and accessibility 1 projects have focused specifically on access, i.e., SD means Standard Definition (resolution). Circa 14, 500 videos, 78% are 720 576 pixels (resolutions from by improving search methods for the domain. 640 ... 768 480 ... 576) × × Only a powerful search tool allows researchers to 2CIF means Common Intermediate Format, which im- quickly find their “needles in the haystack”. (Auer plies a specific (lower) resolution range. Circa 40, 400 videos, 78% are 352 288 pixels (resolutions from et al., 2010; Kemps-Snijders et al., 2009; Kemps- 320 ... 384 240 ... 384×, plus 2, 350 double width videos Snijders et al., 2010; Ringersma et al., 2010; merged from× 2 CIF videos each)
19 Proceedings of Language Technologies for Digital Humanities and Cultural Heritage Workshop, pages 19–26, Hissar, Bulgaria, 16 September 2011. The material in the archive occupies more than Quality Control 40 terabyte of storage capacity, most of which for The whole archive is permanently under active the media recordings. There are 22 gigabytes of quality control. Files can only be uploaded annotation files, 2.5 gigabytes of metadata and 7 by authorized researchers using the web-based gigabytes of auxiliary files. In addition, there is LAMUS4 archive upload and editing tool which currently more than 55 terabyte of “pre-archive” also applies format checks. Those checks ensure data in the pipeline on the way to the archive. that only file formats for which free viewers (and Based our experience, the creation of one terabyte preferably editors) are widely available are stored. e of archive-able data costs around 1.5 million . Archive managers define format rules and make (Wittenburg et al., 2010) gives a recent descrip- statistics about archived data, e.g. about video tion of the state of The Language Archive (TLA). resolutions or audio sampling rates. They also They deal with three aspects pertaining to the run regular consistence checks on the archive, link archive: (1) replication of archived material, bit- structure and file formats. wise copies of the original material – each file is For annotation file formats supported by the stored in six copies in two countries, (2) encoding search methods, files are parsed to verify their syn- and metadata standards, and how they apply to the tactic validity. Parse logs are reviewed to find archive itself, and (3) controlled access to archived problematic files and adapt parsers for common materials. The article provides a good overview of syntax variants, for example for CHAT, ELAN the TLA resources and software used for manag- and Toolbox5. The customized parser for example ing the archive. has heuristical parsing of CHAT participant lists, which in turn can be used to search. Access Methods 3 Searching The archive itself is accessible in a number We aim to help researchers answer their scientific of ways, most of which can be reached from questions. Often these questions may be answered the www.clarin.eu Virtual Language Observatory by providing the right data. To help locate the right (VLO) (Uytvanck et al., 2010). The classic ac- data we provide a variety of browse and search cess method is the IMDI3 browser, which displays methods. In this section we aim to give a brief a tree view on the Directed Acyclic Graph (DAG) overview of each of the available methods. Any type archive structure graph. part or multiple parts of the DAG tree in the IMDI When using the catalogue on the CLARIN browser can be selected to perform any type of VLO portal, the TLA archive and several exter- search on. nal archives can be visited in one browser ses- sion. To enable central access, CLARIN ex- 3.1 Metadata Search changes and harvests metadata with other archives Here we briefly describe the metadata search. The using the OAI protocol. We remark that the metadata search is available from within the IMDI different archives are also accessible separately. browser. The metadata search contains two differ- For instance, the TLA data is accessible on the ent search methods: (1) a keyword search, and (2) web at www.lat-mpi.eu/tools/browser. A faceted an advanced metadata search. search (based on Apache SOLR) of the combined The first performs a quick search for a set of archives is available in the VLO language re- keywords in all available metadata fields. The sec- source inventory described below. The researcher ond allows the researcher to define a set of con- can also browse the virtual language world in straints. These constraints are then used to select the VLO. The virtual language world presents all matching records in the part of the archive that the available corpora on a world map. Faceted is searched on. search and geographical browsing always present 4 the combined archives, not just TLA data. The Language Archive Management and Upload System – www.lat-mpi.eu/tools/lamus TLA archive is also accessible via a number of 5CHILDES CHAT: Child Language Data Exchange other search methods, which we outline below. System, Codes for the Human Analysis of Transcripts childes.psy.cmu.edu/clan/. ELAN EAF: EUDICO Linguistic Annotator www.lat-mpi.eu/tools/elan/. Toolbox, Shoebox: 3Isle MetaData Initiative, an XML metadata standard. www.sil.org/computing/toolbox/.
20 We provide a telling example. To search for au- were in Dutch corpora such as CGN. Simple or dio recordings of young speakers, we can use the single layer search were most common. following two constraints: (1) actor age is smaller In the first half of the analyzed period, only than 10, and (2) the format of the resource is an 11 queries per month used structured multi layer audio file. The second constraint is entered with a search. Most structured search queries had a com- user friendly choice list. Optionally, the researcher plexity of up to four keywords or constraints and can see which choices would match how often. were not in Dutch corpora. Half of the complex Using a fast Apache Digester index, results are queries used a constraint that keywords in two tiers presented without noticeable delay. had to co-occur (same timing). Once a researcher has performed a metadata search they can choose to use the results in three Two possible reasons for the infrequent use of ways: (1) the results can be viewed and printed, Trova in the past are the steep learning curve of (2) the results can be exported as links in an IMDI structured search and slow speed. To address this, file for later use, and (3) the results can be used we improve documentation and teaching. Also, directly to specify the domain of a content search Trova was slow compared to typical web search (described below). engines. Incremental processing already helped by showing the first results while the query was 3.2 Trova Annotation Content Search still running. However, overall processing time Here we briefly describe Trova, the annotation was still high and complete result lists are of more content search. A Trova search always starts from interest in linguistic context than in web searches. a selection of elements in the archive, which are used as the search domain. Typically the search domain consists of a corpus, or of the domain se- Optimizations lected using the metadata search. While all meta- To optimize search performance, searching is per- data is freely accessible, all access to annotation formed in three steps: First, when a researcher content including search is controlled by the ac- enters the search page, the properties of all tiers cess rules6 for the individual corpora. (a tier is a layer of annotation) in the selected Trova supports three different search methods: search domain are processed. Second, as soon (1) the simple search, (2) the single layer search, as a query is made, fingerprinting is used to limit and (3) the multiple layer search. We describe searching to a small set of candidate tiers to im- them in order of complexity. prove query speed. Each tier is indexed with four In all three search modes, the researcher can se- different character n-gram fingerprints. The cur- lect which of the searchable file types should be rent fingerprints group all possible combinations considered: ELAN EAF, CHILDES CHAT, Shoe- of 1 to 4 characters into a limited number of slots. box, Toolbox, text, HTML, XML, PDF, SubRip, A tier can only contain a match for a given key- Praat TextGrid and CSV7. word when it fills at least all slots used by that key- Usage Considerations word. For regular expression search, plain text is extracted from the query to still use partial finger- The Trova application is the main way to search printing. Third, the candidate tiers are processed the online archive. However, after excluding in small groups, displaying hits as they come in. queries made for demonstration, teaching and test- ing purposes, it turned out that Trova was not used While a query is still running, the researcher can heavily in the past: already browse the first results. However, the hits In the period from April 2008 to July 2010, will only be displayed in their final ordering when there were more than 2000 user queries, about 80 they all have been found. At that point, hits can per month, of which 75% unique. Most searches also be viewed sorted by their most frequent con- cordances. Hits can be saved in CSV files using a 6The TLA AMS access management system (www.lat- mpi.eu/tools/ams) allows to define individual and group ac- user-selectable order and choice of columns. From cess rules on the level of corpora, sessions, filetypes and files. each hit, the researcher can navigate to Annex 7 Our database contains circa 123, 000, 000 annotations in (Berck and Russel, 2006), a browser based pre- 750, 000 tiers from 110, 000 parsed files. The most common annotation file types are CHAT (48, 600 files), EAF (28, 100 sentation of the parsed annotation file along with files) and plain text (26, 400 files). corresponding media files.
21 Simple Search have to overlap. In the grid, the X axis corre- The simple search allows searching for keywords sponds to the time axis, while the Y axis corre- in the selected search domain. The search per- sponds to different tiers within one file. Similar to formed using these keywords performs a case- the single layer search, the researcher can activate insensitive substring matching. constraints about the properties of tiers, but now separately for each grid row. An example query Single Layer Search could be ‘pink’ before ‘elephant’ in a text type tier, Single layer search gives the researcher more con- ‘elephant’ overlapping ‘big’ time-wise in a ges- trol over the search than when using the simple ture type tier. In Figure 1 we show an example of search. The researcher can select whether match- the multiple layer search, showing this example. ing should use exact matching, substring match- 3.3 CQL Search Service ing, or regular expression matching. Furthermore the researcher can choose whether he wants the In the context of the European search infrastruc- matching to be case sensitive. It is also possible ture we make available a machine-accessible web- to perform searches over n-grams of annotations. service for content search. This web-service pro- Both n-grams inside and across annotations can vides a search-service on parts of the archive (as be searched. The n-gram search modes support access rights permit) and is integrated in the Euro- single position wildcards (#) and per word nega- pean search infrastructure. The European search tion, e.g., the # NOT(green) house. In the example, infrastructure provides a central location for re- the phrase I went to the big red house yesterday. searchers to perform searches in many different would match. language resources. The web-service is based on We remark that exact match means that one an- SRU/CQL(Morgan, 2004; S.H. McCallum, 2006), notation has to match the keyword exactly. The re- where SRU is the communication protocol and searcher has to be aware that some annotation tiers CQL the search query language. annotate whole utterances as one annotation while CQL search provides functionality based on the 9 others annotate word by word. In some cases, Trova single layer search through a REST inter- searching with a regular expression can be more face. More complex processing will be added in appropriate. the future, as CQL allows to express fairly com- A further option in single layer search is re- plex queries. These queries differ from the 2d stricting the search to a subset of the available grid paradigm of Trova. As stateless REST means tiers. Annotation tiers can be selected by several processing whole queries in one single HTTP ac- properties, namely: name, type, participant, and cess, CQL search can cache intermediate results annotator. The researcher can see how many tiers for later re-use. So when the same researcher match which value and can sort the choice list by makes multiple queries to the same corpus, hits that. This allows to quickly see that a corpus con- will be returned faster. REST queries can option- tains more type pho tiers than type Phonetic tiers8. ally return as soon as some results have been found or wait until all results are ready. Multiple Layer Search 3.4 Virtual Language Observatory Multiple layer search is the most advanced search interface available on the archive. In multiple A separate way in which to browse the meta- layer search, a grid of search terms can be en- data is available in the Virtual Language Ob- tered, with constraints between them. Constraints servatory (VLO). The VLO makes available a on the X axis can be used to require a certain (or faceted browser on the metadata from several lan- minimum or maximum) time or number of annota- guage sources, including our archive. To use the tions between keyword hits. Constraints on the Y VLO, enter the language resource inventory on axis give the researcher fine grained control over the www.clarin.eu/vlo virtual language observa- whether and how keyword hits in different tiers tory page. In faceted browsing, different IMDI and CMDI metadata fields can be used to zoom in 8Unfortunately, there are no widely used controlled vo- cabularies to classify tiers. We plan to use ISOcat / RELcat on corpora. Supported facets include origin (e.g., concept registries to allow a more semantic view on tier prop- MPI, Open Language Archives Community), con- erties. This would allow selections such as tier types with parent category DC-2641: phonetics. 9Representational State Transfer, e.g. via HTTP.
22 Figure 1: Partial screenshot of multiple layer search. Showing the pink elephant search example. tinent, country, language, resource type (e.g., au- to various optimizations described above, such as dio), subject, genre, and organization. fingerprinting of tiers and queries, response times All facet lists are shown with counts of occur- are still reasonably fast. rences which are dynamically updated as the re- To provide some benchmarks, we ran a number searcher makes selections. For example, after se- of searches on a large subset of the archive consist- lecting Dutch (18, 000 resources), the facet origin ing of several sizable corpora: All of DoBeS10, the is updated, showing that most Dutch resources are Dutch Spoken Corpus CGN, the MoLL L2 acqui- from the CGN corpus. The researcher can then sition corpus11 as well as local mirrors of Talkbank proceed to specify more facets, for example select and the CHILDES sub-corpora Biling, Japanese, the Dutch Bilingualism Database (DBD) corpus as Cantonese, Turkish, Spanish, French and German. origin, select a genre, and so on. Facets can be The size of this search space is almost 55, 000, 000 specified in any order and page updates are almost annotations in 354, 000 tiers in 43, 000 files. The instantaneous, using a SOLR database. initial analysis of the search domain can take 20 At any moment, a keyword search on metadata seconds or more but then multiple queries can descriptions can be used to narrow down the re- be done with low further overhead. Finding all sults. From the result list, metadata sets can be occurrences of elephant, needle or haystack in displayed as tables and the researcher can jump more than 50 million annotations can then be done directly to resources such as audio recordings. within 7 seconds and initial results are shown much sooner. Among other things, speed depends 3.5 ELAN Structured Multiple File Search on narrowing down the search space by finger- A modified version of Trova is one of the search printing and on I/O caching at the Linux and Post- functions of ELAN annotation editor. This enables greSQL level. Searching in smaller corpora is the researcher to search in (a group of) annotation much faster. For example searching only in the files while they are still being worked on. Differ- seven mentioned CHILDES sub-corpora, worth ent from Trova, this search parses files on the fly. about 4, 000, 000 annotations, has a set-up time of ELAN can import and export a number of other less than 10 seconds and after that, searches take file formats and the search could parse some of at most a few seconds. them directly. In addition, ELAN supports list-of- All benchmarks were done on an older test constraints style search similar to metadata search. server with two dual core 1.8 GHz AMD Opteron Not using a full corpus database means reduced 265 CPUs, 16 GB of DDR400 RAM (8 modules) speed compared to Trova, but all annotation up- and a small SCSI 160 RAID with 120 MB/s read dates are available in searches immediately. bandwidth. A modern desktop PC can have twice the speed, cores, RAM and disk bandwidth with 4 Annotation Search Benchmark one CPU and a consumer SSD. Trova used up 4.1 Test Corpus and Hardware to three database threads on our test server, Post- greSQL uses one core per thread. PostgreSQL is The various metadata search methods provide re- configured to use 3.5 GB of shared buffers, 1 GB sults almost instantaneously. However, with al- of work memory per task and 6 GB of cache size. most 120, 000, 000 individual annotations in more than 100, 000 annotation files in the archive, con- 10Dokumentation Bedrohter Sprachen tent search is a more demanding task. Thanks 11Project “Modalitat¨ in Lernervarietaten¨ im Langsschnitt”¨
23 4.2 Task Design Block AM AM MD Min Max hits duration (in seconds) We searched for ten keywords each from a set substr of eight languages in our 55M annotation test domain. For German, English and Dutch, we Dutch 9453 13.16 13.23 4.2 27.5 used entries 991 to 1000 of the “top1000” lists English 3532 8.26 6.44 4.2 20.0 of wortschatz.uni-leipzig.de; For French, Span- French 3603 7.25 6.19 3.9 17.4 ish and Turkish, we used entries 991 to 1000 of German 1634 6.57 4.75 2.7 23.0 the word frequency lists on en.Wiktionary.org; For Japanese 689 0.89 0.55 0.3 2.0 Russian, the masterrussian.com list of most com- Russian 50 0.61 0.60 0.5 0.8 mon words was used. No frequency list was avail- Spanish 1336 5.98 5.10 4.1 9.4 able for Japanese, so we used a selection of words Turkish 113 6.80 6.75 2.1 23.1 from “1000 Japanese basic words” from Wik- Table 1: Benchmark results for substring queries. tionary. The Japanese selection contains nouns 30 queries per language, 60 for Japanese. AM = which are translations of words present in the se- Arithmetic Mean, MD = Median. lections of the other languages. For this set of terms12, we investigated three Block AM AM MD Min Max search paradigms in sequence: (1) keyword search regexp hits duration (in seconds) with substring matching (i.e., the keyword can match any part of the annotation), (2) regular ex- Dutch 2834 11.55 10.86 4.7 25.8 pression search, either for “word between word English 2512 9.18 7.71 5.1 26.2 boundaries” (German, English, Dutch, French and French 2134 7.68 7.60 3.7 21.6 Spanish), or for “word starts with. . . ” (Turkish, German 371 5.85 4.77 2.8 11.6 Russian and Japanese)13, and (3) keyword search Japanese 414 0.85 0.51 0.3 1.9 with exact matching (i.e., the keyword must match Russian 15 0.64 0.63 0.5 0.9 the entire annotation). Spanish 882 5.55 4.81 3.8 8.1 All queries were done via the CQL REST inter- Turkish 132 9.56 7.84 3.4 32.2 face three times in a row, requesting to wait un- til all results have arrived. We observe that rep- Table 2: Benchmark results for regexp queries. etitions of queries differ in average speed by less 30 queries per language, 60 for Japanese. AM = than 10%. In addition, we first searched for a bo- Arithmetic Mean, MD = Median. gus word (*start*), taking up to 20 seconds while CQL search caches domain properties before the Searching for English, French, German and Span- timed queries start. ish already is twice as fast, as is Turkish. Turkish words tend to have fingerprints similar to those of 4.3 Results words in other languages. Searching only in the Substring Search Turkish sub-corpus would be a lot faster. In Table 1 we show the results for the substring But why do we get all results within less than matching. We observe that the query speed varies one second for Japanese and Russian, without ex- significantly with language: Our test corpus con- plicitly searching only in relevant sub-corpora? tains a large number of Dutch annotations, so this This is again due to fingerprinting: Text in other task takes the most time and finds the most hits. languages will most likely not contain any cyril- 12We replaced a number of word forms from the original lic, kanji or hiragana characters at all. So even by lists to be more suitable for raw string search as follows: pah- looking only at unigrams, Trova and CQL search nut’ (infinitive) to pahnet, zavod’ (-’ only in one form) to za- can quickly discard most tiers in other languages vod, quer´ıas to quer´ıa, hareket etmek (inf.) to hareket ettiler and istenmek (inf.) to istenem. Of course we used cyrillic when searching for Japanese or Russian words. strings in the actual queries. The latin transliterations are only used for easier reading. Regular Expression Search 13Word boundaries ( bkeyword b) are ASCII-oriented, not covering accented characters,\ but\ do work with punctuation The second block of queries uses regular expres- marks. The regular expression ( s A)keyw works in Uni- sion search, results are shown in Table 2. As ex- code space, anchoring keyw to the\ start|\ of the annotation or a space. However, e.g. opening parentheses or quotation marks pected, we observe fewer hits than with a plain before keyw will not match. substring search. This also explains small speed
24 Block AM AM MD Min Max Exact String Search exact hits duration (in seconds) Our third round of queries only considers exact matches. We show this round in Table 3. While Dutch 1756 10.25 9.18 3.4 26.8 there is some speed gain compared to substring English 64 7.02 5.44 3.6 18.7 matches, related to having fewer hits, it is much French 45 5.91 5.12 3.5 12.7 smaller than expected. Some time is saved be- German 3 5.66 3.88 2.4 22.9 cause string inequality can often be detected with- Japanese 0 0.74 0.40 0.2 2.8 out having to scan the whole string. However, the Russian 6 0.51 0.50 0.4 0.7 set of candidate tiers chosen by fingerprinting is as Spanish 26 4.87 4.26 3.5 7.0 big as for substring search. Turkish 1 5.23 5.58 2.1 7.9 Adding specific indexes can improve exact Table 3: Benchmark results for exact queries. 30 match speed, but will only have an effect on this queries per language, 60 for Japanese. AM = match mode. For example, such an index could Arithmetic Mean, MD = Median. bin whole-string hashes in slots. Another possi- bility would be an index of only the sets of string Block AM AM MD Min Max lengths occurring in each tier. all hits duration (in seconds) Substring searches are most used, especially be- cause not all corpora have annotations at word Dutch 4681 11.65 10.29 3.4 27.5 granularity. Many corpora annotate larger units, English 2036 8.15 6.61 3.6 26.2 such as phrases, sentences or utterances, but at French 1927 6.95 5.58 3.5 21.6 higher quality, e.g., stating recording timestamps German 669 6.03 4.51 2.4 23.0 for them. Searching for annotations which are ex- Japanese 368 0.83 0.52 0.2 2.8 actly elephant will not work in a corpus treating I Russian 24 0.58 0.59 0.4 0.9 saw an elephant. as one atomic annotation. Spanish 748 5.47 4.92 3.5 9.4 Turkish 82 7.19 6.43 2.1 32.2 Overall Speed Table 4: Benchmark results for all queries. 30 Finally, Table 4 gives a summary of all queries in queries per language, 60 for Japanese. AM = our benchmark task: The average and in particu- Arithmetic Mean, MD = Median. lar median query durations in our 55M annotation test corpus are considerably below 10 seconds for most languages. For languages which can be read- gains for Dutch and Spanish. For the other lan- ily identified from their writing system, waiting guages, in particular Turkish, a small speed loss times below one second can be expected. can be seen. Here, two forces act on the process- While it is not visible in this table, our ex- ing load: Searching for (shorter) prefixes means perience shows that long waiting times relate to that fewer tiers can be discarded based on n-gram novel queries for hard to filter words. The slowest fingerprints. More have to be considered, yet those query is the regular expression search for words contain fewer hits because specifying a regular ex- starting with isten. Searching for isten-after-space pression is more restrictive than a substring. In would be faster (37% fewer candidate tiers) but addition, regular expressions take more CPU time would not find annotation-initial occurrences of to process. Note that the fingerprinting only con- isten. Repeating the query later reduces waiting siders the plain parts: For example, a search for time from 32 to 28 seconds, showing the effects of ( s A)yumurt will consider all tiers which sat- \ |\ disk caching. isfy the n-grams of yumurt14. 5 Summary 14The “stemming” of yumurta to yumurt is an artificial ex- ample and not meant to be linguistically correct. Using fin- gerprint tables up to n-gram size 4, from y, u. . . , yu, um. . . In this article we have described the TLA lan- to murt have to be present in a tier to make it a candidate. guage archive and possibilities to search in it. The To balance disk space usage against speed, not all possible archive contains a large and diverse collection of combinations of 1 to 4 Unicode characters are fingerprinted separately. Instead, n-grams are hashed into bins – for exam- language data occupying over 40 terabytes of stor- ple, all possible 4-grams share 2, 000 classes. age for more than 250 languages.
25 We presented a variety of browse and search [Ringersma et al., 2010] Ringersma, J., Zinn, C., and methods available for our archive, developed over Koenig, A. (2010). Eureka! user friendly ac- several years. We described the speed of our an- cess to the mpi linguistic data archive. In SDV - Sprache und Datenverarbeitung/International Jour- notation content search on a small server, using nal for Language Data Processing. a large test corpus. We have listed results from benchmarks, and analyzed them. [S.H. McCallum, 2006] S.H. McCallum (2006). A look at new information retrieval protocols: Sru, Furthermore, we have discussed several current opensearch/a9, cql, and xquery. In WORLD LI- and future optimizations that can improve search BRARY AND INFORMATION CONGRESS: 72ND and browse speed. Of course, our implementation IFLA GENERAL CONFERENCE AND COUNCIL, is also guided by the most common use cases. Seoul, Korea. [Skiba, 2009] Skiba, R. (2009). Korpora in der Zweitspracherwerbsforschung: Internetzugang zu References Daten des ungesteuerten Zweitspracherwerbs. In [Auer et al., 2010] Auer, E., Wittenburg, P., Sloetjes, Ahrenholz, B., Bredel, U., Klein, W., Rost-Roth, Empirische Forschung H., Schreer, O., Masneri, S., Schneider, D., and M., and Skiba, R., editors, und Theoriebildung: Beitrage aus Soziolinguis- Tschopel,¨ S. (2010). Automatic annotation of media ¨ tik, Gesprochene-Sprache- und Zweitspracherwerb- field recordings. In Sporleder, C. and Zervanou, K., sforschung: Festschrift fur Norbert Dittmar editors, Proceedings of the ECAI 2010 Workshop on ¨ , pages Language Technology for Cultural Heritage, Social 21–30. Sciences, and Humanities (LaTeCH 2010), pages [Uytvanck et al., 2010] Uytvanck, D. V., Zinn, C., 31–34, Lisbon. University de Lisbon. Broeder, D., Wittenburg, P., and Gardelleni, M. (2010). Virtual language observatory: The portal to [Berck and Russel, 2006] Berck, P. and Russel, A. the language resources and technology universe. In (2006). Annex – a web-based framework for ex- Calzolari, N., Maegaard, B., Mariani, J., Odijk, J., ploiting annotated media resources. In LREC. Choukri, K., Piperidis, S., Rosner, M., and Tapias, [Johnson, 2002] Johnson, H. (2002). The archive of the D., editors, Proceedings of the Seventh conference indigenous languages of latin america: Goals and on International Language Resources and Evalu- visions. In Proceedings of the Language Resources ation (LREC’10), pages 900–903. European Lan- and Engineering Conference, Las Palmas, Spain. guage Resources Association (ELRA).
[Kemps-Snijders et al., 2010] Kemps-Snijders, M., [Wittenburg and Trilsbeek, 2010] Wittenburg, P. and Koller, T., Sloetjes, H., and Verweij, H. (2010). Trilsbeek, P. (2010). Digital archiving – a neces- Lat bridge: Bridging tools for annotation and sity in documentary linguistics. In Senft, G., editor, exploration of rich linguistic data. In Calzolari, Endangered Austronesian and Australian Aborigi- N., Maegaard, B., Mariani, J., Odijk, J., Choukri, nal languages: Essays on language documentation, K., Piperidis, S., Rosner, M., and Tapias, D., archiving and revitalization, pages 111–136. Can- editors, Proceedings of the Seventh conference on berra: Pacific Linguistics. International Language Resources and Evaluation [Wittenburg et al., 2010] Wittenburg, P., Trilsbeek, P., (LREC’10), pages 2648–2651. European Language and Lenkiewicz, P. (2010). Large multimedia Resources Association (ELRA). archive for world languages. In Proceedings of [Kemps-Snijders et al., 2009] Kemps-Snijders, M., the 2010 ACM Workshop on Searching Spontaneous Windhouwer, M., and Wittenburg, P. (2009). Isocat: Conversational Speech, Co-located with ACM Mul- Remodeling metadata for language resources. In timedia 2010, pages 53–56. Association for Com- International Journal of Metadata, Semantics and puting Machinery, Inc. (ACM). Ontologies (IJMSO), volume 4 (4), pages 261–276.
[Morgan, 2004] Morgan, E. (2004). An introduction to the Search/Retrieve URL Service (SRU). Ariadne.
[Ordelman et al., 2009] Ordelman, R. J. F., Heeren, W. F. L., de Jong, F. M. G., Huijbregts, M. A. H., and Hiemstra, D. (2009). Towards affordable disclosure of spoken heritage archives. Journal of Digital In- formation, 10(6):17.
[Reynaert, 2010] Reynaert, M. (2010). Character confusion versus focus word-based correction of spelling and ocr variants in corpora. International Journal on Document Analysis and Recognition, pages 1–15. 10.1007/s10032-010-0133-5.
26 Digital Library of Poland-related Old Ephemeral Prints: Preserving Multilingual Cultural Heritage
Maciej Ogrodniczuk Wlodzimierz Gruszczynski Institute of Computer Science Warsaw School of Social Sciences Polish Academy of Sciences and Humanities [email protected] [email protected]
Abstract documents. Some of them have been preserved in the single copy, so their availability had been The article presents the results from the evidently restricted. In the course of the project project of the thematic Digital Library the resources were gathered basing on the list of Polish and Poland-related Ephemeral of 2,000 bibliographical entries from Konrad Za- Prints from the 16th, 17th and 18th Cen- wadzki’s publication (Zawadzki, 1990) extended turies, intending to preserve the unique with objects described or discovered after its is- multilingual material, make it available for sue. Selected prints have been commented by his- education and extend it with the joint ef- torians, media experts and linguists to explain less forts of historians, philologists, librarians known background details or presently unintelli- and computer scientists. gible metaphors or symbols. Links have been cre- The paper also puts forward the idea of a ated between related documents (e.g. translations living digital library, created with a closed and their alleged sources or derivates) to facilitate set of resources which can form the basis comparisons of similar materials. For 70% of the for their further extension, thus turning tra- prints their images were obtained and made avail- ditional digital archives into collaboration able in DjVu format. At the end of the project the platforms. revised edition of Zawadzki’s work was published in electronic form. 1 Introduction Apart from this particular work plan, the ulte- Nowadays digital libraries most likely tend to mir- rior idea was to test the concept of using the digital ror traditional libraries. They collect and display library as a collaboration platform for experts from resources as traditional librarians would do, per- different backgrounds basing on the assumption fectly embracing the new archiving capabilities, that real opening resources to the public means not but too often stopping at this border. The the- just providing them for viewing and download, but matic Digital Library of Polish and Poland-related preparing the environment to extend them in the Ephemeral Prints from the 16th, 17th and 18th immediate or more distant future. The technical Centuries (PL. Cyfrowa Biblioteka Drukow Ulot- challenge was to select the computer system which nych polskich i Polski dotyczacych z XVI, XVII could serve this purpose best by mostly configura- i XVIII w. – hence and from here: CBDU, tion and without too much tedious low-level pro- http://cbdu.id.uw.edu.pl) offers a new gramming. This goal was achieved with EPrints approach to the idea of a modern digital library free repository software. by extending the conventional paradigm with us- At present the library is actively used by histori- ing consistent sets of materials as an object pool ans and linguists (not to mention the students) who for new collaboration tasks. In our case the prints seem to benefit the most from the cooperation and digitized in the first step of the project were further further development of the resources. Their recent analyzed and enhanced by experts co-operating on interests include adding transliterations (which al- a digital content platform. ready started with a new project based on the li- The work had been carried out within the brary materials) and using the print texts as source project financed by the Polish state with inten- data for the searchable text corpus. Multilinguality tion to provide public online access to all pre- of texts should also be shortly addressed since the served and described in the literature pre-press first stage of work concentrated mostly on Polish
27 Proceedings of Language Technologies for Digital Humanities and Cultural Heritage Workshop, pages 27–33, Hissar, Bulgaria, 16 September 2011. and German documents. been collated by Konrad Zawadzki in the 1970s The aspect of making the digital library a col- and 1980s and was published as a three-volume laboration platform seems the most important in work with the first volume issued in 1979 and the our study, putting less light into other important last one in 1990. It covers 1967 prints dating from issues of creating thematic libraries, commenting 1501 to 1725, each described with the title (in historical content or digital library-based teaching. modern transcription), issue date and place, printer As such, creation of a digital library may be in name, format, volume size, information on bibli- most cases turned into a development project, be- ographical sources and an exact description of the ing of benefit to the scientific community. title page (with line breaks, font names, illustra- tions etc.). 2 Origin, scope and limitations of the The originals of prints remain in various li- project braries over Europe, but at the period of prepar- ing the bibliography many were successfully bor- The project has been initiated in response to the rowed from their mother institutions to produce need of permanent access to ephemeral prints from microfilmed copies to be included into the re- 16th-18th centuries by researchers coming from sources of The Polish National Library (Za- the academic teams preparing the historical dictio- wadzki’s place of employment). As microfilmed naries of Polish (Institute of Polish Language and resources are not the most comfortable ones to op- Institute of Literary Research at Polish Academy erate on a daily basis, usually their photocopies of Sciences) as well as journalism and translation were used for research and teaching. The online historians (Institute of Journalism, Institute of Ap- era created a new possibility to interact with such plied Linguistics and Institute of German Studies resources (e.g. broaden the scope of materials at the Warsaw University) and students of journal- showed to students) and resulted in applying for ism. funds to create a digital library of metadata and Temporal range of selected materials results images of prints enhanced with information useful from the long history of Polish journalism with for understanding the context of a given object. three significant dates: 1501 – when the first The priority of making prints available for the known press materials in Polish appeared in print daily work (disparate with the need of preserva- (the report on the anti-Turkish treaty signed by the tion of original objects) resulted in the assumption Holy See and several European countries, includ- of building on the quality of microfilms rather than ing Poland, in Buda), 1661 – when the first reg- archiving originals scattered over many libraries ularly issued Polish newspaper “Merkuriusz Pol- in many countries. This also helped minimize ski Ordynaryjny” came out, and 1729 – when costs and speed up the overall process, mostly also “Nowiny Polskie” (Polish news) started regular thanks to the presence of vast majority of materials circulation. The period in between was filled by at the National Library. ephemeral prints – disposable and occasional in- The source of funding was constrained to na- formative publications, playing an important part tional (ministry) level and not to the consortium in the development of Polish writing, serving as a of libraries (owners of microfilmed print copies) the most influential media for important news. knowing the three contradictory factors: The scope of the materials is Poland-related, which combines the Polish sources with prints 1. vast resources of large libraries, such as the published abroad, concerning the Republic of Polish National Library, Poland, political, religious and military issues (e.g. the reports on the famous relief of Vienna in 1683), 2. their limited funds for digitization, but also sensational facts or canards. The materi- als were prepared “live”, mostly by participants or 3. a policy (telling the truth, a reasonable one) observers of the reported events and as such they of digitizing the most valuable resources first. are valuable sociological source of information on mechanisms of spreading information at that time, This combination can make many interesting its reception, propaganda and readers’ interests. materials wait for years to be made available at The list of bibliographical data of prints com- their owner’s institution. Providentially, in March plying with all above-mentioned requirements has 2009 the 12-month project obtained the support
28 Figure 1: A sample bibliographical entry
from the Ministry of Culture and National Her- less of its initial inconsistent formatting (which itage and the Foundation for the Development of was easily introduced at a later stage, basing on Journalism Education. the entry structure). The text has been saved in UTF-8 character encoding to seamlessly represent 3 Towards the thematic digital library all diacritics. Perl scripts have been implemented to split bib- The project team was headed by Wlodzimierz liographical entries into individual data records ac- Gruszczynski and joined forces of computer sci- cording to the description of fields retrieved from entists (Maciej Ogrodniczuk, Jakub Wilk), his- the bibliography introduction (full and short title, torians (Adam Kozuchowski), philologists (Ewa information about author, publisher, format etc.) Gruszczynska, Anna Just, Dorota Lewandowska- In fact, the field list was designed to be more spe- Jaros, Katarzyna Jasinska-Zdun) and librarians cific than the original to support verification of (the team of Maria Piber), coordinated by Grazyna content (e.g. the model of a list of copies with sub- Oblas. fields storing library name and a catalogue num- The process started with scanning Zawadzki’s ber was introduced against the original composite bibliography in the format sufficient to automated string value). The field set has been later used as a OCR processing of the text (200 dpi, greyscale, basis of the target model for the computer system lossless compression of the result files, see Fig. storing the library data with new fields describing 1). The images have been read with ABBYY the project-specific properties going beyond the FineReader 9 with support for several modern lan- traditional bibliographical entry (e.g. commen- guages turned on (French, German, Latin, Polish) taries of an object). The records were then verified since texts could contain transcriptions of names with regular expression patterns testing their con- in various (modern) languages. tents (e.g. publication date standard format) and As a result, a recognized plain text of the bib- content of fields used in further steps (e.g. micro- liography has been obtained, covering not only film catalogue numbers) extracted separately and full bibliographical entries (with additional com- additionally verified. ments, location information etc.), but also all front As the majority of microfilms were available and back matter data (volume introductions, er- at the National Library, the preparation of scans rata, foreign-language abstracts etc.) to be reused has been commissioned mainly to this institution, in the electronic edition of the bibliography. Plain after obtaining formal permission to use the re- text version was used to expedite the task of ex- sulting electronic documents on the project site, traction of individual fields of each entry regard- ultimately available to the general public. Since
29 the project duration was relatively short, the scans material, media-historical or language-historical were produced in batches to let contributors start observations, translations of then common Latin work on the previous portion when the following interjections and translations of foreign texts into one was still in preparation (which sometimes re- Polish or local dictionaries. Such approach seems quired cleaning the microfilm plates or seeking novel in the design of digital libraries. What is improperly catalogued collection). Similarly, all more, the system creates possibility to form a liv- other project phases (metadata proofreading, im- ing thematic information exchange environment port and conversion of scanned materials, prepa- around the library site, making it possible to store ration of commentaries and dictionaries etc.) were expert comments, versions of materials etc. also being carried out simultaneously. To make the scope of description complete and Among many available repository systems for up-to-date, a survey of the library holdings has digital libraries, including a prominent (in Poland) been carried out. The annual volumes of library dLibra (dLibra, 2010), EPrints has been selected professional journals published by ten major sci- as the target storage and publication framework. entific libraries in Poland has been investigated EPrints (EPrints, 2010) is a free, GPL-licensed and 80 new objects (not included in Zawadzki’s multiplatform repository software developed since bibliography) discovered. Since the project was 2000 at the University of Southampton. Its open short of funds to generate their scanned versions, source origin presents a major benefit for projects only their metadata were added to the library. intending not only to deploy it “as is”, but seek- The last phase of the project was preparation of ing possibilities to extend it with new solution- the supplemented electronic edition of Zawadzki’s specific functionalities. The system has been in- bibliography which illustrates how digital content stalled, configured and in most respects translated can help maintain traditional publications. The into Polish (with translations made available to the electronic version was intended to be published, community), setting up the print repository. but cuts in the initial project budget forced the As already stated above, an important organi- team to leave this additional step to future projects. zational assumption made at the beginning of the The new publication layout has been created project was to use EPrints also as a collabora- in LateX. To facilitate usage, in contrast to the tion platform, starting from the very first phases original work, separation into three volumes was of the work. Following this statement, the bibli- not preserved and a single volume was produced. ographical entries (henceforth, metadata descrip- Introductions, lists of printers and print shops as tions) were imported into EPrints even before they well as back matter illustrations were taken over were finally proofread. This stage has been car- from the OCR-ed source and collated. All materi- ried out already in the system, using the workflow als transferred to the digital library have been ex- defined for the project. After metadata has been ported into XML format and were included in the revised against the image (or paper) version of the final edition. This means that all supplements and bibliography, the scanned images of prints were errata which were merged with the library objects converted into DjVu format and uploaded into the were automatically corrected. Back matter content library (each object corresponding to one file). such as lists of titles, people names and geograph- ical names were not transferred from the original Versions of objects (most likely translations or work but were regenerated from the library. alterations of the base text) have been linked bas- All scripts and transformations created through- ing on information available in the bibliography out the project were preserved to facilitate genera- or detected by the experts. For materials only re- tion of new electronic editions of the bibliography ported in literature, when the original is unknown in case new errors are reported by the library users or not preserved, the information on the base lan- or new materials included. This also demonstrates guage has been provided. new collaborative approach to preparation of elec- Going beyond simply making the objects avail- tronic publications able as electronic versions of the bibliographic en- tries with scanned copies, the repository model 4 Library interface, statistics and usage has been extended with new fields storing the new value created by the project: historical commen- The repository is located on the server of the Insti- taries relating to facts and people described in the tute of Journalism of the University of Warsaw and
30 Figure 2: Library homepage
the library has been made available at http:// of recently revised objects replacing the standard cbdu.id.uw.edu.pl1. The site (see home- list of recently added. page on Fig. 2) allows typical browse and search Currently the library holds 2009 objects. 1404 interfaces in Polish or English. Objects can be objects have scans attached (with 11 585 pages browsed according to multiple criteria, includ- in total). 11 languages are represented. The lan- ing those normally used in bibliographical de- guages with widest coverage (over 50 objects) are: scriptions (mostly borrowed from Zawadzki), as German (797 objects), Polish (325 objects – see well as those less typical: thematic, genre-related Fig. 3), Italian (180) and Latin (69); the remain- and other. The native EPrints advanced search is ing ones are Swedish, French, Spanish, English, supplemented with a recently customized simple Dutch, Czech and Danish. Around 200 prints in search with a Google-like single text field. Polish and 50 in German have attached dictionar- Prints relating to the same facts are hyperlinked, ies explaining currently unused words or phrases especially those that are most likely or certainly (giving explanations in contemporary Polish or translations from other texts available in the li- German). Latin dictionaries are also included. brary. Taking into account the number of identi- The final version of the library has been made fied dependencies and benefits resulting from pos- official early 2010, so it has been more than half a sibility of their visual comparison, the system of- year since its resources can be used for interested fers a function to open related documents in a parties and some initial findings from observed us- split window. Moreover, the new implementations age of the library resources can be obtained. Data include enhanced inter-metadata linking capabili- from March-November 2010 show that the library ties, new facets for repository browsing and a list hosted 34 unique visitors daily (47 visits) and is 1 The default language of the interface is Polish; it can stabilizing; approx. 40% of the open pages are be changed to English by selecting the second menu option (“Ustaw jezyk”) and then the first entry on the language list DjVu files (not indexed by popular search engines (English – “angielski”). which should imply human visitors).
31 Figure 3: A sample object
The library has been included into the network ceeded the scope of the project). Other project of Polish Digital Libraries Federation by means of which can benefit from the library resources, cur- The Open Archives Initiative Protocol for Meta- rently being carried out by the Institute of Polish data Harvesting (OAI-PMH, 2002) with metadata Language at the Polish Academy of Sciences is the represented in Dublin Core (ISO, 2009). OAI- dictionary of the historical Polish from 17th and PMH is natively supported by EPrints, so no ad- the first half of 18th century (see SXVII, 2010). ditional implementation was necessary. Another interesting direction is broadening of To advertise the library among the general pub- the scope of materials the library covers. Since the lic a series of dissemination events have been or- project has been closely related to the resources of ganized. They included radio broadcasts (univer- The Polish National Library, it can be extended to sity radio Kampus, Polish news radio TOK FM), cover materials coming from other libraries, most newspaper articles and historical portal news. likely from abroad, both preserving originals and storing their microfilmed copies. With the possi- 5 Further steps and conclusions bility of preparation of copies on site, the costs Recently the library contents found new applica- should not significantly differ from the costs of the tions: it is currently used in the EU-co-funded national project. Starting with Zawadzki’s bibliog- IMPACT project for training Gothic script OCR raphy, there are still around 200-300 objects to be software by ABBYY (the producer of FineReader acquired this way. The geographical key seems an with XIX module for recognition of Fraktur or important factor here: the limitations imposed on “black letter” texts). Simultaneously the tran- Zawadzki before 1989 resulted in underrepresen- scribed versions are being prepared and are tation of resources from the libraries of the coun- planned to be included in the library (respective tries belonging to the former Soviet Union. Last metadata fields were defined in the project, but but not least, Vatican archives can prove to be one only one transcription was filled in since it ex- of the most important source of materials of the
32 described type, with so far limited availability. Zawadzki, K. (1990). Gazety ulotne polskie i Polski Apart from obvious development ideas such as dotyczace z XVI, XVII i XVIII wieku (Polish and Poland-related Ephemeral Prints from the 16th-18th widening the scope of description of gathered ob- Centuries , in Polish). Wroclaw: Zaklad Narodowy jects or deepening the analysis, the idea of creating im. Ossolinskich, Wydawnictwo Polskiej Akademii a collaboration platform of the digital library site Nauk. can be followed. For instance, the system can be extended with new functionalities provided to the library users (not just experts with editing rights) such as adding comments to materials or an inte- grated forum. Such add-ons can prove similarly efficient in development of the materials and in re- lated projects. Despite its small scale, we trust that our li- brary will prove equally useful for old-Polish researchers as much larger heritage accessibil- ity projects such as Europeana (see http:// www.europeana.eu) or ENRICH (European Networking Resources and Information concern- ing Cultural Heritage, see http://enrich. manuscriptorium.com/).
References dLibra (2010). Digital Library Framework . Poznan Su- percomputing and Networking Center affiliated to the Institute of Bioorganic Chemistry of the Polish Academy of Sciences.
EPrints (2010). EPrints free software . Southampton: School of Electronics and Computer Science at the University of Southampton.
Gruszczynski W., Ogrodniczuk M. (2010). Cyfrowa Biblioteka Drukow Ulotnych Polskich i Polski doty- czacych z XVI, XVII i XVIII w. w nauce i dydaktyce (Digital Library of Poland-related Old Ephemeral Prints in research and teaching , in Polish). In Pro- ceedings of “Polskie Biblioteki Cyfrowe 2010” ( Pol- ish Digital Libraries 2010) conference, Poznan, 18- 22 October 2010.
ISO 2009: ISO 15836:2009. Information and docu- mentation – The Dublin Core metadata element set .
OAI-PMH (2002). The Open Archives Initiative Protocol for Metadata Harvesting . Protocol Version 2.0 of 2002-06-14, Document Version 2008- 12-07. http://www.openarchives.org/OAI/open- archivesprotocol.html.
SVXI (1987). Slownik polszczyzny XVI w. (Dictionary of 16 century Polish , in Polish). Krakow: Instytut Badan Literackich PAN.
SVXII (2010). Slownik jezyka polskiego XVII i 1. polowy XVIII w. (Dictionary of 17 century and 1st half of 18 century Polish , in Polish). Warszawa: Polska Akademia Nauk, Instytut Jezyka Polskiego.
33 Rule-Based Normalization of Historical Texts
Marcel Bollmann Florian Petran Stefanie Dipper Ruhr-University Bochum bollmann,petran,[email protected]
Abstract proach clearly outperforms the baseline. However, there is still room for some improvement. This paper deals with normalization of The paper is organized as follows. Sec. 2 dis- language data from Early New High Ger- cusses related work; in Sec. 3, we introduce our man. We describe an unsupervised, rule- data. Sec. 4 addresses the way we derive rewrite based approach which maps historical rules from the data, while Sec. 5 deals with the wordforms to modern wordforms. Rules application of the rules to generate modern word- are specified in the form of context-aware forms. Sec. 6 presents the evaluation, Sec. 7 the rewrite rules that apply to sequences of conclusion. characters. They are derived from two aligned versions of the Luther bible and 2 Related Work weighted according to their frequency. The evaluation shows that our approach Baron et al. (2009) present two tools for normal- (83%–91% exact matches) clearly outper- ization of historical wordforms. VARD consists forms the baseline (65%). of a lexicon and user-defined replacement rules, and offers an interface to edit and correct automat- 2 1 Introduction1 ically normalized wordforms. DICER is a tool that derives weighted context-aware character edit Historical language data differs from modern data rules from normalized texts. The algorithm that in that there are no agreed-upon, standardized creates these rules is not described in their paper, writing conventions. Instead, characters and sym- though. bols used by the writer of some manuscript in parts Jurish (2010) compares different methods to reflect impacts as different as spatial constraints or normalize German wordforms from 1780–1880. dialect influences. This often leads to inconsistent The methods include mappings based on pho- spellings, even within one text written up by one netic representations and manually created rewrite writer. rules. The highest F-score (99.4%) is achieved by The goal of our research is an automatic map- an HMM (Hidden Markov Model) that selects one ping from wordforms from Early New High Ger- of the candidates proposed by the other methods. man (ENHG, 14th–16th centuries) to the cor- Research on normalizing historical language responding modern wordforms from New High data has also been done in the field of Information German (NHG). Enriching the data with modern Retrieval, applied to historical documents. They wordform annotations facilitates further process- address the task reverse to ours: mapping modern ing of the data, e.g. by POS taggers. wordforms to historical (or dialectal) wordforms. In this paper, we present a rule-based approach. Ernst-Gerlach and Fuhr (2006) run a Given an input wordform, (sequences of) char- spellchecker on their data (19th century Ger- acters are replaced by other characters according man) to detect wordforms that differ from NHG to rules that have been derived from two word- wordforms, and to generate normalized wordform aligned corpora. The results show that our ap- candidates. Context-aware rules that rewrite 1We would like to thank the anonymous reviewers for character sequences are derived from the pairs. helpful comments. The research reported here was fi- They achieve an F-score of 60%. nanced by Deutsche Forschungsgemeinschaft (DFG), Grant DI 1558/4-1. 2http://corpora.lancs.ac.uk/dicer/
34 Proceedings of Language Technologies for Digital Humanities and Cultural Heritage Workshop, pages 34–42, Hissar, Bulgaria, 16 September 2011. ENHG AM anfang schuff Gott Himel vnd Erden [...] VND Gott schuff den Menschen jm zum Bilde / NHG Am Anfang schuf Gott Himmel und Erde [...] Und Gott schuf den Menschen ihm zum Bilde , EN In the beginning created God heavens and earth and God created the man himin the image ENHG zum Bilde Gottes schuff er jn / Vnd schuff sie ein Menlin vnd Frewlin . NHG zum Bilde Gottes schuf er ihn ; und schuf sie ein Mannlein¨ und Fraulein¨ . EN in the image of God created he him and created them a male and female Table 1: The original ENHG and the modernized NHG version of Genesis 1:1 and 1:27, along with an English interlinear translation.
Hauser and Schulz (2007) use a corpus from 3 TheCorpus ENHG and a dictionary from NHG to derive re- In our approach, replacement rules are derived placement rules of (sequences of) characters. To from word-aligned parallel corpora. A source that this end, they first assign ENHG wordforms to provides a parallel corpus in many languages, in- NHG dictionary entries, using Levenshtein dis- cluding historical ones, is the bible. tance to pick suitable candidate entries; word- The collected works of Martin Luther are freely forms with ambiguous assignments are excluded. available from the web.3 They include several ver- The rule frequencies are used as weights for the sions of his bible translation, modernized to vary- rule applications. For generating ENHG lem- ing degrees. We chose the original ENHG version mas from NHG lemmas, they achieve F-Scores of the 1545 bible (which has been enriched with between 56.9% (without weights), 66.2% (with modern punctuation) as well as a revised NHG weights derived from lexicon mappings), and version of it, which uses modern spelling and re- 71.2% (with perfect training pairs). places extinct words by modern ones.4 Strunk (2003) uses weighted Levenshtein dis- Table 1 shows text fragments in both versions. tance to generate dialectal wordform variants for Differences concern capitalization (AM – Am, an- IR of Low Saxon texts. Weights are manually de- fang – Anfang), character reduplication (schuff – fined and encode phonetic similarity. schuf, Himel – Himmel), deletion (Erden – Erde), Our approach is similar to Ernst-Gerlach and insertion, or replacement (Frewlin – Fraulein¨ ). Fuhr (2006) and Hauser and Schulz (2007) in Compared to other texts from that time, the lan- that we derive replacement rules of character se- guage of Luther’s 1545 bible is rather close to quences from aligned pairs. The algorithms to NHG, since the evolution of NHG was heavily learn rules differ, though. Ernst-Gerlach and Fuhr influenced by Luther’s bible translation (Besch, (2006) specify recursive definitions that take into 2000). Furthermore, printed texts in general show account rewrite sequences and contexts of varying more consistent spelling than manuscripts, and use sizes; rules can refer to characters or to the under- abbreviations to a lesser extent than manuscripts specified classes ‘vowel’ and ‘consonant’. Hauser (Wolf, 2000); no abbreviations occur in Luther’s and Schulz (2007) extract n-gram sequences of 1545 bible. Hence, we expect that our approach varying size from the aligned wordforms, and can be transferred and applied to other printed learn n-gram mapping rules. Furthermore, our ap- texts more easily than to manuscripts. proach differs from theirs in that our training pairs stem from aligned corpora. Alignment The files contain one bible verse per line. A verse usually corresponds to one sen- The evaluations cannot be easily compared be- tence; some verses, however, consist of more than cause it is not clear to what extent the language one sentence. Sometimes the assignment of sen- data base is comparable with regard to its varia- tences or phrases to verses had been changed from tion. The data from Jurish (2010) contains 59.2% the original to the modernized version to an as- identical word pairs (types), the data from Ernst- signment that is presumably closer to the original Gerlach and Fuhr (2006) 94%. In our data, only 3 51% of the pair types align identical words. For instance: http://www.sermon-online.de. 4The original 1545 version is incorrectly called “Alt- Furthermore, in our task, a historical form is deutsch” (‘Old German’) in the archive. Our NHG text most often mapped to one modern equivalent (which is possibly the 1892 revision) is a rather conserva- tive version, while the 1912 and 1984 revised versions also form; in the reverse task, a modern form is mapped contain corrections of mistranslations by Luther and, hence, to multiple historical variants. deviate from the original ENHG text to a greater extent.
35 Greek version. The verses were rectified manu- Type Example Freq Eval ally so that each version had the same number of 1. Unproblematic vnd – und ‘and’ 609 77% verses. A few OCR mistakes were also manually 2. Differing truncken – trunkenen 261 18% inflection ‘drunken’ corrected (e.g. 3ott was replaced by Gott ‘god’). 3. Differing oben – obenan ‘ontop’ 40 5% The entire corpus was then tokenized using the to- affixes kenizer script supplied with the Europarl corpus 4. Closer modern noch – weder ‘nei- 0 – form exists ther/nor’ (Koehn, 2005), and afterwards downcased. 5. Extinct form stundlin¨ - an dem 1 – Since there were still asymmetries in the as- Tage ‘on that day’ signment of phrases and sentences to the verses 6. Incorrect zwey ‘two’ – fur¨ ‘for’ 25 0% 7. Unclear cases fur ‘for’ (?) – fur¨ ‘for’ 64 19% between the versions, the resulting corpus was not yet properly aligned. We applied a sentence aligner to our data, the Gargantua toolkit (Braune Table 2: Alignment types: types 1–5 are correct to and Fraser, 2010). Next, the words of each aligned various degrees, type 6 is incorrect. For each type, verse pair were aligned using the GIZA++ toolkit its frequency in the sample and evaluation results (Och and Ney, 2003). for the more frequent types are given (see Sec. 6). The modernization often involves transforming words into phrases and vice versa, such as soltu – bly not be learned. Type 4 and 5 pairs (which oc- sollst du ‘should you’, or on gefehr – ohngefahr¨ cur very rarely) could be used to derive mappings ‘approximately’ (literally: ‘without danger (of of entire words rather than character sequences. saying so)’). Hence, it is crucial that the word Type 6 alignments clearly constitute noise. aligner can handle 1:n/n:1 alignments. Upon man- We further computed the number of target types ual inspection, we found a lot of misalignments a source word maps to. The pie chart in Fig. 1 with rarer tokens, many of which involve num- shows that the vast majority of source types map bers, such as zweiundzwanzig ‘twenty-two’, which to only one target type, with a significant minority would actually correspond to the multi-word to- mapping to two forms. The proportion of source ken zwey vnd zwenzig in the original 1545 version. types mapping to 4–8 target types was so negligi- These were probably misaligned because their fre- ble that they were merged in the pie chart. The quency in the corpus is not high enough to train a quantity of source tokens that map to more than translation model. one target type on the other hand is pretty large. To minimize noise in our system’s input, align- Even though the majority still has 1–2 mappings, ment pairs with a length difference of more about 20–30% of source tokens map to more than than five characters were excluded from fur- 5 target types, as the central bar plot of Fig. 1 ther processing. Since the two texts are highly shows. However, if we exclude targets that oc- similar—around 65% of the pairs align identical cur only five times or less, the graph shows a more wordforms—a length difference of that magnitude balanced picture (right bar plot). Since the appli- rarely leads to meaningful alignments. cation of rules is based on their frequencies, it is highly likely that the impact of the infrequent rules Some corpus statistics To assess the quality of is balanced out by the dominant ones. the resulting word pairs, we had a small sample of 1,000 pairs of aligned non-identical wordforms Procedure The resulting corpus consists of from the evaluation corpus manually inspected by 550,000 aligned pairs of words or phrases. We a student assistant. We identified six types of randomly picked 20% of the alignment pairs for alignments, listed in Table 2.5 Instances that were a development corpus, which was used for the de- difficult to classify are assigned to a special class. velopment of the rule extraction and application For deriving replacement rules, type 1 align- processes described below. Another 40% were af- ments are the perfect input. Pairs of type 2 and 3 terwards used to extract the replacement rules for are still useful, to a certain extent: correct rules can the following experiments (= training corpus), and be derived from the word roots; mapping of differ- a final 20% were picked for an evaluation corpus. ing inflection and affixes, however, should proba- 4 Normalization Rules 5Throughout the paper, the examples are taken from the development corpus while the figures are calculated based on We used a modified algorithm for Levenshtein dis- the evaluation corpus. tance which not only calculates the numerical edit
36 Training Evaluation Training 1 37836 Evaluation 2 70335 3 54846 4−8 50295 122929
85.67% 20705 17345
0.9% 28861
1.79% 26896 Source tokens (%) Source tokens Source tokens (%) Source tokens 11.63% 10542 36724 8574 14955 14391 13264 6369 11810 5257 9154 10129 19520 17269 3344
Target types per source types 5581 2169 2116 0 0 0 2553 230 0 0 5 10 15 20 25 30 0 10 20 30 40 50 60 1 2 3 4 5 6 7 8 9 10 14 1 2 3 4 5 Number of target types Number of target types Figure 1: Target types per source type/token. The pie chart shows the result for source types (evaluation corpus), the central bar plot shows the results for all source tokens, the right bar plot only considers tokens with a frequency > 5 (training and evaluation corpus). The columns are labeled by absolute frequencies. distance, but also outputs the respective edit opera- the problem that there may be multiple optimal tions (substitution, insertion, and deletion of single paths. In fact, this situation arises quite often. characters) that map the strings to each other. The Ex. (2) shows two optimal paths/alignments for record of edit operations was enriched with infor- the pair jrem – ihrem ‘their’.6 mation about the immediate context of the edited Input j r e m character. Ex. (1) shows two sample edit opera- (2) a. Operations s + = = = tions, using the notation of phonological rewrite Output i h r e m rules. Input j r e m b. Operations + s = = = (1) a. ε h / j r Output i h r e m → (‘h’ is inserted between ‘j’ and ‘r’) The ambiguity is obviously of no consequence b. v u / # n for the numerical distance, which is two for both → (‘v’ is replaced by ‘u’ between the left cases, but it makes a big difference for the rules. word boundary (‘#’) and ‘n’) In particular, it is usually very clear that one of the alignments is the “correct” one (reflecting facts Determining the context for these edit opera- about language change, here: Ex. (2a)), while the tions is not straightforward, because applying one other one is an implementation artifact (Ex. (2b)). rule can change the context for other rules. Since the Levenshtein implementation applies the rules Rule sets and sequence-based rules To solve from left to right, we decided to use the (new) tar- the ambiguity problem, we first pick a random get word for the left context and the (unaltered) path by preferring substitution over deletion over source word for the right context (see also Fig. 2). addition on the cell level in the matrix. The rules derived this way are combined to rule sets after- In addition to canonical replace- Identity rules wards. This is done by inspecting the positions ment rules, our rule induction algorithm also pro- in the source and target word where the edits are duces identity rules, i.e. rewrite rules that map a made. Whenever a series of edits occurs at the character to itself. Identity rules reflect the fact same target or source position, we assume that that the majority of words remain unaltered when this is actually an insertion or deletion of a se- mapped to their modernized forms, and many quence of characters, such as an affix. When- words are modified by few characters only. Iden- ever edits occur at adjacent positions, we assume tity rules and actual rewrite rules are intended to that it is a substitution of a character sequence by compete with each other during the process of another. By merging substitutions with adjacent rewriting. deletions/additions, we account for character se- Multiple optimal paths Since the dynamic pro- quences of variable length on each side of the rule. gramming algorithm works by determining a 6Operations: ‘+’ means insertion, ‘–’ deletion, ‘s’ substi- least-cost path through a matrix, we are faced with tution, ‘=’ identity.
37 flad e r n have been generated, an epsilon identity rule is
# e r r e n . . . inserted between each pair of non-insertion rules, i.e. at each position where no insertion has taken zyp r e sse n place. The epsilon identity rules are taken to mean that no insertion should be performed in the re- Figure 2: The graph illustrates the actual map- spective context, thereby restricting the actual in- pings derived from fladernholtz – zypressenholz sertion of characters. ‘cypress wood’. The central row shows the rule Rank Frequency Rule contexts and how they are determined. = 1 24,867 ε ε /n # = 2 18,213 ε → ε / e r → In the example jrem – ihrem above (Ex. (2)), = 3 18,200 ε ε / e n = 4 17,772 ε → ε /# d the first two rules would be merged since they are = 5 14,871 ε → ε /r # = 6 14,853 n → n / e # derived from adjacent non-identity edit operations. → This leads to the correct solution to the multi-path s 20 8,448 v u/# n → problem, shown in Ex. (3). - 176 1,288 f ε /u f + 239 932 ε→ l/o l → Input j r e m 156 1,443 j ih/# r (3) Operations s = = = 272 796 j → ih/# n Output ih r e m 329 601 j →ih/# m 605 263 ε → d/t u Note however that the rule merging does not al- 879 142 ss→ ß/o e → ways come up with entirely correct alignments. For the word pair fladernholtz – zypressenholz Table 3: Sample rankings and rules ‘cypress wood’, optimal alignment would map flader zypresse/# n whereas the actual → Ranking of the rules Applying the rule induc- algorithm outputs smaller mappings, see Fig. 2. tion algorithm to the development corpus yielded Indiscriminate merging of identity rules would re- about 1.1 million rule instances (training corpus: sult in highly specific mappings of entire words 2.2 million) of about 6,500 different types (train- rather than character sequences—hence, identity ing: 7,902). These were sorted and ranked ac- rules are never merged. cording to their frequency. Table 3 lists sample Epsilon identity rules In the system developed instances of rules. Not surprisingly, none of the so far, insertion rules are rather difficult to han- top-ranked rules modifies the input word. Rank 6 dle. In generating modern wordforms by means is taken by the first rule that maps some real char- of the rewrite rules (see Sec. 5), they tend to ap- acter rather than ε to itself (identity rules, ‘=’). ply in an uncontrollable way, garbling the words in Rank 20 features the first substitution rule (‘s’), the process. This is due to the fact that the condi- etc. The bottom part of the table lists frequent tions for the application of insertion rules are less sequence-based rules. The rule ranked 605th maps specific: while substitutions and deletions require the empty string to a whitespace followed by ‘d’. the left hand side (LHS) and the context to match This rule applies in mappings such as soltu – sollst in the source word, insertions are constrained by du ‘should you’. their two context characters only, since the LHS of 5 Generating Normalized Wordforms the rule is empty. At word boundaries, the prob- lem gets even worse, since only one context side Normalizing ENHG texts is done on a word-by- is specified here. Furthermore, substitution and word basis, i.e. the input of the normalizing pro- deletion rules compete against the identity rules cess is always a single wordform. Words are pro- for their LHS, which reflects the fact that the ma- cessed from left to right; for each position within jority of characters are not changed in a mapping a word, applicable edit rules are determined. As to the modernized form—but no similar competi- with the rule extraction process, the left context tor exists so far to curb the application of insertion is taken from the output already generated up to rules. that point, while the right context is always taken We therefore introduced epsilon identity rules: from the input word. If a rule is applied, its right- after all replacement rules for an alignment pair hand side is appended to the output string, and the
38 next character from the input word is processed. Original bible words Generated words The process continues until the end of the word Old Modernized Best-rule Best-prob. has been reached. 1. vnd und X X Rules with sequences of characters on the left- jrem ihrem X X hand side (LHS) are applicable at the position of vmbher umher X X 2. wetter wetter X *dieuetter their first LHS character. In that case, if the rule is krefftige kraftige¨ X *krefftige applied, processing continues with the next char- vrteil urteil X *urteill acter that is not part of the LHS. 3. fewr feuer *feur X soltu sollstdu *soltu X 4. ermanen ermahnen *ermanen *ermannen Epsilon rules Epsilon rules, and epsilon identi- zween zwei *zween *zwen ties in particular, require special consideration to work within this system. If an epsilon identity is Table 4: Example mappings (prior to the dictio- applied, no other epsilon rules should be allowed nary lookup) from the development corpus. ‘X’ for the same context, otherwise insertions would means that the word is generated correctly by the not be blocked. To achieve this, epsilon is treated respective method, ‘*’ marks incorrectly gener- like an ordinary letter with regard to the LHS, and ated wordforms. Words listed under 1. are gener- words are preprocessed so that exactly one ep- ated correctly by both methods, words under 4. by silon is placed between each character and at word neither of them, the rest by one of the methodes. boundaries. For example, if the input word is jrem ‘theirs’, it is converted internally to the following disadvantage: as the left context for a rule depends form: on the output of the previous one, applying one (4) # ε j ε r ε e ε m ε # rule can result in different rules being applicable at later positions in the word. If we always apply Now, if an epsilon rule is applied, the read/write the most frequent rule, this can create situations head moves to the next character, thereby ensur- in which only very low-frequent rules are applica- ing that no other epsilon rule can be applied at the ble later, indicating that the resulting wordform is same time in the same position. This also prevents unlikely to be correct. Also, when applying a rule application of multiple insertion rules but at the with a sequence of characters on the LHS, replace- same time still permits insertion of multiple char- ment rules that modify any but the first character acters, since those are merged into sets during rule of that sequence are never even considered. extraction. Of course, epsilon characters have to For this reason, we came up with another be ignored for all purposes except for the LHS of method that takes into account the frequencies of replacement rules; in particular, they do never in- all applied rules across the whole word. fluence rule contexts or prevent the recognition of (ii) ‘Best-probability’: In the second approach, character sequences on the LHS. each generated variant is assigned a probability score. We define the probability of a replace- Ranking methods For each character and its ment rule as its frequency divided by the sum context within a word, there will usually be a num- of all rule frequencies. The word probability is ber of applicable rules to choose from. As our aim calculated from the probabilities of the rules that is to generate exactly one (modernized) form for were used to generate it; for this, we used the each input word, a decision has to be made about weighted harmonic mean, with the length of the which rule to apply. Two approaches were tested: LHS as weights. If the LHS contains a sequence, (i) selecting the rule which had the highest fre- length is counted including additional epsilons be- quency during rule acquisition (‘best-rule’); and tween each character. This way, all variants gen- (ii) selecting the word with the highest probabil- erated from the same input word have the same to- ity score (‘best-probability’), which is calculated tal weight, regardless of whether sequence-based based on rule frequencies. rules were used or not. (i) ‘Best-rule’: In the first approach, if more Table 4 lists sample mappings as they result than one rule application is possible at a given po- from both methods. sition, we simply select the rule with the highest frequency. The evaluation shows that this method Dictionary lookup Quite often, the highest- already works quite well, however, it also has a ranked rules generate non-existing words. This
39 Total Identical Tokens NLD Count Count Ratio Mean SD ± All 109,972 71,163 64.71% 0.1019 0.1630 ± Old bible text Identical 71,163 71,163 100.00% 0.0000 0.0000 (Baseline) Non-identical 38,809 0 0.00% 0.2889 ± 0.1460 Unknowns 2,911 1,190 40.88% 0.1215 ± 0.1359 ± All 109,972 91,620 83.31% 0.0408 0.1039 Identical 71,163 70,467 99.02% 0.0018 ± 0.0198 Best-rule method Non-identical 38,809 21,153 54.51% 0.1122 ± 0.1483 Unknowns 2,911 1,390 47.75% 0.1081 ± 0.1357 ± All 109,972 92,172 83.81% 0.0396 0.1041 Best-probability Identical 71,163 69,358 97.46% 0.0042 ± 0.0279 method Non-identical 38,809 22,814 58.79% 0.1046 ± 0.1509 Unknowns 2,911 1,255 43.11% 0.1201 ± 0.1407 ± All 109,972 100,074 91.00% 0.0251 0.0913 Best-probability + Identical 71,163 70,854 99.57% 0.0008 ± 0.0126 dictionary method Non-identical 38,809 29,220 75.29% 0.0697 ± 0.1423 Unknowns 2,911 2,238 76.88% 0.0646 ± 0.1428 ± Table 5: Evaluation of exact matches and normalized Levenshtein distance (NLD) compared to the modernized bible text; separately for all tokens (All), tokens that are or are not identical in both old and modernized version (Identical/Non-identical), and tokens that were not seen in the training corpus (Unknowns). The best result for each class is indicated in bold, the second-best in bold italics. can be avoided by combining the methods de- cal first step of an evaluation is to check how many scribed above with a dictionary lookup. For this, words from the ENHG text from 1545, when pro- all variants are generated and then matched against cessed with our algorithms, exactly match their a dictionary. From all variants that are covered by modernized NHG counterparts. Before normaliza- the dictionary, the one with the highest score (best- tion, the ratio of identical tokens in the historical rule or best-probability) is selected as the output and the modernized text is about 65%, i.e., only form. If no variant can be generated in this way, a third of all wordforms even differ at all. This the input word is left unchanged. The dictionary is the baseline for our algorithm; any normalizing used here consists of all wordforms from the mod- process that results in less than 65% exact matches ernized NHG Luther bible.7 has likely done more harm than good, and it would be better to leave all words unchanged. Table 5 6 Evaluation shows that both ranking methods we employed, For evaluation, we generated normalized forms of best-rule and best-probability, achieve match ra- all words in the evaluation corpus and compared tios above 83% (lines ‘All’, column ‘Ratio’), them to their aligned forms in the modernized which is a significant increase from the baseline; bible text. Two methods of comparison were ap- combining the best-probability method with the plied: (i) counting the number of identical word- dictionary lookup even yields 91% exact matches. forms; and (ii) calculating the average normalized Our normalization approach is not only successful Levenshtein distance (NLD). Full evaluation re- in changing historical forms to modern ones, but sults are shown in Table 5. Results for the dictio- also in correctly leaving most of the wordforms nary method are only reported in combination with unchanged that do not need to be changed (97.46– the best-probability method, which clearly outper- 99.57%; lines ‘Identical’, column ‘Ratio’). form the combination with the best-rule method. Results from evaluating the dictionary method on the annotated sample set of non-identical word Exact matches As our aim is to generate mod- pairs are shown in Table 2. Although the sam- ernized wordforms from historical ones, the logi- ple size is very small, the numbers suggest that 7Using a dictionary with wordforms from current newspa- our approach is mostly suitable for pairs of type 1 per texts turned out problematic, since modern abbreviations, (besides identical word pairs); in particular, it can typos, etc., result in too many false positives with the dictio- nary lookup. Moreover, the vocabulary of newspaper texts only ‘repair’ some inflection or affixes (types 2–3). differs considerably from religious texts. Incorrect mappings (type 6) are not produced.
40 Total Identical Tokens NLD a slight improvement for the best-rule method, Count Count Ratio Mean SD while the best-probability method has even in- ± Best- 18,352 696 3.79% 0.2483 0.1368 creased the average NLD. With the latter method, ± rule 18,352 0 0.00% 0.2443 0.1226 a high percentage of mismatches (10.14%) results ± Best- 17,800 1,805 10.14% 0.2372 0.1495 from source words that did not need to be changed. prob. 17,800 0 0.00% 0.2447 ± 0.1297 ± Combined with the dictionary lookup, the ratio 9,898 309 3.12% 0.2876 0.1528 of such cases is considerably reduced (to 3.12%). Dict. 9,898 0 0.00% 0.2789 ± 0.1479 ± Even the dictionary method is not able to com- pletely avoid superfluous modifications. The rea- Table 6: NLD evaluation of word pairs that do not son is that there are ambiguous words such as match their aligned word after normalization. The waren, which can either be mapped to wahren first line of each method shows the NLD before ‘true’ or left unchanged: waren ‘were’. This rule application, the second line afterwards. means that at the type level it cannot be decided which of the two mappings is correct. Instead, we would need context information at the token level, Normalized Levenshtein distance However, which is beyond our word-based approach.10 simply counting correct guesses does not take into account near-misses, where the algorithm edited Method comparison Even though the results only a part of the word correctly, and is also not for both ranking methods are quite close when a very fine-grained way of evaluation. There- evaluated on all word pairs (83.31% versus fore, we chose to use normalized Levenshtein dis- 83.81%), the difference is statistically significant tance (Beijering et al., 2008) to assess whether within a confidence level of 99.9%, i.e., the best- our normalized variants are ‘closer’ to the correct probability method results in better normalizations modern version than the (non-normalized) source on average. This is especially reflected in the words. The NLD of a word pair is defined as the numbers for the non-identical word pairs, where Levenshtein distance divided by the length of the the difference between the two methods is even longest possible alignment of the two words.8 To greater. On the other hand, the best-rule method evaluate a set of word pairs, we calculate the aver- performs better on ‘unknowns’, i.e. words which age NLD of all word pairs in that set. This mea- were not already part of the training set (see Ta- sure is not ideal for our task, since short words ble 5). This seems to indicate that a combination are unduly penalized for being wrong, but it has of both methods could be favorable. the advantage of being relatively intuitive; e.g., a The overlap of the word lists generated by the NLD of 0.5 indicates that roughly every second two methods is 93.13%, showing that there is a no- character in a word was altered. ticeable percentage of words (around 3%) which Comparing the old and the modernized bible is modernized correctly with one method but not text yields an average NLD of 0.1019; as two the other. One crucial difference is the normaliza- thirds of all words are already identical, this num- tion of second person verb forms ending in –tu, ber is quite low. The three normalization methods e.g. soltu ‘should you’, which should be modern- reduce that number to 0.0408–0.0251; this reduc- ized to sollst du. These forms show a contrac- tion stems, in parts, from the higher match ratio. tion of verb and pronoun and are quite common in To evaluate whether the normalization improved the original ENHG bible text. With the best-rule the wordforms even if they do not exactly match method, they do not get changed at all, as the ep- the aligned form, we re-evaluated average NLD silon identity rules are ranked higher than the ones on the sets of words that did not result in an exact that would perform the necessary insertions. The match. The results, given in Table 6,9 show only best-probability method, on the other hand, out- puts the correctly modernized form; rules that pro- 8As a side effect of our rules induction process, we can cess the letter u appearing word-final after t have easily calculate the number of alignment slots, since it equals a very low probability, thus decreasing the total the total number of edit and (non-epsilon) identity rules. probability of the unmodified wordform. 9Words considered here include (i) words from identical word pairs that unnecessarily have been modified (this, e.g., Finally, combining the methods with a dictio- concerns 696 words with the best-rule method) and (ii) words nary lookup results in remarkable improvements. from non-identical word pairs that have not been normalized successfully (best-rule: 18,352–696 = 17,656 words). 10Jurish (2010) takes context information into account.
41 The number of exact matches increases from determine the significance of the association be- 83.31% to 88.66% (best-rule), and from 83.81% tween a rule and its context, and to potentially ab- to 91.00% (best-probability). As can be seen from stract the rules from their specific contexts. Table 5, the dictionary lookup on the one hand Another open question is the handling of multi- helps to avoid superfluous normalization (with tokens as source words. Since currently the rule identical word pairs). On the other hand, it also application operates on single words, multiple improves on rule application, by filtering out rules source tokens can never be merged to a single (or combinations of rules) that do not result in sen- target token, even though that happens quite fre- sible words. In contrast to rules, which operate quently in our texts. locally, the dictionary lookup has access to the en- tire wordform and thus serves as a complementary References “guide” for suitable rule selection. Alistair Baron, Paul Rayson, and Dawn Archer. 2009. Since we use the complete modernized Luther Automatic standardization of spelling for historical bible as the source of our dictionary, all correctly text mining. In Proceedings of Digital Humanities. normalized wordforms are guaranteed to be listed Karin Beijering, Charlotte Gooskens, and Wilbert in the dictionary. In this sense, the results repre- Heeringa. 2008. Predicting intelligibility and per- sent an upper bound of this method. However, the ceived linguistic distances by means of the Leven- shtein algorithm. Linguistics in the Netherlands, larger a dictionary is, the higher will be the chance pages 13–24. of “false friends”, i.e. wordforms that accidentally Werner Besch. 2000. Die Rolle Luthers fur¨ die deut- match a generated wordform. sche Sprachgeschichte. In Werner Besch et al., edi- tor, Sprachgeschichte. Ein Handbuch zur Geschichte 7 Conclusion der deutschen Sprache und ihrer Erforschung, pages 1713–1745. de Gruyter, 2nd edition. We showed that using only unsupervised learning, Fabienne Braune and Alexander Fraser. 2010. Im- a minimum of knowledge engineering, and freely proved unsupervised sentence alignment for sym- available resources, it is possible to map histori- metrical and asymmetrical parallel corpora. In Pro- ceedings of COLING, Poster Volume, pages 81–89. cal wordforms to their modern counterparts with a high success rate. Even the simplest implemen- Andrea Ernst-Gerlach and Norbert Fuhr. 2006. Gen- erating search term variants for text collections with tation of the process performs far better than the historic spellings. In Proceedings of the 28th Euro- baseline, a success that we were able to further im- pean Conference on Information Retrieval Research prove upon. (ECIR 2006). Springer. Open issues include the multi-path problem, Andreas W. Hauser and Klaus U. Schulz. 2007. Unsu- which is still not entirely solved, despite the intro- pervised learning of edit distance weights for retriev- duction of sequence-based rules (see Section 4 and ing historical spelling variations. In Proceedings of the First Workshop on Finite-State Techniques and especially Fig. 2). There are a number of potential Approximate Search, pages 1–6. solutions that we could pursue. The rule extrac- Bryan Jurish. 2010. More than words: Using to- tion process could be modified to output all possi- ken context to improve canonicalization of histori- ble paths from the source to the target word. This cal German. Journal for Language Technology and would require some means to decide on the most Computational Linguistics, 25(1):23–39. plausible path, such as the total number of rules af- Philipp Koehn. 2005. Europarl: A parallel corpus ter the single-character rules have been merged to for statistical machine translation. In Proceedings of MT Summit. sets. Phonetic or even graphemic similarity (such as the common substitutions of u and v) could also Franz Josef Och and Hermann Ney. 2003. A sys- tematic comparison of various statistical alignment be taken into account. models. Computational Linguistics, 29(1):19–51. Another issue would be to replace our simple Jan Strunk. 2003. Information retrieval for languages heuristic of sequence determination by the use of that lack a fixed orthography. Seminar Paper, Stan- an association measure, such as the log-likelihood ford University. http://www.linguistics. ratio or the Fisher-Yates test, to determine which rub.de/˜strunk/LSreport.pdf. rules are merged to sequences. This could include Norbert Richard Wolf. 2000. Handschrift und Druck. identity rules in the sequences, which might solve In Werner Besch et al., editor, Sprachgeschichte. Ein Handbuch zur Geschichte der deutschen Sprache problems such as the one presented in Fig. 2. und ihrer Erforschung, pages 1705–1713. de Association measures could be further used to Gruyter, 2nd edition.
42 Survey on Current State of Bulgarian-Polish Online Dictionary
Ludmila Dimitrova Ralitsa Dutsova Rumiana Panova IMI-BAS Veliko Tărnovo University and Veliko Tărnovo University and Acad. G. Bonchev St bl. 8 IMI-BAS Master Program IMI-BAS Master Program 1113 Sofia, Bulgaria Sofia, Bulgaria Sofia, Bulgaria [email protected] [email protected] [email protected]
not need any necessary hardware on the local Abstract computer or any installation of necessary soft- ware. The only condition is that the user's com- The dictionaries are among the well- puter be equipped with a web browser. This is known tools for applications in everyday why online dictionaries are so easy to distribute life, education, sciences, humanities, and and use. A programmer of such software can eas- human communication. The recent de- ily and promptly correct any potential shortcom- velopments of information technologies ing that arises, since the application is installed contribute to the design and creation of on a web-server. Another advantage of online new software tools with a wide range of dictionaries is the possibility of changes in their applications, especially for natural lan- content such as deletion or addition of new dic- guage processing. The paper presents an tionary entries. online bilingual dictionary as a techno- The first Bulgarian-Polish online dictionary logical tool for applications in digital was designed to be a general purpose dictionary humanities, and describes the structure oriented to the casual user and be available by and content of the bilingual Lexical Da- open access via the Net. Authorized users can be tabase supporting a Bulgarian-Polish provided with the ability to include within entries online dictionary. It focuses especially on links to other entries, and to update or edit easily the presentation of verbs, which form the online (through the correction of eventual mis- richest from a specific characteristics takes, or the addition of new entries or new in- viewpoint linguistic category in Bulgar- formation about headwords). ian. The main software modules for web- For the realization of these purposes a bilin- presentation of this digital dictionary are gual lexical database (LDB) supporting such a also shortly described. web-based dictionary and ensuring a good search system has to be developed. Besides, whenever 1 Introduction possible the LDB should automatically generate a new (whether a single or multiple) structure/s Recent developments in information technology of entries for the Polish-Bulgarian dictionary us- have been successfully implemented in natural ing the appropriate information from a Bulgar- language processing, producing numerous tools ian-Polish entry. with a wide range of applications. Digital dic- The building of bilingual digital dictionaries is tionaries, being large-scale data repositories, are a complex and difficult process, due to the scar- a popular tool for applications in everyday life, city of formal models that adequately reflect the education, social sciences and digital humanities. specific linguistic features of a given natural lan- Actually, every dictionary contains a large guage. The lexical database has been chosen as a amount of language data, but a digital one con- technological platform to support the Bulgarian- tains incomparably more because it is a dynamic Polish online dictionary for free presentation and collection of dictionary entries and has the poten- open access in the Internet. tial for infinite growth: new entries can be added The design of the Bulgarian-Polish LDB fol- without limitation. lows the CONCEDE model for dictionary encod- All kinds of digital data are now accessible ing with some extensions and modifications. The from remote computers via the Net. Online dic- project CONCEDE 1 has built lexical databases tionaries freely published in Internet are accessi- ble to every user through a URL-address. In or- der to use this kind of dictionary, the user does 1 CONCEDE INCO-Copernicus project no. PL96-1142 Consortium for Central European Dictionary Encoding
43 Proceedings of Language Technologies for Digital Humanities and Cultural Heritage Workshop, pages 43–50, Hissar, Bulgaria, 16 September 2011. in a general-purpose document-interchange for- because the translation correspondences in Polish mat, for the six Central and East European lan- are different. The index of type "Function" guages: Bulgarian, Czech, Estonian, Hungarian, counts the groups of grammatical functions that Romanian, and Slovene. correspond to a particular part of speech of the The project has produced lexical resources specified Bulgarian word. that respect the guidelines for encoding diction- For example, for the following entry aries (Ide and Véronis, 1995) and so are com- приятелски adi. przyjacielski; adv. po patible with other TEI-conformant resources. przyjacielsku The LDB model offers a standardized hierar- chical tree-structure of a dictionary entry with a two structures of type “Function” are created. understandable semantics. It is formally de- The first structure (index n=”1”) represents the scribed in (Erjavec et al., 2000, 2003). grammatical function adjective of a headword The first LDB for Bulgarian was developed “приятелски”/friendly/ (part of speech is under the CONCEDE project. It contains more adjective), and the second structure (index than 2700 lexical entries (Dimitrova et al., 2002) n=”2”) represents the grammatical function prepared in accordance with encoding standards adverb (part of speech is adverb): established by the TEI (Text Encoding Initia-
44 son, tense, mood, itype, as these all have their карт’инна ~я
hw: the headword; used for alphabetization and
45 To represent the Bulgarian verbs more ade- 4 Relational Database of the Bulgarian- quately, in Bulgarian-Polish LDB new content Polish Online Dictionary tags were added: The lexical database serves as the basis for de- • to represent the conjugation of verbs - signing the relational database which is the initial the 46 ory required when reading large XML docu- line dictionary. The dictionary is at an experi- ments compared to the SAX technology. mental stage and intended for research purposes, The DOM parser for transforming the LDB of but it will also be widely applicable to the con- the Bulgarian–Polish dictionary into RDB is pro- trastive studies of Bulgarian and Polish, in a sys- grammed in Java. In this way it can be run on tem for human and machine translation, as well different platforms independent of the architec- as in education. ture or the operating system. Future implementation will include some “search” functions with a query, where the 6 Online dictionary – Brief Description search parameters are fixed and which as a result will extract and show to the user the relevant in- The Bulgarian–Polish online dictionary is real- formation from the Bulgarian-Polish LDB – dic- ized by the web-based application supporting by tionary entry (entries), for example, to show the Bulgarian-Polish LDB and RDB. This web- transitive Bulgarian verbs, or Bulgarian adjec- based application is experimental, and the struc- tives that serve also as adverbs. ture of the text fields is not permanently deter- mined. References The implementation of the web-based applica- tion is based on the following technologies: Andreychin et al. 1994. Bulgarian Explanatory Dic- Apache, MySQL, PHP and JavaScript. These are tionary /Dictionary of the Bulgarian Language. 4th free technologies originally designed for devel- revised edition, prepared by D. G. Popov/ Nauka i oping dynamic web pages with greater function- Izkuvstvo Publishing House, Sofia, 1994 (In Bul- ality. garian) The web-based application consists of two Dimitrova, L., Pavlov, R., Simov, K. 2002. The Bul- main software tools: administrator and end-user garian Dictionary in Multilungual Data Bases. module (Dimitrova et al., 2009b). The adminis- Journal Cybernetics and Information Technolo- trator module serves to create the database and gies. 2 (2): 12-15, 2002 update the dictionary. Access to the administra- Dimitrova, L., Koseska-Toszewa, V., Satoła- tive module is restricted to authorized users (so Staśkowiak, J. 2009a. Towards a Unification of the called administrators) (Table 7 in Appendix). Classifiers in Dictionary Entry. In Garabík (Ed.), After logging onto the system the administrator Metalanguage and Encoding Scheme Design for has possibilities to access the database and to Digital Lexicography. Bratislava, 48-58, 2009 enter new entries, headwords, classifications, or Dimitrova, L., Panova, R., Dutsova, R. 2009b: Lexi- to edit/delete existing ones. cal Database of the Experimental Bulgarian-Polish The web-based end-user interface is bilingual. online Dictionary. In Garabík (Ed.), Metalanguage The user can choose the input language (Bulgar- and Encoding scheme Design for Digital Lexicog- ian or Polish) with possibilities to search for raphy. Bratislava, 36-47, 2009 translation in both directions Bulgarian-to- Erjavec, T., Evans, R., Ide, N., Kilgarriff, A. 2000. Polish, or Polish-to-Bulgarian. The Bulgarian-to- The Concede model for lexical databases. In Pro- Polish translation will display the whole informa- ceedings of the 2nd International Conference on tion existing in the dictionary entry but the oppo- Language Resources and Evaluation, LREC'00, site translation will be made only from the main Athens, ELRA, 2000 senses of the Bulgarian headwords (Figure 4). Erjavec, T., Evans, R., Ide, N. Kilgarriff, A. 2003. Next, an example shows how the Bulgarian From Machine Readable Dictionaries to Lexical verb повярвам /believe/ is inserted in the data Databases: the Concede Experience. In Proceed- base through the administrative module of the ings of the 7th International Conference on Com- web application (Figure 2) (especially the infor- putational Lexicography, COMPLEX'03, Budapest, mation about its transitivity, semantic features Hungary, 2003 and conjugation type), and further, how this in- Ide, N., Véronis, J. 1995. Encoding dictionaries. In formation is displayed on the screen to the end- Ide, N., Veronis, J. (Eds.) The Text Encoding user (Figure 3). (The Figures 2 – 4 are shown in Initiative: Background and Context. Dordrecht: the Appendix.) Kluwer Academic Publishers, 167-179, 1995 Sławski, F. 1987. Podręczny słownik Bułgarsko- 7 Conclusion and Future works Polski z suplementem. PW „Wiedza Powszechna”, Warszawa, 1987 The paper presents briefly the Bulgarian-Polish LDB that supports the first Bulgarian-Polish on- 47 Appendix: Figure 1. Relational database upon the LDB of the Bulgarian-Polish online dictionary Figure 2. Administrative panel – 1st step of inserting of a Bulgarian verb Figure 3. Web presentation for end users - translation from Bulgarian to Polish 48 Figure 4. Web presentation for end users - translation from Polish to Bulgarian Field Comments id Id homonym_index Index of the homonym (if null, no homonym exists) bg_word Bulgarian headword suffix Suffix plural Plural form for a noun is_plural_rare Frequency of usage of the plural form for a noun (null – normal, 0 - often, 1 – rare) conjugation Conjugation form for a verb (2 p., present) conjugation_type Type of conjugation for a verb (1, 2 or 3) has_gender Whether a noun has feminine and neuter gender gender_feminine Feminine gender form for an adjective gender_neuter Neuter gender form for an adjective id_explanation Foreign key to “explanation” id_bg_word Id of the referent Bulgarian word referent_bg_word Referent Bulgarian word Table 1. Presentation of the Bulgarian headwords Field Comments id Id id_bg_word Foreign key to “bg_word" functional_homonym_index Index of the functional homonym group Table 2. Functional homonymy of the Bulgarian headwords Field Comments id Id id_bg_word Foreign key to “bg_word” example Example of the headword type Type of the usage (1 - Derivation; 2 - Phrase; 3 - Example) pl_translation Polish translation id_explanation Foreign key to “explanation” Table 3. Derivations, phrases or examples of the Bulgarian headwords and their translation in Polish 49 Field Comments id Id id_bg_word_functional_homonym Foreign key to “bg_word_functional_homonym” pl_word Polish headword sense_index Index of the sense alternative_sense_index Index of the alternative sense latin_translation Latin translation of the word id_explanation Foreign key to “explanation” Table 4. Presentation of the Polish headwords Field Comments id Id id_pl_word Foreign key to “pl_word” example Example in Polish id_explanation Foreign key to “explanation” Table 5. Examples of the Polish headwords Field Comments id Id explanation Explanation Table 6. Explanations of the headwords, derivations, phrases and examples Field Comments id Id username Username password Password first_name Name last_name Family name Table 7. Administrative users’ authorization 50 Language Technology Support for Semantic Annotation of Iconographic Descriptions Kamenka Staykova Gennady Agre IICT, BAS, Bulgaria IICT, BAS, Bulgaria [email protected] [email protected] Kiril Simov Petya Osenova IICT, BAS, Bulgaria IICT, BAS, Bulgaria [email protected] [email protected] Abstract teresting example of TEL in humanities. The scenario requires an intensive use of multimedia The paper describes an approach for semantic objects stored in existing heterogeneous digital annotation of multimedia objects implemented libraries. for the purposes of SINUS Project1. Semantic In the SINUS environment a TEL-oriented ap- annotations are supported by semantic annota- plication is created hierarchically, starting by tion models based on ontological presentation converting an autonomous system for storing and of knowledge concerning Bulgarian Icono- retrieving a multimedia data (digital library) to a graphy. The process of semantic annotation in- cludes automated data-lifting procedure and Web service, then transforming this service into user-directed approach. The paper pays atten- a semantically-oriented digital library facilitated tion to a specific variant of the semantic an- by Web services and ontologies, and finally, ex- notation process directed by the user - applica- tending the library into a learning system based tion of Language Technologies for semi-auto- on service oriented architecture. mated creation of semantic text annotations The current paper presents the processes of se- (tags) based on analysis of descriptive texts. mantic annotation of multimedia objects (MO) The ‘ontology-to-text’ approach has been ad- implemented in the SINUS environment. Section apted to the needs of the iconographic domain. 2 describes the basic decisions taken for organiz- Initial experiments are established to support ing such annotations. Section 3 presents the first the user during the process of manual semantic annotations in the context of SINUS environ- attempts to apply language technologies in order ment. to develop a user-directed approach for semi- automatic creation of annotations. Section 4 dis- 1 Introduction cusses the future work. The main objective of the research project SI- 2 Semantic Annotation of Multimedia NUS is to provide a semantic technology-based Objects in SINUS Project environment facilitating development of Techno- logy-Enhanced Learning (TEL) applications, The domain of Bulgarian Iconography is a fruit- which are able to reuse existing heterogeneous ful field to show how different multimedia docu- software systems. The SINUS environment has a ments (the digital photos of iconographic works, service oriented architecture allowing unified texts, video records, etc.) could be used in TEL representation and use of heterogeneous systems applications. The multimedia resources for SI- as Web services. The environment is tested on a NUS demo-examples come from the Multimedia use case, which applies the basic TEL principles Digital Library “Virtual Encyclopedia of East- to the process of Learning-by-Authoring Christian Art” described in (Pavlova-Draganova (Dochev and Agre, 2009). The domain of Bul- et al., 2007) and marked as “the Library” from garian Iconography is chosen for constructing a here on. Its content is accessible via a special SINUS Project scenario, since it provides an in- Web service developed in the SINUS environ- ment. 1 “Semantic Technologies for Web Services and Techno- logy Enhanced Learning” (SINUS) sinus.iinf.bas.bg 51 Proceedings of Language Technologies for Digital Humanities and Cultural Heritage Workshop, pages 51–56, Hissar, Bulgaria, 16 September 2011. 2.1 Resources of Semantic Annotation linked. For example, the root ontological concept of SINUSBasic Ontology is Iconographical Ob- The Objects of semantic annotation in SINUS ject. Such concepts as Author, Iconographical project are multimedia objects presenting in- School, Collection are used as root-concepts in formation in digital form about icons, wall-paint- SINUSSpec Technology ontology. ings, miniatures and other iconographical works; Basic Semantic Annotation Model (Basic SAM) also pictures and different texts concerning the is presented in the picture bellow. iconographical works; information about authors, places, dating periods, religious characters and so op Author on. The Library uses a fixed annotation schema t c n Title e op o j i t dp b for organizing all the resources and the available a O c i l op f i a t Iconographical School c n data. In order to allow more flexible and deep i op e h d p I a op r op reasoning about the iconographical knowledge, g o op n Object Location op y o g c the SINUS semantic space extends that schema I o op l o n Object Location n o h i to present the knowledge in a formalized, onto- t c (Creation) p e i n r T o c i logy-like manner. t op s e p op l i dp r D a i c dp r Object Date e s e The Ontologies. SINUSBasic Ontology is the e s e t s a a D a B B M main conceptual model of the SINUS semantic op e s op a e e B Dimensions u space. The fixed annotation schema of the Lib- u q op q i i n e op n e t h n h n c a c e rary is taken as a ground for creating this onto- o t Identification Notes i c e e dp t S r T T p S i e l l l r n t a a c a o c logy, in order access to be provided to the Lib- i c c s c t i a i i i r e h h h d a p D p p n h a a a rary from an upper, semantic level. However, the o r r r C C g g g o o o n n n o SINUSBasic Ontology itself (with minor excep- o o c c c I I tions) is created following the main principles of I the standard SIDOC-CRM (Crofts et al., 2010). Some of the links between concepts represent The SINUSBasic Ontology is implemented in object properties, others – datatype properties. OWL and comprises 58 classes, 38 object prop- Some of the object properties are realized as erties and 28 data-type properties. Main classes chains of 2 or 3 properties. Many of the datatype are: Iconographical Object with its sub-classes properties lead to textual data providing access to Icon, Wall-Painting, Miniature, Mosaic, Vitrage the descriptive texts collected in the Library. and so on, Author, Iconographical Scene, Char- Extended Semantic Annotation Model (Exten- acter, Iconographical Technique, Base Material ded SAM) adds 14 new features to the Basic and so on. SAM of Iconographical Object individuals. All The SINUS semantic space contains the so called these additional features are supported by SI- “specialized ontologies”, which encode experts’ NUSSpec Technology ontology as properties. knowledge on particular aspect of the Bulgarian For example, such features are: base_has_com- Iconography domain. It is assumed that special- ponent, gilding_has_type, laquering_has_even- ized ontologies represent additional, more spe- ness, primer_has_filler, etc. In this way the in- cialized domain knowledge that is not contained stances of Iconographical Object, class defined in the “basic” Library. For example, the special- in SINUSBasic ontology, is linked to concepts of ized ontology on religious characters gives ac- Primer, Gilding, Lacquering, Filler, etc., defined cess to such notions as Canonical Character, in SINUSSpec Technology ontology. Apostle, Hierarch, etc., the specialized ontology Semantic Repository. SINUS environment em- on iconographical technology gives access to no- ploys SESAME RDF Semantic Repository that tions as Soft Material, Solid Material, Lacquer- provides sufficient reasoning and standard func- ing, Resin, Primer, Plaster, etc. At the current tionalities of semantic repositories for realizing stage of work the SINUSSpec Technology onto- the SINUS scenario. All repository functionalit- logy is implemented in OWL and could be ies are accessible through the SINUS User Inter- loaded into SINUS semantic space on demand. face. The ontology contains 16 classes, 14 object prop- erties and 45 ontological individuals. Some con- 2.2 Search Process cepts of the SINUSSpec Technology ontology The semantic annotation of MO in SINUS is or- represent extensions of concepts introduced in ganized as a two step process: at the first step a SINUSBasic ontology, and in this way basic do- MO of interest should be found, and at the main ontology and specialized ontologies are second step the desired new annotations should 52 be added (manually or semi-automatically) to the semantic annotation model of MO contains sev- object description. Semantic search of multime- eral links to descriptive texts concerning the MO. dia objects starts with preparing a “natural lan- For example, each individual of the class Base of guage”-like query, which is constructed on the SINUSBasic Ontology is connected through the base of described above SINUS ontologies and datatype property has_Base_Description to the presented in user-friendly graphical way. The particular text kept in the Library. An example of query is automatically transformed into SPAR- a short text describing the base of a particular QL form, which is sent to the Extended Search Iconographical Object is given bellow. Engine – a special component of the SINUS en- BG: Основата е от иглолистна дървесина с vironment responsible for searching the informa- два кошака, добре запазена. Гипсов tion in the SINUS repositories. The component грунд, нанесен тънко и равномерно. “lowers” the corresponding part of the query to EN: The base is of softwood with two the Library and then “lifts” the answer represen- keys, well kept. Plaster ground coat, ted at the semantic level to the semantic reposit- applied thinly and evenly. ory, where the whole SPARQL query is ex- ecuted. Practically, during this data lifting pro- Most of the descriptive texts contain a lot of ter- cess some data from the Library is transformed minological notions of a particular domain and to several SINUSBasic Ontology individuals that many of the terms are defined in the correspond- are added to ontologies stored in the semantic re- ing specialized ontologies. The main idea of pository. The search result, which usually is a set semi-automatic semantic annotations is to help of (identifiers to) multimedia objects, is presen- substantially the user in his/her attempt to annot- ted to the user via the SINUS User Interface. ate MO with notions presented in Extended Se- mantic Annotation Model. The support consists 2.3 Semantic Annotation Process of access to preliminary created semantically an- Additional semantic annotations of MO made notated texts, which makes some (ontological) by the user are also supported. This user-directed notions visible and sensitive, and also “technic- semantic annotation process allows the user to ally” prepared to be used further in the process of add some new (specialized) annotation features used-directed semantic annotation. to existing MO annotations or to create “basic” The (preliminary) semantic annotations of texts annotations for a new MO. The extension to the are created off-line and stored in such a way that basic annotation model is supported by the SI- they can be seen as indexes to MO and used for NUSSpec Technology ontology presented above. on-line searching and retrieving the objects. The The process of user-directed semantic annotation text annotation procedure is implemented as a has the following steps enabled by the SINUS special Web service accessible from the SINUS User Interface: environment. The output of this process is a set 1. All properties of a concrete object selected by of XML files, so in order to use them in the SI- the user are displayed. The number of properties NUS environment they have to be accessible depends on special ontologies the user is going to during the on-line process of creating new an- use for creating the annotations. Each property notations. The annotations (tags) in the texts are could be displayed with particular value (known treated as parts of preliminary semantic annota- annotation) or the value could be still unknown. tions of particular MO. They could be acknow- In such case, a list of possible values of the prop- ledged, extended or denied by the user during the erty (stored in the corresponding ontology) is semantic annotation process. The annotations proposed as options to the user. suggested in texts are shown to the user as “de- 2. The user can either change a displayed value fault” values of the corresponding properties of of a selected property (if this annotation has been the MO. SINUS platform has to be equipped created earlier by him or semi-automatically) or with a special procedure that “translates” the an- the user can create a new annotation by selecting notated text into form of Extended Semantic An- a value from the corresponding list, if the current notation Model. value of this property is empty. The task of text annotation could be formulated 3. After completing the annotation process and in this way: given an ontology and text, return the user can save the new annotations in the SI- annotated text, which is sensitive to the ontolo- NUS semantic repository. gical notions. This general task is known as On- Opportunities for semi-automatic semantic an- tology-to-Text relation and is still a research notation by use of descriptive texts analysis. The challenge in the crossroad of Language Techno- 53 logies and Semantic Technologies. Language ideally considered as an extension of a general Technologies operate with specific methods and language deep grammar which is adapted to the tools to annotate text documents semantically. concept annotation task. Minimally, the concept annotation grammar consists of a chunk grammar 3 Semantic Text Annotation for SINUS for concept annotation and (sense) disambigu- Project ation rules. The following picture demonstrates this part of the Ontology-to-Text relation. Semantic text annotation presented here is based on a model of Ontology-to-Text relation de- veloped within (Simov & Osenova, 2007; Simov & Osenova, 2008). Ontology-to-Text relation is defined with the help of two intermediate com- ponents: (terminological) lexicon and concept annotation grammar. The lexicon plays twofold role. First, it interre- lates the concepts in the ontology to the lexical knowledge used by the grammar in order to re- cognize the role of the concepts in the text. Second, the lexicon represents the main interface between the user and the ontology. This interface allows the ontology to be navigated or represen- The chunk grammar for each term in the lexicon ted in a natural for the user way. For example, contains at least one grammar rule for recogni- the concepts and relations might be named with tion of the term. As a preprocessing step we con- terms used by the stakeholders in their everyday sider annotation with grammatical features and activities and in their own natural language. This lemmatization of the text. The disambiguation could be considered as a first step to a contextu- rules exploit the local context in terms of gram- alized usage of the ontology in a sense that the matical features, semantic annotation and syn- ontology could be viewed through different tactic structure, and also the global context, such terms depending on the context. For example, the as topic of the text, discourse segmentation, etc. material names will vary from very specific Currently we have implemented chunk grammars terms within the domain of iconography to more for Bulgarian and English. common names used when a set of icons are ex- For the implementation of the annotation gram- hibited to a wider audience. As the image depicts mar we rely on the grammar facilities of the it, the lexical items contain the following inform- CLaRK System (Simov et al., 2001). The struc- ation: a term, contextual information determining ture of each grammar rule in CLaRK is defined the context of the term usage, grammatical fea- by the following DTD fragment: tures determining the syntactic realization within the text. In the current implementation of the lex- icons the contextual information is simplified to two values, common term and others. Each rule is represented as a line element. The rule consists of regular expression (RE) and cat- egory (RM = return markup). The regular expres- sion is evaluated over the content of a given XML element and could recognize tokens and/or annotated data. The return markup is represented as an XML fragment which is substituted for the The second component of the Ontology-to-Text recognized part of the content of the element. relation, the concept annotation grammar, is Additionally, the user could use regular expres- 54 sions to restrict the context in which the regular tem. The actual annotation is done by the follow- expression is evaluated successfully. The LC ele- ing format: ment contains a regular expression for the left context and the RC for the right one. The element 55 Acknowledgments The research is supported by the National Sci- ence Fund of Bulgaria under the project No. D- 002-189. 56 The Tenth-Century Cyrillic Manuscript Codex Suprasliensis: the creation of an electronic corpus UNESCO project (2010–2011) Hanne Martine Eckhoff David J. Birnbaum Anisava Miltenova Tsvetana Dimitrova University of Oslo University of Pittsburgh Bulgarian Academy of Bulgarian Academy of Sci- Kanonhallveien 10e Department of Slavic Sciences ences 0585 Oslo Languages and Litera- Institute for Literature Bulgarian Language Insti- h.m.eckhoff@ifikk. tures 52 Shipchenski prohod tute uio.no 1417 Cathedral of am- 52 Shipchenski prohod Learning [email protected] [email protected] [email protected] plies complications in its philological interpreta- Abstract tion; see also Abicht and Schmidt, 1896). In section 2, this paper presents information This paper presents an overview of principles about the content, condition, and history of the and problems connected with the preparation manuscript. Section 3 reviews efforts in digitiza- of an electronic edition of the largest Old tion of the manuscript, and section 4 discusses Church Slavonic manuscript, the Codex Su- previous electronic editions of the deciphered prasliensis, in the context of a project funded text, reviewing problems with representation and by UNESCO. Specifications of the manu- script, its history, and previous paper-based availability and solutions adopted by the editors. and electronic editions are discussed, together Section 5 gives an overview of the principles of with a strategy for the preparation of a com- application of morphosyntactic annotation condi- plete digital edition, including newly acquired tioned by the chosen annotation tool and strat- digital images, electronic text, analysis and egy. The conclusion in section 6 explores distinc- commentaries, parallel Greek text, and up- tions among the publication of a text, digitization dated bibliography. In particular, our paper of a manuscript, development of language cor- sheds light on automating the morphosyntactic pora, and a true electronic edition of the text, annotation of the text and the difficulties that which is the goal of the UNESCO project. had to be resolved in this part of the project. 2 The Manuscript 1 Introduction The Codex Suprasliensis is a Cyrillic manuscript, The UNESCO-funded project The Tenth-Century arguably copied at the end of the tenth or the be- Cyrillic Manuscript Codex Suprasliensis aims at ginning of the eleventh century (Krǎstev and Bo- digitizing the largest Old Church Slavonic manu- jadžiev, 1999). It is the largest extant Old Church script, the Codex Suprasliensis Slavonic manuscript and it is associated with the (http://csup.ilit.bas.bg/). Preslav literary school. This early Cyrillic manuscript has been dated The Codex contains twenty-four vitae of to the end of the tenth or the beginning of the Christian saints for the month of March and eleventh century and has been published three twenty-three homilies for the triodion cycle of times on paper (Miklošič, 1851; Severjanov, the church year. In content it is a lectionary me- 1904; Zaimov and Capaldo, 1982–83). The most naeum (or panaegyricon), combined with homi- recent of these, the two-volume edition by Zai- lies from the movable Easter cycle, most of mov and Capaldo (1982, 1983), was published which written by or attributed to John Chryso- more than two decades ago and contains photo- stom (http://csup.ilit.bas.bg/node/7). graphic images of the entire manuscript; a tran- According to most researchers, the Miscellany scription reproduced from Severjanov, 1904 and was not translated as a stable compilation from corrected (not entirely without error) against the any single Byzantine menological or facsimile; and a Greek text (compiled from mul- hagiographical manuscript. Rather, it was com- tiple Byzantine sources, which necessarily im- 57 Proceedings of Language Technologies for Digital Humanities and Cultural Heritage Workshop, pages 57–61, Hissar, Bulgaria, 16 September 2011. piled from texts translated at different times, long simile is an interim stage in the project, and the before the compilation of the Codex Supraslien- photographs will eventually be republished to- sis. Presumably, at least one of the sources was gether with a transcription that will be fully an- the Glagolitic Epiphanius homily. Folio 104v of notated, accompanied by commentary and up- the manuscript has a marginal note that reads dated bibliography. g(ospod)i pomilui retъka amin (‘Lord have Some previously unknown source materials, mercy on Retъk. Amen’), and some researchers including some Byzantine originals identified have suggested that Retъk is the name of a only after the publication of the Zaimov and Ca- scribe. paldo edition in the early 1980s, have been used The language of the manuscript follows the in the preparation of the Greek text of the new Preslav literary norm of the tenth century. It is edition. considered the most representative source of lin- Eventually a diplomatic transcription of the guistic information about canonic Old Church text of the Codex Suprasliensis will be published Slavonic because of its size and because it con- together with critical apparatus, parallel Greek tains texts otherwise unattested in the early me- text, vocabulary, and grammatical analysis (in diaeval Slavic tradition. The codex is, thus, the the form of corpora annotation). The annotation main source for studying the language, writing, of the electronic corpus is at initial stage, with and culture of Bulgaria during the Preslav period. only one piece, namely the Life of St. Paul the The Codex Suprasliensis is written on parch- Simple, completely annotated, and another (the ment and shows careful writing and craftsman- Life of St. Paul and St. Juliana) under active ship. It was discovered in 1823 in a Uniate Basil- preparation. ian monastery in Supraśl (then in Lithuania, now in Northeastern Poland in the Podlaskie Voi- 4 Electronic text vodeship) by Canon Michał Bobrowski. Bo- browski sent it for study to the Slovenian scholar The principles of manuscript description follow a Jernej Kopitar. After Kopitar’s death, the first proposal developed in the context of The Reper- 118 folios were donated to the University Li- torium of Old Bulgarian Literature and Letters, brary in Ljubljana, where they are still kept. The which includes descriptions, in both English and following 16 leaves were purchased by A. F. By- Bulgarian, of some 350 mediaeval Slavic manu- čkov in 1856 and are now kept in the Russian scripts dated from the eleventh to the beginning National Library in St. Petersburg. The remain- of the eighteenth century. The Repertorium was ing 151 leaves were part of the collection of the designed in conformity with important standards Counts Zamoyski. The last, so-called Warsaw and guidelines in humanities computing (Mil- part had disappeared during World War II and tenova, Boyadzhiev, and Velev, 2000; Birnbaum, were long considered lost until re-emerging in 1996). The description and analysis of the Cyril- the US. In 1968, those folios were returned to lic manuscripts contain comprehensive data Poland, where they are now part of the manu- drawn de visu from old texts script collection of the National Library in War- (http://clover.slavic.pitt.edu/repertorium/). saw. The first electronic version of the Codex Su- The Codex Supraslianis has been listed in prasliensis was a 7-bit ASCII transliteration pre- UNESCO’s Memory of the World Register since pared under the direction of Jouko Lindstedt and 2007. distributed by the Corpus Cyrillo-Methodianum Helsingiense: An Electronic Corpus of Old 3 Digitization Church Slavonic Texts (CCMH, http://www.helsinki.fi/slaavilaiset/ccmh/) and the In the present project, digital images of all three TITUS project (http://titus.uni- parts of the Codex Suprasliensis, currently lo- frankfurt.de/texte/etcs/slav/aksl/suprasl/supra.ht cated in repositories in three different countries m). These transcriptions contain numerous errors (the National Library in Warsaw, Poland; the and come completely without context and critical National Library of Russia in St. Petersburg; and apparatus (no images, Greek text, commentary, the National University Library in Ljubljana, grammatical annotation or analysis, etc.). The Slovenia), were reunited for publication in a sin- new edition under development takes the Hel- gle electronic edition. The digital images are al- sinki transcriptions as a starting point, converts ready available at http://csup.ilit.bas.bg/galleries. the text from ASCII to Unicode, corrects the er- The separate publication of the photographic fac- 58 rors, and includes the full range of supporting grated part of the PROIEL parallel treebank of materials listed above. ancient Indo-European languages. The core of A pilot model of an electronic edition of a the treebank is the New Testament in its Greek small part of the Codex Suprasliensis with a original and its earliest translations into each of search program was developed in 2008 (Birn- the other project languages. PROIEL features an baum, 2008) at the University of Pittsburgh electronic version of the Codex Marianus fully (http://paul.obdurodon.org). This electronic edi- annotated for morphology, syntax, and various tion of the Life of St. Paul the Simple was devel- other linguistic features. It has also been auto- oped in accord with the procedures and priorities matically aligned with the Greek Gospels at to- described above: it is based on a corrected ver- ken level (Eckhoff and Haug, 2010). sion of the text published by the CCMH, accom- Test annotation of the Codex Suprasliensis is panied by parallel Greek (from the Zai- currently in progress. Observations and solutions mov/Capaldo edition), a new English translation, discussed in this section of the paper were drawn detailed linguistic commentary, and photo- from the process of annotating of the Life of St graphic facsimiles. Linguistic analysis in the Paul the Simple and the Life of St. Paul and St. commentary conforms to notation developed in Juliana (the annotated text is currently available Oscar Swan’s Old Church Slavic Inflectional at: http://foni.uio.no:3000/ ). Morphology (2008). The PROIEL annotation tool (available at the There are many collections and editions of same site) was developed with certain needs in classical and mediaeval texts (such as the Per- mind: seus Project, When confronted with novel text styles and http://www.perseus.tufts.edu/hopper/), but most orthographical conventions (different from the of them are manually annotated. No rule-based already annotated Codex Marianus), annotation morphological guesser is currently available for initially is primarily manual, but it becomes in- Old Church Slavonic, partially because of trou- creasingly automatic as the tool learns from op- blesome orthography, although there is prelimi- erator input. Because the annotation for some nary finite-state morphology under development languages, including Old Church Slavonic, is by Roland Meyer (http://rhssl1.uni- being performed on a diplomatic transcription of regensburg.de:8080/OCS/). a text with substantial orthographic variation (ra- The research project Pragmatic Resources in ther than on the normalized texts that are used Old Indo-European Languages (PROIEL), more commonly in other disciplinary philologi- which aims at developing morphosyntactic cal traditions), morphological analyzers and syn- means for the annotation of and research into the tactic parsers are not available for all of the pro- information structure in Ancient and Hellenistic ject languages. Greek, Latin, Gothic, Classical Armenian, and Annotators had to be recruited internationally Old Church Slavonic (Haug and Jøhndal, 2008), due to the specialized knowledge required. The has developed a statistical morphological guesser application was, therefore, built to work with and a semi-manual syntactic annotation tool sup- standards-compliant browsers, which did not ported by a set of morphology-based rules. The require the annotators to perform any extra in- corpus to be built for the electronic edition of the stallation. For the annotators of Old Church Sla- Codex Suprasliensis will be annotated manually, vonic texts, the tool supports transliterated input, but with the assistance of the morphological obviating the need for a specialized keyboard guesser already developed by the PROIEL pro- layout interface. ject and trained for Old Church Slavonic mor- Texts are imported in a simple XML format, phology on the Codex Marianus (Haug et al., where they are split into tokens (words) based on 2009). Thus, the Codex Suprasliensis will be an- spacing, and roughly into sentences based on notated for morphology, syntax, and other fea- punctuation. After the import and coarse auto- tures in the PROIEL annotation interface, and the matic segmentation, the annotation proceeds as information will be exported in XML for incor- follows: poration into the projected electronic edition. First, there is adjustment of sentence division. Since punctuation is not a reliable guide to sen- 5 Morphosyntactic Annotation tence division in Old Church Slavonic, sentences must often be split or merged. The morphosyntactic annotation tool to be used Second, the imported tokenization must be in the Codex Suprasliensis project is an inte- checked and corrected manually. A linguistic 59 analysis of the text may need to normalize the In addition to the morphosyntactic annotation, word boundaries of the edition. In particular, there is an interface for annotating information contractions of prepositions and nouns may need status and anaphoric relations. There is also an to be dissolved. option for customized tagging at the token, lem- Third, morphological annotation and lemmati- ma, and sentence level. This option has been zation are implemented. The PROIEL annotation used to tag semantic features (such as animacy), tool provides guesses for morphological features derivational morphology (such as prefixation), and lemmata based on previous reviewed annota- and textual features (such as direct speech). tions (Haug et al., 2009). The annotations are all stored in a relational In the first stages of the annotation of the ini- database, but may be exported in various XML tial samples from the Codex Suprasliensis, the formats. The rich linguistic information provided guesser recognized only 15% of the words on the by the PROIEL-style annotation may, thus, be basis of its prior annotation of the Codex Mari- interwoven in XML format into an electronic anus. After annotating 2000 tokens of the Codex text edition that also takes the many textological Suprasliensis, the accuracy of the guessing more concerns implicit in the Suprasliensis project into than tripled, to approximately 50%. The low ini- account. The resulting edition will thus be one tial result and rapid improvement is mostly due that can serve a very wide audience with differ- to the use of diacritics in the Codex Supraslien- ent needs and interests. sis, and we are developing an orthographic nor- malizer that will temporarily strip diacritics to 6 Conclusion facilitate recognition and automated linguistic The paper outlines the stages in creating an elec- tagging. tronic edition of the Codex Suprasliensis: the The lemmata were entered with support from a digitization of the manuscript, preparation of the transliteration device, which also provides guess- electronic text, and application of morphosyntac- es based on extant lemmata. The lemmatization tic annotation. All of these tasks can constitute follows part-of-speech classification. A single objectives of separate projects (manuscript dig- form may, therefore, belong to several lemmata. itization; electronic text publication; language For example, there are no fewer than four lem- corpora compilation), but none of them alone mata with the form jako: a subjunction, a relative would be sufficient to produce an electronic edi- adverb, and two regular adverbs that are deemed tion of the manuscript. Such an edition depends to have sufficiently different functions to be sep- on all of these products, as well as the publica- arated (one meaning ‘as, like’ and the other serv- tion and annotation of the Byzantine sources, and ing as an introductory ‘for’). Morphological the development of indices, a lexicon, glossary, analysis disambiguates the morphological fea- bibliography, and others. The project therefore tures as far as possible based on syntax and con- unites the efforts of an international working text, and the information is further stored in the team with members with different but compli- database as a positional tag in the form of a mentary qualifications for the joint work on the string of symbols where each morphological fea- edition. The electronic version of the Codex Su- ture represented by a given symbol has a fixed prasliensis will be freely available under a Crea- slot (for positional tags, see also Hajič, 2004). tive Commons BY-NC-SA license. Fourth, the annotators apply syntactic annota- tion in an enriched variety of dependency gram- mar (Haug, 2010). This level relies on overt ele- Reference ments and makes it possible to keep word order information and syntactic analysis in separate Hajič, Jan. 2004. Disambiguation of Rich Inflection layers, which is essential in dealing with free- (Computational Morphology of Czech). Karolinum word-order languages such as Old Church Sla- Charles University Press, Prague. vonic and Greek. The syntactic annotation is per- Severjanov, S. 1904. Suprasl′skaja rukopis′ [Codex formed with a simple tool that provides good Suprasliensis, vol. 1-2]. Pamjatniki staroslavjan- guesses from a set of morphologically based skago jazyka, volume 1, 1-2. Sanktpeterburg. rules. Zaimov, Jordan and Mario Capaldo. 1982. Suprasŭl- Fifth comes the review stage, where the mor- ski ili Retkov sbornik (Codex Supraslinesis or the phological and syntactic analysis is reviewed by Retkov Manuscript), volume 1, Bulgarian Acad- project members, and, when found correct, pub- emy of Sciences Press, Sofia. lished on the PROIEL website. 60 Zaimov, Jordan and Mario Capaldo. 1983. Suprasŭl- ski ili Retkov sbornik (Codex Supraslinesis or the Retkov Manuscript), volume 2, Bulgarian Acad- emy of Sciences Press, Sofia. Swan, Oscar. 2008. Old Church Slavic. Inflectional Morphology, volume 1, Berkeley Slavic Special- ties, Berkeley. Miltenova, Anissava, Andrei Boyadzhiev, and Stani- mir Velev. 2000. Computerized Manuscript Corpus Data: Results and Further Development. Bulgarian Studies at the Dawn of the 21st Century: a Bulgar- ian-American Perspective. Sixth Joint Meeting of Bulgarian and North American Scholars. Blagoev- grad, Bulgaria, May 30–June 2, 1999. Gutenberg, Sofia, 237–243. Birnbaum, David J. 1996. Standardizing Characters, Glyphs, and SGML Entities for Encoding Early Cyrillic Writing. Computer Standards and Inter- faces, 18: 201–52. Birnbaum, David J. 2008. Paul the Not-So-Simple. Scripta & e-Scripta, 6: 23–45 Krǎstev, Georgi and Andrej Bojadžiev. 1999. Su- pras′lski sbornik: problemi na xronologijata i kom- pozicijata v iztočnoto-pravoslavie i v evropejskata kultura. Materiali ot meždunarodnata naučna srešta, posvetena na 1100 godišninata ot načaloto na Zlatnija vek v bǎlgarskata kultura. Varna, 2–3 Juli 1993. Guturanov, Sofia, 192–197. Miklosich, Fr. 1851. Monumenta linguae pa- laeoslovenicae e codice Suprasliensis. Vindo- bonae, 456. Abicht, R. and H. Schmidt. 1896. Quellennachweise zum Codex Suprasliensis. Archiv für slavische Phi- lologie, 18: 138-155. Haug, Dag. T. T. 2010. PROIEL Guidelines for Anno- tation: http://folk.uio.no/daghaug/syntactic_guidelines.pdf Eckhoff, Hanne Martine and Dag Trygve Truslew Haug. 2010. Aligning Syntax in Early New Testa- ment Texts: the PROIEL Corpus. Wiener Slav- istischer Almanach. Haug, Dag T. T., Marius Jøhndal, Hanne Martine Eckhoff, Eirik Welo, Mari J. B. Hertzenberg, and Angelika Muth. 2009. Computational and Linguis- tic Issues in Designing a Syntactically Annotated Parallel Corpus of Indo-European Languages. Traitement Automatique des Langues, volum 50. Haug, Dag T. T., and Marius Jøhndal. 2008. Creating a Parallel Treebank of the Old Indo-European Bi- ble Translations. http://www.hf.uio.no/ifikk/english/research/project s/proiel/Activities/proiel/publications/marrakech.p df 61 SentiProfiler: Creating Comparable Visual Profiles of Sentimental Content in Texts Tuomo Kakkonen Gordana Gali ć Kakkonen School of Computing Department of Croatian Language and University of Eastern Finland Literature [email protected] University of Split, Croatia [email protected] from the general trends in SA research in more Abstract ways than one. First, we apply SA to a field in which it has not, to the best of our knowledge, been applied before, namely the analysis of novels for literary We introduce the concept of sentiment pro- research. Second, rather than concentrating on files, representations of emotional content in determining the polarity (negative vs. positive) texts and the SentiProfiler system for creating of the analyzed document, we aim at providing and visualizing such profiles. We also demon- strate the practical applicability of the system more detailed classification affective content of in literary research by describing its use in the target texts. Third, although visualization of analyzing novels in the Gothic fiction genre. SA results has attracted some interest from the Our results indicate that the system is able to opinion mining and SA community, research support literary research by providing valuable contributions that particularly address the visual insights into the emotional content of Gothic comparison of SA results are scarce. novels. We introduce a system called SentiProfiler that generates visual representations of affective 1 Introduction content of texts and uses techniques to outline the differences and similarities between the pairs In recent years, non-topical text analysis has be- of texts under scrutiny. The design of the tool is come an active area of research. In contrast to centered on the idea of sentiment profiles (SPs), “traditional” text analytics in which the aim is to hierarchical representations of affective contents extract and process the facts and concepts, non- of the input documents. The SPs and the visual topical text analysis aims at characterizing the graphs representing them that the system gener- attitudes, opinions and feelings present in a text. ates are in effect summaries of the sentiment Automatic detection and classification of sen- content of the input documents. The system timents has several potential areas of application. makes use of various NLP and visualization re- Hence, it has become an established field of nat- sources and tools in tandem with Semantic Web ural language processing (NLP) research and a technologies. rising number of papers and articles dedicated to In addition calculating frequencies of senti- the analysis of the affective content of texts is ment-bearing words, a scoring measure is used to being published each year. Feelings and senti- determine the prevalence of sentiments in the ments are often represented within texts in com- target texts. The hierarchical organization of the plex and subtle ways, which makes their auto- sentiment classes enables the system to aggregate matic detection an interesting research challenge. these scores in order to gauge the prevalence of Much of the work in sentiment analysis (SA) specific groups of sentiments. Our visualization is concentrated on business applications, in par- technique uses vertex colors to denote the differ- ticular the analysis of user-generated online con- ences between the SPs that are being compared. tent, such as customer reviews and tweets, in or- This allows for quick browsing and detection of der to tap into what is sometimes referred to as differences between the emotional content of the the “Wisdom of the Crowds.” This work departs target documents. 62 Proceedings of Language Technologies for Digital Humanities and Cultural Heritage Workshop, pages 62–69, Hissar, Bulgaria, 16 September 2011. We demonstrate the practical applicability of and analyzing customer opinions. They also sug- the SentiProfiler system by considering the ways gested a type of visual representation that con- in which it can support literary research by al- veys customer opinions by augmenting scatter- lowing visual comparisons of pairs of SPs cre- plots and radial visualization. ated from works of a literary genre rich in dark The only research work we are aware of that and gloomy topics (and hence negative emo- has applied SA to literary research is reported in tions), namely Gothic literature. The fact that (Taboda et el., 2006). In contrast to our work, Gothic novels are divided into two distinct sub- rather than analyzing novels, Taboda et al. used genres of terror and horror allows us to experi- SA techniques to extract information on the ment with the comparison functions of the sys- reputation of six early 20 th century authors based tem. on writings concerning them. The paper is organized as follows. In Section 2, we outline the background of this research by 2.2 Tools and technologies applied shortly summarizing works on visualization of The SentiProfiler system uses WordNet-Affect SA results and describing the tools and technolo- (Strapparava and Valitutti, 2004) as the source gies on top of which SentiProfiler has been built. for emotion-bearing words. WordNet-Affect is a Section 3 introduces the architecture of the Sen- linguistic resource for the lexical representation tiProfiler system and explains the functioning of of affective knowledge. It was developed on the each of its main components. The practical ap- basis of WordNet (Miller, 1995) through the se- plicability of the system is demonstrated and dis- lection and labeling of the synsets (the WordNet cussed in Section 4. Section 5 concludes by technical term for semantically equivalent summarizing the paper and considering direc- words) representing affective concepts. Word- tions for future work. Net-Affect defines a hierarchy of emotions, in which the items are referred to as emotional 2 Introduction categories. Each emotional category is linked 2.1 Related work with a set of WordNet synsets that contain the words that are connected with the emotional Most work on SA visualization has dealt with category. The WordNet-Affect hierarchy con- summarizing analysis results for collections of tains four main categories of emotions: negative, documents. Often, basic visualization methods, positive, ambiguous and neutral. such as bar charts (Liu et al., 2005) and temporal The WordNet-Affect hierarchy and the corre- graphs (Fukuhara et al., 2007) are utilized. The sponding synsets from WordNet are represented Pulse system by Gamon et al. (2005) generates in SentiProfiler as an ontology that is automati- treemap visualizations that display topic clusters cally generated from the WordNet-Affect hierar- and their associated sentiments. The size of the chy and WordNet synset definitions. The ontol- boxes indicates the number of sentences in the ogy is created and accessed by using the Jena topic cluster, and the color denotes the average framework (http://jena.sourceforge.net/). Jena is sentiment of the sentences belonging to that a well-known and stable Java framework for topic. Chen et al. (2006) generated multiple visu- building Semantic Web applications that pro- alizations (such as decision trees and term varia- vides, among other things, an API for manipulat- tion networks) in order to enable the analysis of ing Semantic Web resources in the Resource De- conflicting opinions. scription Framework (RDF) format. One of the rare works that discusses visual SentiProfiler makes use of several other well- comparison of SA results (Gregory et al., 2006) known freely available Java tools and libraries. introduced a system that combines lexicon loo- The MIT Java WordNet Interface (JWI) kup-based SA with a visualization engine. The (http://projects.csail.mit.edu/jwi/) is an API for paper described an experiment with the well- interfacing with WordNet. JWI is used by Senti- known Hu and Liu (2004) customer review data- Profiler for retrieving the relevant synsets from a set. local copy of the WordNet dictionaries. The most relevant of the recent work on visu- GATE (Cunningham et al., 2002) is a widely alization of SA results is presented in Wu et al. used and flexible framework for developing text (2010). They introduced OpinionSeer, a system analysis systems. It is commonly applied in the that visualizes hotel customer feedbacks that is NLP research community. SentiProfiler utilizes based on a visualization-centric analysis tech- GATE in ontology-based tagging of sentiment- nique that considers uncertainty for modeling bearing words. 63 JUNG (the Java Universal Network/Graph sentiment classes . The words from the relevant Framework) (http://jung.sourceforge.net/) is a WordNet synsets are used as the individuals that library for the modeling, analysis, and visualiza- instantiate each sentiment class. Finally, the on- tion of graphs. It provides an extensible set of tology is written in an RDF/XML file that can be graph operations, visualization methods and lay- browsed and edited with any RDF-aware ontol- outs. SentiProfiler usea JUNG for the visualiza- ogy editor. This allows automatically generated tion of sentiment profiles. ontologies to be manually inspected and modi- fied to the specific needs particular analysis 3 SentiProfiler tasks. The JitterOnto ontology of negative senti- 3.1 Introduction ments that was generated for the experiments on The SentiProfiler system consists of three main Gothic literature described in Section 4 consisted components: ontology and ontology factory, sen- of the 147 classes under the negative-emotion timent analyzer and SP visualizer. Figure 1 out- branch of the WordNet-Affect hierarchy. The lines the system architecture. maximum depth from the root class negative- As shown in Figure 1, the ontology that de- emotion to a leaf sentiment class was five. This scribes the hierarchy of the emotions to be de- was the case, for example, with the sentiment tected is generated automatically from WN- class negative-emotion/general-dislike/negative- Affect and WordNet data (see Section 3.2). The fear/negative-unconcern/heartlessness/cruelty . sentiment analyzer component (Section 3.3) con- There are 823 individuals in the ontology, which sists of a GATE pipeline and a set of Java classes means that the sentiment classes are described by that analyze the tags assigned by GATE and cre- a total of 823 nouns, verbs and adjectives. ates the SPs of the input documents. The visual- Hence, each class has on average of 5.6 instantia- izer component (Section 3.3) is responsible for tions. For instance, the cruelty class mentioned displaying SPs in a format that supports easy above is instantiated by the words “cruel,” “cru- visual comparison. elly,” “cruelty,” “mercilessness,” “pitilessness,” “ruthlessness” and “unkind.” 3.2 Ontology of sentiments Figure 2 shows an extract from the branch of The automatic ontology creation process from the ontology that has to do with the set of emo- WordNet-Affect and WordNet data works as fol- tions that are classified as negative-fear . The lows. First, the XML file containing the relevant other seven classes that immediately follow the part of the WordNet-Affect hierarchy is con- root class negative-emotion are as follows: anxi- verted and stored in a graph data structure. Next, ety , daze , despair , general-dislike , humility , in- an ontology representing the sentiment hierarchy gratitude and sadness . is created that contains the WordNet-Affect sen- timent categories as classes. We refer to these as Sentiment analyzer GATE pipeline Profile visualizer Input Tokenizers Profile for document 1 document 1 Profile filter Visualization for Morphological Sentiment Input comparison analyzer profiler Profile matcher document 2 Profile for document 2 and comparator Onto Root gazetteer Ontology factory Sentiment WN-Affect JENA Ontology WordNet JWI Figure 1. Architecture of the SentiProfiler system. 64 Figure 2. Extract from JitterOnto showing the two levels of more specific sentiment classes under the parent class negative-fear. The figure was created with the Protégé editor (Stanford Center for Biomedical Informatics Research, 2011). tology-based annotations on the given content. In 3.3 Analysis of affective content the SentiProfiler GATE pipeline Onto Root Gaz- Sentiment analysis in SentiProfiler consists of etteer marks up in the input document the words creating a SP of an input document. A SP is a that match with an individual found in the ontol- hierarchy of sentiment classes that contains all ogy. The relevant sentiment class names are used the classes of the source ontology that occurred as the tags in the output. in the document, and the classes that are part of a Next, a graph presentation of the sentiment path from such a class to the root (negative- class hierarchy (i.e. the sentiment profile) is cre- emotion in case of the ontology used for the ex- ated for the input document in which each graph periments in this paper). That is, those sentiment vertex is associated with the number of times a classes that did not appear in the input document word relating to the relevant sentiment class ap- or have any children that appeared in the docu- pears in the document. In addition to the fre- ment are not included in the SP. quency counts we define a score that measures For instance, let us have a document in which the prevalence of each sentiment class. The sen- the only negative sentiment classes that appear timent class scores (SCMs) measure the relative are chill and timidity . Following the paths from frequency with which a specific sentiment si ap- negative-emotion in Figure 2 to chill and timidity pears in a document. SCM score is defined as gives us the SP of the document. follows: The creation of SPs consist of three phases: S SCM 1 = i detection of sentiment-bearing words, relating i words each such word with the relevant sentiment class where |S | is the number of times a word instanti- and, finally, constructing the hierarchy that de- i ating the sentiment class s appears in the docu- scribes the SP of a document. i ment and |S | is the total number of word tokens SA is performed in SentiProfiler with a GATE i in the document. For instance, let us have a pipeline that consists of three basic ANNIE document with 1000 word tokens. Three of the components (sentence splitter, word tokenizer sentiment-bearing words belong to the class s . and POS tagger), GATE morphological analyzer 1 The SCM score for the sentiment class s is 0.15. and an ontology-based tagging tool. Onto Root 1 An aggregate SCM score (ASCM) is defined Gazetteer (Damljanovic et al., 2008) is a GATE for all the non-leaf sentiment classes. It is calcu- processing resource that dynamically constructs lated as the sum of the SCM scores of all the sen- a gazetteer from an ontology and creates, in timent classes that succeed the current sentiment combination with other GATE components, on- classes in the hierarchy plus the SCM score of 65 the current sentiment class. This score provides a they depict and evoke in the fictional character as way of comparing whole branches of the SP well as in the reader him/herself. Although the rather than one single sentiment class at a time. works of both subgenres of the Gothic novels Figure 3 illustrates the concept. contain emotions such as anxiety, fear and gloom, in terror these emotions are more connected to a threat, real or perceived, rather than actual events of cruelty and violence. What makes this genre of novels so suitable for testing the profile comparison capabilities of SentiProfiler is, first, that they can be expected to contain a relatively high amount of (negative) emotional content. Second, the fact that there are two subgenres of Gothic novels provides a way of testing the practical use of our system in com- Figure 3. An example of SCM and ASCM scores. The paring SPs. If SentiProfiler is able, in addition to scores that are not inside parentheses are SCMs (mul- creating SP visualizations of Gothic novels, to tiplied by 10 for easier interpretation). The figures distinguish differences in the types of emotions inside the parentheses are aggregate SCM values. present in the SPs generated for samples of the 3.4 Visualization two Gothic novel subgenres it provides evidence that the system can be used as a practical tool for In the visualization of an SP, each sentiment supporting literature research. class is represented as a vertex that indicates, in Due to the nature of the target texts, we con- addition to the name of the sentiment class, any centrated our analysis on negative emotions. combination (depending on the system configu- What we were interested in observing, in particu- ration) of frequency, SCM and ASCM values. lar, were differences in the SPs that support the User can also observe the occurrences of the sen- distinction between the two subgenres of Gothic timent-bearing words pertaining to specific sen- literature as it is understood in the theory of lit- timent category along with the context in which erature. What we were expecting to be able to the word appeared. recognize is that horror novels contain emotions The matching algorithm compares the profiles that can be described as a sort of an “aftershock”, in order to find the sentiment classes that are pre- a display of disgust that appears after a horren- sent in only one of the profiles. We refer to these dous event has occurred. Terror, in contrast, as additional sentiment classes . It also calculates raises anxiety and timidity caused by the fear of the differences between the scores of those something terrible happening in the near future. classes that occur in both of the profiles. The In a sense, terror is of a more “psychological” sentiment classes that receive a higher score are nature than horror. Varma (1966) puts it suc- referred to as higher score sentiment classes . As cinctly: the difference between the two subgenres a result of the matching process, the vertices rep- “is the difference between awful apprehension resenting additional and higher score sentiment and sickening realization: between the smell of classes are denoted by specific user-modifiable death and stumbling against a corpse.” colors in the visualization. Figure 4 illustrates an extract from the com- parison of a horror and terror novel. The two 4 Experiments with Gothic Novels novels used in the comparison were Matthew We evaluated the practical applicability of the Lewis’s (1796) “The Monk: A romance”. It is SentiProfiler system by analyzing the SPs of considered as one of the prime examples of nov- Gothic novels. Such novels consist of stories of els in Gothic horror. Ann Radcliffe’s (1794) “terror and suspense, usually set in a gloomy old “The Mysteries of Udolpho” enjoys a similar 1 castle or monastery” (Baldick, 2004). The Gothic status in the Gothic terror genre . literary genre is further divided into works of terror and horror. Many explanations of the dis- tinction between the two subgenres have been put forward in the literary research community (for example, Botting, 1996). In essence, the dis- 1 All the novels for the research reported in this paper tinction between terror and horror can be sum- were obtained from the Project Gutenberg web site at marized as the intensity and the type of emotions http://www.gutenberg.org. 66 Figure 4. An extract from a sentiment profile comparison of the horror novel (The Monk) with the terror novel (Udolpho) from the perspective of the horror novel. Green vertices denote higher score sentiment classes. The red vertex (antipathy) indicates an additional sentiment class. The SP extract in Figure 4 indicates, as ex- ments that have to do with emotions that can pected under the definitions of the two subgen- be described as “less intense” and less con- res, that the novel representative of the horror nected with cruelty and the shock caused by subgenre receives higher SCM scores for the acts of violence. In addition to the sentiment sentiment classes pertaining to, for example, classes illustrated in Figure 5, this was the case, disgust and nausea. The same was observed, for instance, with the sentiment classes impa- for example, for the sentiment class cruelty tience and depression. (not shown in the figure). The color coding of vertices and the ability Figure 5 shows an extract from the same to zoom in and out (see Figure 6 for an exam- pair-wise comparison that was depicted in Fig- ple) enable the literary scholar to easily locate ure 4, but in this case from the perspective of of the branches in the SP that show a signifi- the terror novel. cant number of differences between the novels explored. Moreover, the dialog that shows the contexts in which the words linked with a spe- cific sentiment class occurred helps the com- parative literature researcher to make more detailed analyses of, for instance, style and discourse. Table 1 summarizes some of the observed differences between the SPs created based on the terror and horror novel discussed above, and the following two canonical works in the horror and terror subgenres: Frankenstein; or, The Modern Prometheus (Shelley, 1818) and the Castle of Otranto (Walpole, 1764). Figure 5. An extract from a comparison of the ter- The table does not give an exhaustive list of ror novel with the horror novel showing additional all the differences, but rather concentrates on sentiment classes diffidence and hesitance and those classes that support the notion of divid- higher score class timidity. ing the Gothic subgenres based on the “sever- As visualized in Figure 5, as expected, the ity” of emotions. terror novel more frequently contained senti- Figure 6. Zoomed-out view of a sentiment profile comparison that demonstrates how the system can be used for locating interesting branches in a sentiment profile. In this sample case, attention is drawn to the large branch in the middle, as it contains several blue vertices (denoting additional sentiment classes) and a mixture of red and green vertices. The user can zoom in to the interesting section of the visualization by turning the mouse wheel. 67 Sentiment classes Novel pair Higher score Additional T H T H T T H Franken- timidity, shyness, horror, disgust, hesitance, solici- Udolpho - stein anxiousness hate, lividity tude, insecurity sorrow, depres- horror, disgust, Otranto Monk - nausea, lividity sion, anxiety repugnance timidity, anxiety, horror, disgust, hesitance, diffi- Udolpho Monk nausea, lividity impatience repugnance, cruelty dence Franken- hopelessness, im- horror, disgust, Otranto - nausea, trepidation stein patience repugnance Table 1: Pair-wise comparison of the two horror and terror novels. The two columns under the heading “higher score” lists examples of the sentiment classes that received higher score in the terror (T) and the horror (H) novel, respectively. The columns under the heading “additional” contain examples of sentiment classes that were present only either in the terror or the horror novel. The results reported in Table 1 indicate that insights into the target texts that would be diffi- differences can indeed be observed in the relative cult, if not impossible, to obtain with the tradi- frequency and presence of certain sentiment tional “pen and paper” research methods that are classes between representatives of the two sub- typically used in the field. genres of Gothic literature. Sentiment classes The preliminary experiments we reported in such as timidity, anxiety and shyness were more Section 4 indicated that the tool can provide in- frequent in the terror novels than they were in the teresting insights for literary researchers. Senti- representatives of the horror subgenre. Horror Profiler was able to detect differences between and disgust were more frequent sentiment classes the sentiments present in example novels of in horror novels in all the four pair-wise com- Gothic terror and horror subgenres. Moreover, parisons. Nausea was among sentiments that many of the differences observed by the tool were present only in the horror novels included supported the literature theoretical distinction in the comparison. between these two subgenres. It is also interesting to note that the terror Our planned future work includes applying the novel Castle of Otranto had additional categories SentiProfiler tool to conduct a comprehensive aggravation and wrath (not shown in Table1) that study of a larger set of Gothic novels in order to were not present in either of the horror novels. verify whether the commonly accepted definition This observation did not seem to support the ex- of the difference between the emotional content pected distinction between terror and horror nov- present in horror and terror subgenres holds true. els. However, further analysis of relevant re- There are various ways in which SentiProfiler search literature (for instance, Hume (1969)) re- could be improved and extended. First, we plan vealed that, while Otranto is often considered as to apply the system to a study of Gothic novels. a terror novel, it is somewhat of a borderline case Since we are dealing with books from the eight- between the two subgenres. Being able to capture eenth and nineteenth centuries, extending the such an ambiguity gives additional support to the JitterOnto ontology of negative emotions with practical applicability of SentiProfiler. words that are not in use in modern English would presumably increase the accuracy of the 5 Conclusion system in that particular area of application. The system has many potential uses beyond li- We introduced the concept of sentiment profiles terary research. Negative emotions play a role in (SPs) and the SentiProfiler system for creating anti-social behavior. One of the future applica- easily comparable SPs created from pairs of tions of SentiProfiler and the JittersOnto ontol- documents. The system is built on the basis of ogy used in the experiment reported in this paper various well-known NLP and Semantic Web includes automatic detection and prediction of technologies and tools. We demonstrated the use potential acts of extreme anti-social behavior of the system by describing how it can support (such as school shootings) based on messages research in comparative literature. The visual posted online. It is important to note that, despite comparisons allow the literary scholar to gain the focus of this paper, SentiProfiler is designed 68 and implemented in a way that allows any of the Acknowledgments WordNet-Affect sentiment hierarchy branches, This work was supported by the project entitled and in fact any ontologically represented class “Detecting and Visualizing Changes in Emotions hierarchy, to be used. Hence, there is no reason in Texts” funded by the Academy of Finland. why the tool and the method could not be applied in more general case of SA, rather than focusing on negative emotional content. References Robert D. Hume. 1969. Gothic Versus Romantic: A Revaluation of the Gothic Novel. PMLA , 84(2): Chris Baldick. 2004. The Concise Dictionary of Liter- 282-90. ary Terms (2nd edition) . Oxford University Press, Oxford, UK and New York, USA. Matthew G. Lewis. 1796. The Monk: A Romance . Printed for J. Saunders, Waterford, Ireland. Fred Botting. 1996. Gothic (The New Critical Idiom) . Routledge, New York, USA. Bing Liu, Minqing Hu and Junsheng Cheng. 2005. Opinion Observer: Analyzing and Comparing Opi- Chaomei Chen, Fidelia Ibekwe-SanJuan, Eric San- nions on the Web. Proc. of the 14th International Juan and Chris Weaver. 2006. Visual Analysis of Conference on World Wide Web , Chiba, Japan. Conflicting Opinions. Proceedings of the IEEE Symposium on Visual Analytics Science and Tech- George A. Miller. 1995. WordNet: A Lexical Data- nology , Baltimore, Maryland, USA. base for English. Communications of the ACM , 38(11):39-41. Hamish Cunningham, Diana Maynard, Kalina Bontcheva and Valentin Tablan. 2002. GATE: A Ann Radcliffe. 1794. The Mysteries of Udolpho . Framework and Graphical Development Environ- Printed for G.G. and J. Robinson, London, UK. ment for Robust NLP Tools and Applications. Pro- Mary Shelley. 1818. Frankenstein; or, The Modern ceedings of the 40th Anniversary Meeting of the Prometheus . Lackington, Hughes, Harding, Mavor Association for Computational Linguistic , Phila- & Jones, London, UK. delphia, Pennsylvania, USA. Stanford Center for Biomedical Informatics Research. Danica Damljanovic, Valentin Tablan and Kalina 2011. Protégé . http://protege.stanford.edu/ (Ac- Bontcheva. 2008 A Text-based Query Interface to cessed March 23rd, 2011). OWL Ontologies. Proceedings of the 6th Lan- guage Resources and Evaluation Conference , Mar- Carlo Strapparava and Alessandro Valitutti. 2004. rakech, Morocco. WordNet-Affect: an Affective Extension of Word- Net. Proceedings of the 4th International Confer- Tomohiro Fukuhara, Hiroshi Nakagawa and Toyoaki ence on Language Resources and Evaluation , Lis- Nishida. 2007. Understanding Sentiment of People bon, Portugal. from News Articles: Temporal Sentiment Analysis of Social Events. Proceedings of the International Maite Taboada, Mary Ann Gillies and Paul Conference on Weblogs and Social Media , Boulder, McFetridge 2006. Sentiment Classification Tech- Colorado, USA. niques for Tracking Literary Reputation. Proceed- ings of LREC Workshop Towards Computational Michael Gamon, Anthony Aue, Simon Corston-Oliver Models of Literary Analysis , Genoa, Italy. and Eric Ringger. 2005. Pulse: Mining Customer Opinions from Free Text. Proceedings of the 6th Horace Walpole. 1764. The Castle of Otranto . Tho- International Symposium on Intelligent Data Anal- mas Lownds, London, UK. ysis , Madrid, Spain. Devendra P. Varma. 1966. The Gothic Flame: Being Michelle L. Gregory, Nancy A. Chinchor, Paul Whit- a History of the Gothic Novel in England - Its Ori- ney, Richard Carter, Elizabeth Hetzler and Alan gins, Efflorescence, Disintegration, and Residuary Turner. 2006. User-Directed Sentiment Analysis: Influences . Scarecrow Press, New York, USA. Visualizing the Affective Content of Documents. Yingcai Wu, Furu Wei, Shixia Liu, Norman Au, Proceedings of the Workshop on Sentiment and Weiwei Cui, Hong Zhou and Huamin Qu. 2010. Subjectivity in Text , Sydney, Australia. OpinionSeer: Interactive Visualization of Hotel Minqing Hu and Bing Liu. 2004. Mining Opinion Customer Feedback. IEEE Transactions on Visu- Features in Customer Reviews. Proceedings of alization and Computer Graphics , 16(6):1109- 19th National Conference on Artificial Intelli- 1118. gence , San Jose, California, USA. 69 Character Profiling in 19th Century Fiction Dimitrios Kokkinakis Mats Malm Center for Language Technology & Department of Literature, History of Språkbanken, Department of Swedish Ideas and Religion University of Gothenburg, Sweden University of Gothenburg, Sweden [email protected] [email protected] Similarly to social network extraction, there are Abstract numerous imaginable semantically oriented rela- tionships between named entity pairs, this paper This paper describes the way in which per- however only examines interpersonal ones. It sonal relationships between main characters also provides a brief description of the lexical th in 19 century Swedish prose fiction can be resources and extended named entities, used by a identified using information guided by named Swedish named entity recognition (NER) system entities, provided by a entity recognition sys- applied for the annotation of the collection. The tem adapted to the 19th century Swedish lan- guage characteristics. Interpersonal relation NER system is to a great extent rule-based and extraction is based on the context between uses a large set of lexically-driven resources. The two relevant, identified person entities. The system originates from a generic NER system relationships extraction process also utilizes used for annotation of modern Swedish which the content of on-line available lexical se- has been enhanced and improved by respecting mantic resources (suitable vocabularies) and common orthographic norms of nineteenth- fairly standard context matching methods that century Swedish spelling. provide a basic mechanism for identifying a One of the purposes in mind for this work is wealth of interpersonal relations. Such rela- to test the applicability of Natural Language tions can hopefully aid the reader of a 19th- Processing (NLP) technologies in data from a century Swedish literary work to better un- derstand its content and plot, and get a bird’s deviant domain and time period than the ones the eye view on the landscape of the core story. technology is designed for (i.e., contemporary, modern Swedish) in order to get a clearer picture 1 Introduction of the strengths and weaknesses of the resources and tools and thus identify ways to further im- Digitized information and the task of storing, prove the obtained outcomes. This way we can generating and mining an ever greater volume of facilitate the extraction of content-related seman- (textual) data become simpler and more efficient tic metadata, an important element in the man- with every passing day. Along with this opportu- agement, dissemination and sustenance of digital nity, however, comes a further challenge: to repositories. create the means whereby one can tap this great Name extraction in combination with filtering potentiality and engage it for the advancement of scripts that model the vocabularies, as well as (scientific) understanding and knowledge min- fairly standard context matching methods pro- ing. The goal of this research is to generate a vide a mechanism for identifying interpersonal complete profile for all main characters in each relations that can also aid the reader of a literary arbitrary volume in a literature collection of 19th work to better understand its content and plot, century fiction. We also aim at a methodology and get a bird’s eye view on the landscape of the that should be easily transferable to any other core story. Despite the risks of spoiling the en- piece of literary work. A complete profile im- joyment that some readers of the narrative would plies an exhaustive list of any kind of interper- otherwise have experienced without revealing of sonal relationships, such as Friend Of and Anta- any plot elements, we still believe that such sup- gonist Of that can be encountered between the porting aid can be used for an in-depth story un- main characters in a literary work. derstanding (for the human reader). Moreover, creating biographical sketches (e.g., birthplace) 70 Proceedings of Language Technologies for Digital Humanities and Cultural Heritage Workshop, pages 70–77, Hissar, Bulgaria, 16 September 2011. and extracting facts for entities (e.g., individuals) characters speak within 300 words each other, can be easily exploited in various possible ways and finally, a social network is constructed from by NLP technologies such as summarization and the conversations. Nodes are named speakers and question answering (e.g., Jing et al., 2007). edges appear if there was a conversation between two characters, a heavier edge means more con- 2 Related Work versations. Our approach is mainly influenced by the work by Hasegawa et al. (2004) who pro- Natural-language processing is an attractive ap- posed an unsupervised, domain-neutral approach proach to processing large text collections for to relation extraction by clustering named entity relation extraction (usually defined as a relation pairs according to the similarity of context words predicate ranging over two arguments, e.g., con- intervening between two entities and selecting cepts or people) and there exist a number of the most frequent words from the context to label techniques that have applicability to any type of the relation. text; for a general review see Hachey (2009). Such techniques can facilitate more advanced 3 Material: a Prose Fiction Corpus research on literature and provide the appropriate mechanisms for generating multiple views on Prose fiction is a just one type of textual material corpora and insights on people, places, and that has been brought into the electronic “life” events in a large scale, through various types of using large scale digitized efforts. Prose fiction is relations. an essential source within many disciplines of Relation extraction was introduced in the mid humanities (history, religion, sociology, linguis- 1990s by the Template Element and Template tics etc) and social studies and an invaluable Relation tasks in MUC-6 (Message Understand- source for understanding the movements of soci- ing Conferences) and followed by the ACE (Au- ety by its ability to demonstrate what forces and tomatic Content Extraction) Relation Detec- ideas are at work in the society of its time. Prose tion/Recognition tasks (cf. Doddington et al., fiction is complex and difficult to use not only 2004). Since then it has been an active and fruit- because of interpretational complexity but also ful area of research, partly driven by the explo- because of its limited availability. The "19th sion of the available information via the Web and Century Sweden in the Mirror of Prose Fiction" partly by the evidence that embedded relations (Det svenska 1800-talet speglat i prosafiktionen) are useful for various NLP tasks such as Q&A project (2009-12) aims at developing a represen- and Information Retrieval. tative corpus which mirrors society at given Relation extraction approaches (particularly points in time, chronologically selected in such a binary ones) can be classified in various ways. way that historical comparisons can be made. Knowledge engineering approaches (e.g., rule- The material is all fiction, written in the original based, linguistic based), learning approaches and published separately for the first time, that (e.g., statistical, machine learning, bootstrapping) appeared in Swedish during the years 1800, and hybrid ones; for an overview of techniques 1820, 1840, 1860, 1880 and 1900 (300 publica- see Jinxiu (2007). Learning approaches become tions, ca 60,000 pages). The material provides a more and more common in the open domain i.e. whole century of evolution and social, aesthetic, large corpora of web scale, cf. Agichtein & Gra- scientific, technical, cultural, religious and phi- vano, (2000); Christensen et al. (2010); relations losophical change. are also of particular interest and prominent in the (bio)medical domain; e.g. Rosario & Hearst 3.1 Lexical Resources (2004); Giles & Wren (2008); Roberts et al. The main focus of this research is the extraction (2008). Elson et al. (2010) describe a method to of main character profiles1, in literary archives extract social networks from literature (nine- and as a starting point we only look into interper- teenth-century British novels and serials) de- sonal relationships. There is a number of suita- pending on the ability to determine when two ble, freely available resources that we have characters are in conversation. The authors use a started to exploit in order to aid the relation iden- named-entity tagger to automatically locate all tification process, particularly the RELATION- the names in a novel and then a classifier that automatically assigns a speaker to every instance 1 Currently, this work is similar to the extraction of social of direct speech in the novel using features of the networks but in the long run it is also desirable to extract surrounding text. A “conversation” occurs if two more than merely interdependency relations of individuals (e.g., birth place, workplace etc.). 71 SHIP2 vocabulary and two Swedish lexical se- also a number of not so suitable near synonyms mantic resources, namely the FrameNet++3 and such as kompis ‘buddy’, therefore we had to ma- the Swesaurus4. These resources are useful and nually go through such near synonym lists and provide the appropriate machinery for our goals, discard erroneous entries. namely to both identify appropriate relationship oriented lexical units and also appropriate rela- 3.2 Named Entities and Animacy tionship labels. There has been some work in the past on defin- The RELATIONSHIP vocabulary defined by ing and applying rich name hierarchies, both Davis & Vitiello (2010) is a good starting point specific (Fleischman & Hovy, 2002) and generic for the labeling of the interpersonal relations. In (Sekine, 2004) to various corpora. However, in their work Davis & Vitiello provide a description other approaches (Kokkinakis, 2004) the wealth of 35 possible relationships that can occur be- of name types is captured by implementing a tween individuals. The description is not unprob- fine-grained named entity taxonomy by keeping lematic since some of these relationships may be a small generic set of named entity types as main partially overlapping or even tautological such as types and modeling the rest using a subtype me- ChildOf vs. AncestorOf / DescedentOf and frien- chanism. In this latter work a Person entity (a dOf vz. closeFriendOf, The two other resources, reference to a real word entity) is defined as namely the Swedish Swesaurus (Borin & Fors- proper nouns – personal names (forenames, sur- berg, 2010), that is fuzzy synsets in a WordNet- names), animal/pet names, mythological names, like resource under active development, and the names of Gods etc. – and common nouns and Swedish FrameNet++ (Borin et al., 2009) pro- noun phrases denoting groups/sets of people. In vides a large, and constantly growing number of this work the rule-based component for Person synonyms and related words that are important entity identification utilizes a large set of desig- for the relation extraction task. nator words (e.g., various types of nominal men- In the Swedish FrameNet++ such words are tions) and phrases (e.g., typically verbal con- called lexical units and are described by a num- structions) that require animate subjects, a rele- ber of frames. A frame is a script-like structure vant piece of knowledge which is explored for of concepts, which are linked to the meanings of the annotation of animate instances in literary linguistic units and associated with a specific texts and other related tasks (cf. Orasan & Evans, event or state. A number of frames and particu- 2001). These designators are divided into four larly the lexical units encoded therein are rele- groups according to their semantic denotation: vant for interpersonal relationship extraction, such frames are for instance the Person- nationality or the ethnic/racial group of a per- al_Relationship (with lexical units: flickvän ‘girl son (e.g. tysken ‘the German [person]’) friend’ and make ‘husband’, etc.), the Kinship profession (e.g. läkaren ‘the doctor’) (with lexical units: barnbarn ‘grandchild’, bror family ties and relationships (e.g. svärson ‘brother’, brorsdotter ‘niece’, dotter ‘daughter’, ‘son in law’; moster ‘aunt [from the mother’s etc.) and the Forming_Relationship (with lexical side]’) units: förlova_sig ‘become engaged with’, gif- individual that cannot be unambiguously ca- ta_sig ‘marry with’, etc.). These frames are semi- tegorized into any of the other three groups automatically mapped to the RELATIONSHIP (e.g. patienten ‘the patient’) vocabulary and their containing lexical units be- come the actual lexical manifestation of the rela- Animacy markers are further marked for gender tionship in question. Similarly, we have experi- (male, female or unknown/unresolved such as mented with the Swedish Swesaurus in order to barn ‘child’). An example of animacy annotation identify synonyms for some of these lexical is given below. In this example the animacy units. This way we can increase the amount of attribute, ANI, has a value FAF which stands for the words that can be part of various relations FAmily and Female, while the attributes TYPE types. Thus, for the word kollega ‘colleague’ we and SuBType refer to PeRSon and HUMan re- can get a set of acceptable near synonyms such spectively: 2 attribute is that it can be helpful for ruling out http://vocab.org/relationship/.html 3 http://spraakbanken.gu.se/eng/swefn some erroneous non-allowable, gender-bearing 4 http://spraakbanken.gu.se/swe/forskning/swefn/swesaurus 72 relations such as the one given in example 5. In We automatically annotated each available vo- this example there in an obvious anomaly in- lume in the collection with a slightly tuned to volved considering the otherwise erroenous final 19th century Swedish system; for details cf. Borin relation, SiblingOf. Stina is recognized as female et al. (2007) and Borin & Kokkinakis (2010). We (through the attribute UNF) while in the preced- started by first clustering all possible context ing context the word broder 'brother' implies a lengths and also applied template pattern match- following mention of a male gender, such fea- ing once again on all possible contexts. After tures could be perhaps rule out spurious relation- some experiments we split the process into two hips. separate ones guided by the number of tokens between the entities. Very short contexts can be 4 Method quickly and reliably captured in a pattern match- ing fashion. Therefore, we decided to first apply NLP techniques, such as Information Extraction, template pattern matching involving two recog- provide methods for identifying domain specific nized person entities and matched the intervened relations and event information. From a initial context with the lexical information extracted perspective such methods seem to be doomed to and modeled from the various lexical sources. fail since each literary work is in itself a kind of Example of manually designed, pattern matching closed world or domain where one may deal with templates are provided below: death and resurrection and another on travelo- gues. However, each piece of work has certain GrandparentOf: morfar|mormor|farfar|farmor|morfader|... general characteristics that can be captured by applying fairly standard NLP components such 73 (1´) Muhammeds=>dotter/ParentOf=>Fatima (4´) Mafalda=>syster/SiblingOf => Linda ParentOf (Fatima, Muhammed) SiblingOf (Mafalda, Linda) (2´) Strindberg=>träffat/HasMet=>Nennie (5´) *Ivar=>brodern/SiblingOf => Stina HasMet (Strindberg, Nennie) ?SiblingOf (Ivar, Stina) (3´) Taube=>anställde/EmployerOf=>Marie (6´) Modén => kallades/Relationship => Susanne Cederlöf Moderat EmployerOf (Taube, Marie Susanne Cederlöf) Relationship(Modén, Moderat)) For contexts between 4 and 10 tokens we pro- 5 Results duce context vectors (bag of words) from all in- The context similarity between extracted pairs of tervening tokens of all contexts, with the exclu- entities can be measured in various ways. We sion of punctuation and numerical tokens. We applied hierarchical clustering (Seo & Shneider- chose not to include the very short contexts since man, 2002) with complete linkage and with co- pattern matching is reliable for short window sine similarity as a similarity measure. We then sizes. After some test we limited the maximum manually evaluated obtained clusters and picked- window to be 10 tokens;, larger size of interven- up the top-5 most frequent words in these clus- ing tokens introduce in many cases noisy results. ters as a means to characterize the cluster and Examples 4-6 illustrate cases with a context be- tried to map these to the RELATIONSHIP voca- tween the person entities of >2 tokens. Note that bulary. Unfortunately, this activity revealed limi- the extracted relations in example 5´ is actually tations since it was challenging to point to an erroneous probably caused by one of the context appropriate label possibly because the data was words, namely brodern 'the brother'. This could too limited in size and also because most clusters be actually eliminated if the animacy attribute had very few members (see more discussion be- ANI=UNF (Female) could be considered, a case low). For the evaluation (Precision, Recall and F- left for future developments of this work. Exam- score) we chose to examine in more detail three ple (6) illustrates another issue namely that of a randomly distinct volumes (see the References' potential relations that cannot be captured by the section). Precision (Pr) is the fraction of relation existing vocabulary; i.e. a tautology. For such instances that is correct, and for clustering Pr relations there seems to be a default one, labeled, hc the correct contexts that could be mapped among Relationship which is defined as "A class whose the contexts clustered automatically. Recall (R) members are a particular type of connection ex- is the fraction of relation instances that has been isting between people related to or having deal- correctly extracted among all possible that in- ings with each other", which we also use5. volve two person named entities. Table 1 summarizes these results, here All is (4) 74 TYPE="PRS" SBT="HUM" ANI="UNU">Kurt 75 ing and/or her world knowledge, also a number NAACL HLT First International Workshop on of implicit relations could be inferred (e.g. X Formalisms and Methodology for Learning by ChildOf Y implies Y ParentOf X). Moreover we Reading. Pp 52–60, Los Angeles, California. would like to explore co-reference (pronominal Ian Davis and Eric Vitiello Jr E. 2010. RELATION- references) since it plays an important role for SHIP: A vocabulary for describing relationships profiling (biographical) extraction and for recog- between people. This work is partially supported by the Centre for George Doddington, Alexis Mitchell, Mark Przybock- i, Lance Ramshaw, Stephanie Strassel and Ralph Language Technology (CLT) 76 Ellen Riloff. 1996. Automatically Generating Extrac- tion Patterns form Untagged Text. Proceedings of the Thirteenth National Conference on Artificial Intelligence (AAAI-96), pp. 1044-1049. Angus Roberts, Robert Gaizauskas and Mark Hepple. 2008. Extracting Clinical Relationships from Pa- tient Narratives. BioNLP 2008. Pp 10–18, Ohio, USA. Barbara Rosario and Marti A. Hearst. 2004. Classify- ing Semantic relations in Bioscience Texts. Pro- ceedings of the 42nd Annual Meeting on ACL. Barcelona, Spain. Satoshi Sekine. 2004. Definition, Dictionaries and Tagger for Extended Named Entity Hierarchy. Pro- ceedings of the Language Resources and Evalua- tion Conf (LREC). Lisbon, Portugal. Jinwook Seo and Ben Shneiderman, 2002. Interac- tively Exploring Hierarchical Clustering Results. IEEE Computer, Vol. 35:7, pp. 80-86. 77 Diachronic Stylistic Changes in British and American Varieties of 20th Century Written English Language Sanja Stajnerˇ and Ruslan Mitkov Research Group in Computational Linguistics University of Wolverhampton Stafford Street, Wolverhampton, WV1 1SB, UK S.Stajner, R.Mitkov @wlv.ac.uk { } Abstract it is hypothesised that changes in certain aspects of an author’s writing style ought to be detectable by In this paper we present the results of a study investigating the diachronic changes of four using appropriate methods and stylistic markers. stylistic features: average sentence length, Au- Inspired by this hypothesis, one of the main goals tomated Readability Index, lexical density and of our study is to investigate whether diachronic lexical richness in 20th century written En- changes in certain aspects of the writing style used glish language. All experiments were con- in a specific text genre can be detected by using ducted on the largest existing diachronic cor- the appropriate methods and stylistic markers (fea- pora of British and American English – the tures). Brown ‘family’ corpora, employing NLP tech- niques for automatic extraction of the fea- Different authors, even from the same period tures. Additionally, we compare the trends of time, will exhibit various styles. Consequently, of changes between the two English varieties this will be epitomised by very heterogeneous re- and make suggestions for future studies of di- sults in each observed year and genre, making gen- achronic language change. eral changes in style not readily detectable. There- 1 Introduction fore, our first goal is to establish a methodology which would be appropriate for the investigation As time elapses, language changes and those of these types of diachronic changes. This would changes are present at various levels of the lan- lead to more precise and statistically justified con- guage structure: vocabulary, phonology, morphol- clusions in the corpus-based studies of diachronic ogy and syntax (Kroch, 2001). Most of the previ- linguistics. ous studies of language change in English tended Our second goal is to explore diachronic to focus on the phonetic and lexical rather than changes of several well-known stylistic markers: the stylistic or syntactic changes. Furthermore, the average sentence length, Automated Readability vast majority of the early sociolinguistic and his- Index, lexical density and lexical richness in two torical linguistic studies of language change did major English language varieties – British and not provide textual evidence for their claims, i.e. American and to detect whether those changes they were not corpus-based or the used corpora were present in both of those varieties. As we are were not large and representative enough. Bauer using the mutually comparable corpora of British (1994) set higher methodological standards for di- and American English, we are able to compare the achronic studies as he was among the first who changes of these two language varieties and to ex- sought to support his statements with textual ev- amine whether they followed the same trends of idence (Mair and Leech, 2006). stylistic changes. In his study of authorship attribution, Holmes We base our methodology on the largest pub- (1994) defines style of the text “as a set of mea- licly available diachronic corpora of the 20th cen- surable patterns which may be unique to an au- tury written English language – Brown ‘family’ thor”. In the context of language change, this defi- corpora (Leech and Smith, 2005) and NLP tools nition could be slightly amended and the style de- provided by the state-of-the-art Connexor Machi- fined as a set of measurable patterns which may be nese Syntax parser1. unique to a particular period of time. In an exten- sive overview of applications in stylochronometric approaches in the last sixty years (Stamou, 2008) 1www.connexor.eu 78 Proceedings of Language Technologies for Digital Humanities and Cultural Heritage Workshop, pages 78–85, Hissar, Bulgaria, 16 September 2011. 2 Related Work pus Collection2. Two of these contain texts written in British English, published in 1961 and 1991, re- Various studies in the field of stylistic variation spectively: and change use both the historical and sociolin- The Lancaster-Oslo/Bergen Corpus of British guistic approach. Typical examples of this can • be seen in the early studies of author’s and pe- English (LOB) riod styles (Gordon, 1966; Adolph, 1968; Ben- The Freiburg-LOB Corpus of British English nett, 1971). A preponderance of subsequent stud- • (FLOB) ies allude to the textual dimensions and relations used in Biber (1988) and follow the works of The other two contain texts written in American Biber and Finegan (1986, 1988, 1989). Westin English, published in 1961 and 1992, respectively: (2002) and Westin and Geisler (2002) explore stylistic changes in the 20th century in the Corpus The Brown University corpus of written • of English Newspaper Editorials (CENE) across American English (Brown) the five linguistic dimensions described in (Biber The Freiburg-Brown Corpus of American and Finegan, 1988). They employ the methodol- • ogy based on a multi-dimensional framework pre- English (Frown) sented in (Biber, 1985; 1988). All four corpora share the same sampling frame, The emergence of the FLOB and Frown corpora as the LOB3, FLOB4 and Frown5 corpora were de- in the 1990s, compared with the previous LOB and signed to closely match the structure of the Brown Brown corpora, offered new possibilities for di- corpus6 with the aim to provide an opportunity to achronic studies of written English language in the compare diachronic changes within two major va- 20th century across two major English varieties – rieties of the written English language (Leech and American and British English. Mair and Hundt Smith, 2005). (1995), Mair (1997, 2002), Mair, Hundt, Leech Each corpus is a one-million-word corpus, con- and Smith (2002), Smith (2002, 2003a, 2003b), sisting of 500 texts of about 2000 running words Leech (2003, 2004), Leech and Smith (2006) and each, selected at a random point in the original Mair and Leech (2006) exploited the possibilities source. The sampling range covers 15 text genres, of the Brown ‘family’ corpora by investigating the which can be grouped into four more generalised trends of changes in various lexical, grammatical categories: and syntactic features. All of these studies were conducted on manually corrected POS tagged cor- Press (Press: Reportage; Press: Editorial; • pora and used the log likelihood test as a measure Press: Review) of the statistical significance of the results. General Prose (Religion; Skills, Trades and To our best knowledge, there is no mention • Hobbies; Popular Lore; Belles Lettres, Bi- of any investigation of diachronic changes of the ographies, Essays) four stylistic features we used in our study nor studies which describe automatic extraction of Learned (Miscellaneous; Science) the features from the raw text corpora by using • NLP techniques. All methodologies used in the Fiction (General Fiction; Mystery and De- • previous studies of language changes required a tective Fiction; Science Fiction; Adventure great amount of human annotation or manual cor- and Western; Romance and Love Story; Hu- rections which are time-consuming and labour- mour) intensive or they relied on the use of very specific language tools. In this study we separately investigate diachronic stylistic changes in each of the four main text cat- egories. 3 Corpora 2http://www.hit.uib.no/icame 3 The Brown ’family’ corpora (Leech and Smith, http://khnt.aksis.uib.no/icame/manuals/lob/ 4http://khnt.aksis.uib.no/icame/manuals/flob/ 2005) consist of four mutually comparable cor- 5http://khnt.aksis.uib.no/icame/manuals/frown/ pora publicly available as part of the ICAME Cor- 6http://khnt.aksis.uib.no/icame/manuals/brown/ 79 4 Methodology Contracted negative form and its antecedent verb are treated as separate tokens. E.g. in the case Although all four corpora are available in their of isn’t, the verb and negation are treated as two tagged versions with the annotated sentence separate tokens is and not and correspondingly as boundaries, those boundaries are not consistent two separate base forms be and not. throughout them. The LOB and FLOB tagged ver- Treatment of ’s depends on its role in the sen- sions were manually corrected and therefore 100% tence (Connexor Oy, 2006a). In the cases where ’s accurate, while the Brown and Frown corpora are represents a genitive form, e.g. “ ... Roy’s United not manually corrected. Furthermore, some in- Federal Party ...” (LOB:A01), Roy’s is treated as consistencies regarding the sentence boundaries one token Roy’s and corresponding base form is in the cases of direct speech and itemised sen- roy. In other cases where ’s represents the contrac- tences are present among different corpora. There- tion of the verb to be (is) or to have (has), e.g. “ fore, we decided to use the raw text versions of ... he’s nice ... ” (LOB:P05), the personal pronoun the LOB, FLOB, Brown and Frown corpora (di- and verb contraction are treated as two separate to- vided into original portions of 500 texts of approx- kens – he and is. Accordingly, they are treated as imately 2,000 words each) and parse them with two separate base forms – he and be. the Connexor Machinese Syntax parser in order to achieve more consistent sentence splitting, tokeni- 4.2 Lemmatisation sation and lemmatisation. The sentence bound- In this sub-section, we describe the results of the aries identified by the parser offered a fairer com- lemmatisation process for the three word types – parison of the results among the corpora which possessive pronouns, derived adverbs and EN and was of primary importance for this study. ING forms, as they differ between the current and Connexor Machinese Syntax parser is based on the previous versions of the parser. linguistic methods and its lexicon contains hun- dreds of thousands of base forms. For compi- 4.2.1 Base forms of possessive pronouns and lation of the lexicon various large corpora were derived adverbs used, among which the most common were news The current version of the Machinese parser as- texts, bureaucratic documents and literature (Con- signs to possessive pronouns their own base forms nexor Oy, 2006b). In cases when the word can- (lemmas), e.g. the base form of the word yours not be found in the lexicon, word class and base in the current version of the parser is yours, while form are determined by using the heuristic meth- in the previous versions possessive pronouns were ods (Connexor Oy, 2006b). POS accuracy of the assigned the base forms of their corresponding Machinese Syntax parser measured on Standard personal pronouns, e.g. the base form of the word Written English (benchmark from the Maastricht yours in the previous versions of the parser was Treaty) was 99.3% with no ambiguity (Connexor you (Connexor Oy, 2006b). Oy, 2006b). The earlier research of Samuelsson Base forms of derived adverbs, e.g. im- and Voutilainen (1997) reported excellent accu- mediately, fundamentally, absolutely, directly racy of the software used as the base for the cur- (LOB:A01) are assigned immediately, fundamen- rent version of the parser. The fact that the lexicon tally, absolutely, directly as their base forms in the of the Machinese parser was built by using simi- current version of the parser while in the previ- lar text genres as those represented in the Brown ous versions they were assigned the following base ’family’ corpora ensures a high accuracy of the forms: immediate, fundamental, absolute, direct. analysis completed in this study. In the following two sub-sections we list some 4.2.2 EN and ING forms of the specificities of the tokenisation and lemma- The current version of the parser assigns different tisation procedures used by the Machinese parser. base forms for EN and ING forms which repre- This is important for better understanding and in- sent the present or past participle of a verb than terpreting the results provided in this study. for those representing corresponding nouns or ad- jectives. Previous versions of the parser were as- 4.1 Tokenisation signing the same base forms for all four possible The Connexor Machinese parser has a specific cases of EN and ING forms. For example, in the treatment of the contracted negative form and ’s. sentence: 80 “The Government decided to adjust the Average sentence length was measured as the financing ...” (LOB:A01) total number of words divided by the total number of sentences for each text (eq.1), using the sen- the assigned base form for the noun financing is tence and word boundaries returned by the parser. financing while the older versions of the parser Words containing only punctuation marks were would assign finance as the base form for the same not counted. word financing in the given context. total number of words Similarly, in the sentence: ASL = (1) total number of sentences “Sir Roy is violently opposed to Automated Readability Index (Senter and Africans getting an elected majority in Smith, 1967; Smith and Kincaid, 1970) belongs Northern Rhodesia ...” (LOB:A01) to the formative era of readability studies and was listed among eleven most commonly used read- the assigned base form for the adjective elected ability formulas (McCallum and Peterson, 1982) is elected while the older versions of the parser of that time. It is calculated using the following would assign elect as the base form for the same formula: word elected in the given context. c w In the above sentence, the words opposed and ARI = 4.71 + 0.5 21.43 (2) w s − getting are assigned oppose and get as the base forms in all versions of the Machinese parser, as where c, w and s represent, respectively, total num- they represent the past and present participle of the ber of characters, words and sentences in the text. verbs oppose and get, respectively. The result of the Automated Readability Index gives the US grade level necessary to understand 5 Experimental settings the given text. Lexical density is computed as the number of We conducted two sets of experiments: unique word types (tokens) divided by the total number of tokens in the text (eq.3). Stylistic diachronic changes in British En- • glish in the period 1961–1991 number of unique tokens LD = (3) total number of tokens Stylistic diachronic changes in American En- • glish in the period 1961–1992 Low lexical density indicates many repetitions of the same words throughout the text, while high The first experiment was conducted on LOB lexical density suggests a use of a wider range of (1961) and FLOB (1991) corpora and the second vocabulary. This feature has been used as a stylis- experiment on Brown (1961) and Frown (1992) tic marker in (Ule, 1982) and for dating works in corpora. (Smith and Kelly, 2002). In both experiments we investigated the di- Lexical richness is computed as the number of achronic changes over the four main text cate- unique lemmas divided by the total number of to- gories: Press, General Prose, Learned and Fiction kens in the text (eq.4). separately, as the preliminary results had shown number of unique lemmas different trends of changes among these four text LR = (4) categories. total number of tokens As stylistic markers we used the following four The use of lexical richness separately from lexi- features: cal density was proposed by Corpas Pastor et al. Average sentence length (ASL) (2008) who argued that “lexical density is not in- • dicative of the vocabulary variety of an author as Automated Readability Index (ARI) it counts morphological variants of the same word • as different word types”. Following this argument, Lexical density (LD) we make a distinction between lexical density and • lexical richness and investigate both features sep- Lexical richness (LR) arately. • 81 6 Results and Discussion Automated Readability Index shows a statisti- cally significant increase over the observed period The results of the first and second experiment are 1961–1991 in the Press and Prose text categories given separately (sub-sections 6.1 and 6.2, respec- (Table 1) which can be interpreted as a tendency tively). The results of each experiment are divided to render texts in these categories in a difficult-to- into two tables, where the first table contains the read manner. results of the investigation of diachronic changes The results of the first experiment (Table 1) in- of ASL and ARI, and the second table contains re- dicate that ASL did not change significantly in the sults of the investigation of diachronic changes of period 1961–1991 in any of the four investigated LD and LR. Each feature contains two columns – text categories of British English. ‘change’ and ‘p’. LD LR Column ‘change’ represents the relative change Genre of the feature over the period 1961–1991/2, mea- change p change p Press +7.43% 0.000 +7.17% 0.000 sured as a percentage of the starting value in 1961. Prose +3.90% 0.000 +4.40% 0.000 The sign before the value signifies the direction Learned +2.35% 0.248 +1.76% 0.416 of the change; ‘+’ corresponds to an increase and Fiction +3.92% 0.008 +4.28% 0.012 ‘ ’ to a decrease. Both the starting and end- − ing values in years 1961 and 1991/2, respectively, Table 2: British English: LD and LR. were previously calculated as an arithmetic mean of the feature values in all texts of the relevant text On the basis of the data analysed (Table 2), we category (Press, Prose, Learned or Fiction) and can draw the conclusion that the vocabulary was corpus (LOB or FLOB in the sub-section 6.1 or enriched in three text categories of British English Brown and Frown in the sub-section 6.2). – Press, Prose and Fiction, over the observed pe- riod 1961–1991. The strongest intensity of these Column ‘p’ represents the p-value of the two- changes can be noticed in the Press category. tailed t-test applied on the results obtained for each text separately (in the corresponding text category 6.2 Diachronic changes in American English and corpus) and used as a measure of statistical significance. For the results with a p-value lower Results of the second experiment – diachronic than the chosen critical value 0.05, we are more stylistic changes in American English over the pe- than 95% sure that they represent real diachronic riod 1961–1992 are presented in Table 3 and Table changes rather than being a consequence of faulty 4. sampling. Those results were considered as statis- ASL ARI Genre tically significant and reliable enough to be used change p change p for making further hypotheses. Press 4.90% 0.034 1.43% 0.598 Prose −3.35% 0.164 +−3.32% 0.242 Given that the corpora used in the two ex- Learned −10.49% 0.000 3.29% 0.278 Fiction − 7.37% 0.058 −9.92% 0.082 periments are mutually comparable, we are able − − to make a comparison of the trends of changes between British and American English in sub- Table 3: American English: ASL and ARI. section 6.3. The Press and Learned text categories man- 6.1 Diachronic changes in British English ifested a decrease of ASL in the observed pe- Results of the first experiment – diachronic stylis- riod 1961–1992 (Table 3). This could be inter- tic changes in British English over the period preted as an example of colloquialisation – “a ten- 1961–1991 are presented in Table 1 and Table 2. dency for the written language gradually to ac- quire norms and characteristics associated with the ASL ARI spoken conversational language” (Leech, 2004), Genre change p change p as it is known that shorter sentences are a char- Press +4.39% 0.114 +6.43% 0.046 acteristic of the spoken language. Prose +1.64% 0.490 +9.07% 0.002 Learned 5.05% 0.060 +3.07% 0.254 The results in Table 3 indicate that ARI did Fiction −4.85% 0.184 1.97% 0.726 − − not change significantly in the period 1961–1991 in any of the four investigated text categories of Table 1: British English: ASL and ARI. American English. 82 Lexical density and lexical richness demon- glish in both starting and ending years – 1961 strated a statistically significant increase only in and 1991/2, for each genre and each feature sep- the Prose text category (Table 4). arately. These results were consistent with the re- sults of the previous two experiments – diachronic LD LR Genre changes in British and diachronic changes in change p change p Press +2.03% 0.084 +0.71% 0.588 American English, thus supporting the hypotheses Prose +4.06% 0.000 +3.85% 0.004 made in the previous two subsections. Learned +3.89% 0.082 +3.90% 0.092 Of most interest were the results obtained from Fiction 2.51% 0.106 3.04% 0.088 − − the Press category (Table 5), as they suggested the presence of Americanisation – “the influence Table 4: American English: LD and LR. of north American habits of expression and be- haviour on the UK (and other nations)” (Leech, These results lead to the conclusion that the vo- 2004) in the observed period 1961-1991/2. They cabulary used in the Prose category of American indicated that the influence between the British English was enriched over the observed period and American English written styles was more 1961–1992. pronounced in the Press category than in the other 6.3 British vs. American diachronic changes three – Prose, Learned and Fiction. 1961 1991/2 From the results presented in the above four tables Feature we are able to make a comparison of the trends of Br. Am. Br. Am. ASL 20.63 21.50 21.53* 20.45 diachronic changes between British and American ARI 11.34 12.08* 12.07 11.91 English for the four investigated stylistic features LD 36.37 37.65* 39.08 38.41 and provide some general observations. LR 32.61 33.83* 34.95 34.07 The central conclusion is that British and Amer- ican English do not follow the same trends of Table 5: Synchronic comparison: Press category. stylistic diachronic changes in all genres and for all features. The most striking differences are in In Table 5, the value of the LD and LR calcu- the cases of ASL and ARI. They demonstrated lated using equations 3 and 4 (Section 5) is mul- statistically significant changes in only one of the tiplied by 100. Statistical significance of the dif- two investigated varieties of the English language. ferences between the feature values in British and ASL had a statistically significant decrease in the American English is measured by the two-tailed Press and Learned categories of American English t-test. The results significant at the 0.05 level are over the period 1961–1991 (Table 3), while at the shown in bold. In those cases, the higher of the same time did not manifest any statistically signif- two values (Br. or Am.) is marked with an ‘*’. icant changes in any of the four text categories of Statistically significant increases of the ARI, British English (Table 1). ARI demonstrated a sta- LD and LR in the Press category of British En- tistically significant increase in the Press and Prose glish during the period 1961–1991 (Table 1 and categories of British English (Table 1) and no sta- Table 2) together with the significantly higher val- tistically significant changes in any of the four text ues of these features in the same text category of categories of American English (Table 3). American English in 1961 (Table 5) could be ex- LD and LR manifested a statistically significant plained using the aforementioned Americanisation increase in three text categories of British English hypothesis. – Press, Prose and Fiction (Table 2). In Ameri- 7 Conclusions can English, this trend was followed only in the Press category, while the other two text categories On the basis of the data analysed in the previous – Prose and Fiction, did not exhibit any statisti- section, we can draw the following conclusions cally significant changes of LD or LR (Table 4). about the trends of stylistic changes in the British In order to better understand noticed diachronic and American varieties of English language over changes and investigate possible influence be- the period 1961–1991/2: tween the two English language varieties, we con- 1) The Prose text category followed the same ducted an additional experiment of synchronic trend of enriching the vocabulary in both varieties comparison between British and American En- of the English language. 83 2) Average sentence length had a statistically EMMC 2008-0083, Erasmus Mundus Masters in significant decrease in the Press and Learned text NLP & HLT. categories in American English and no statistically significant changes in any of the four text cate- gories in British English. References 3) Automated Readability Index had a statisti- Adolph Robert. 1966. The Rise of Modern Prose Style. cally significant increase in the Press and Prose Cambridge, Mass.: M.I.T. Press. text categories in British English and no statisti- Bauer Laurie. 1994. Watching English change: and cally significant changes in any of the four text introduction to the study of linguistic change in stan- categories in American English. dard English in the twentieth century. London: Additionally, the presented study allowed us to Longman. make several more general conclusions: Bennett R. James. 1971. Prose Style: A Historical Ap- 1) NLP tools can be successfully used in the proach through Studies. San Francisco: Chandler. studies of language change and make use of the raw text corpora customising it for the specific Biber Douglas. 1985. Investigating Macroscopic Tex- purposes thus saving a great amount of human ef- tual Variation through Multifeature / Multidimen- sional Analyses. Linguistics, 23: 337–360. fort for annotation or manual corrections. 2) Stylistic changes are present and are notice- Biber Douglas and Finegan Edward. 1986. An Initial able even after the 30 year time gap in various cat- Typology of English Text Types. In: J. Aarts and W. egories of the written language. Meijs, eds. Corpus Linguistics H: New Studies in the Analysis and Exploitation of Computer Corpora. 3) Different genres show different trends of Amsterdam, Rodopi, 19–46. stylistic changes over the same period of time. Therefore, it is of great importance to investigate Biber Douglas. 1988. Variation across speech and writing. Cambridge: Cambridge University Press. them separately in order to obtain a better picture of the process of language change and the possible Biber Douglas and Finegan Edward. 1988. Drift in influences between different genres. three English genres from the 18th to the 20th cen- 4) As different language varieties show differ- tury: A multi-dimensional approach. In: M. Kyt, O. Ihalainen, and M. Rissanen, eds. Corpus linguis- ent trends of stylistic changes inside the same text tics, hard and soft. Proceedings of the Eighth Inter- categories, no general conclusions about the trends national Conference on English Language Research of stylistic changes should be made before a de- on Computerized Corpora: 83–101. Amsterdam: tailed investigation in each of the language vari- Rodopi. eties. Furthermore, results of the separate investi- Biber Douglas and Finegan Edward. 1989. Drift and gations of stylistic changes among different lan- the evolution of English style: A history of three guage varieties enable a better understanding of genres. Language, 65: 487–517. the noticed trends and their possible mutual influ- Connexor Oy. 2006a. Machinese Language Model. ence. However, it is important to ensure that the corpora of different language varieties are mutu- Connexor Oy. 2006b. Machinese Language Analysers. ally comparable and thus any similarities or dif- ferences among their trends of change are not due Corpas Pastor Gloria, Mitkov Ruslan, Afzal Navid and Pekar Viktor. 2008. Translation Universals: Do to different sampling methods and text selection. they exist? A corpus-based NLP study of conver- Finally, we demonstrated the possibility of us- gence and simplification. In Proceedings of the ing the Brown ‘family’ corpora and NLP tech- AMTA. Waikiki, Hawaii. niques for the investigation of diachronic stylis- Gordon A. Ian. 1966. The Movement of English Prose. tic changes. It has created a path for many other Bloomington: Indiana University Press. stylistic features to be investigated using the same corpora and utilising the full potential of the cur- Holmes I. David. 1994. Authorship Attribution. Com- rent state-of-the-art NLP tools and techniques. puters and the Humanities, 28(2): 87–106. Springer Netherlands. Acknowledgements Kroch Anthony. 2001. Syntactic change. In: M. Baltin and C. Collins, eds. The Handbook of Contempo- This project was supported by the European Com- rary Syntactic Theory. Malden, MA: Blackwells, mission, Education & Training, Erasmus Mundus: 699–730. 84 Leech Geoffrey. 2003. Modality on the move: 246–253. Association for Computational Linguis- the English modal auxiliaries 1961-1992. In: R. tics, Stroudsburg, PA, USA. Facchinetti, M. Krug and F. Palmer, eds. Modality in contemporary English. Berlin/New York: Mou- Senter, R. J. and Smith E. A. 1967. Automated read- ton de Gruyter, 223–240. ability index. Technical Report (AMRLTR-66-220). University of Cincinnati, Cincinnati: Ohio. Leech Geoffrey. 2004. Recent grammatical change in English: data, description, theory. In: K. Aijmer and Smith A. Edgar and Kincaid J. Peter. 1970. Derivation B. Altenberg, eds. Advances in Corpus Linguistics: and Validation of the Automated Readability Index Papers from the 23rd International Conference on for Use with Technical Materials. Human Factors: English Language Research on Computerized Cor- The Journal of the Human Factors and Ergonomics pora (ICAME 23) Gteborg 22-26 May 2002. Ams- Society, 12(5): 457–464. terdam: Rodopi, 61–81. Smith A. Joseph and Kelly Colleen. 2002. Stylis- Leech Geoffrey and Smith Nicholas. 2005. Extend- tic Constancy and Change Across Literary Corpora: ing the possibilities of corpus-based research on En- Using Measures of Lexical Richness to Date Works. glish in the twentieth century: a prequel to LOB and Computers & the Humanities, 36(4): 411–431. FLOB. ICAME Journal, 29: 83–98. Smith Nicholas. 2002. Ever moving on? The pro- Leech Geoffrey and Smith Nicholas. 2006. Recent gressive in recent British English. In: P. Peters, P. grammatical change in written English 1961–1992: Collins and A. Smith, eds. New frontiers of cor- some preliminary findings of a comparison of Amer- pus research: papers from the twenty first Interna- ican with British English. In: A. Renouf and A. Ke- tional Conference on English Language Research on hoe, eds. The Changing Face of Corpus Linguistics. Computerized Corpora, Sydney 2000. Amsterdam: Amsterdam: Rodopi, 186–204. Rodopi, 317–330. Mair Christian and Hundt Marianne. 1995. Why is Smith Nicholas. 2003a. A quirky progressive? A the progressive becoming more frequent in English? corpus-based exploration of the will + be + -ing A corpus-based investigation of language change in construction in recent and present day British En- progress. Zeitschrift fr Anglistik und Amerikanistik, glish. In: D. Archer, P. Rayson, A. Wilson and T. 43: 111–122. McEnery, eds. Proceedings of the Corpus Linguis- tics 2003 Conference: 714–723. Lancaster Univer- Mair Christian. 1997. The spread of the going-to- sity: UCREL Technical Papers. future in written English: a corpus-based investiga- tion into language change in progress. In: R. Hickey Smith Nicholas. 2003b. Changes in the modals and and St. Puppel, eds. Language history and linguistic semi-modals of strong obligation and epistemic ne- modelling: a festschrift for Jacek Fisiak on his 60th cessity in recent British English. In: R. Facchinetti, birthday. Berlin: Mouton de Gruyter, 1536–1543. M. Krug and F. Palmer, eds. Modality in contempo- rary English. Berlin/New York: Mouton de Gruyter, Mair Christian. 2002. Three changing patterns of verb 241–266. complementation in Late Modern English: a real- time study based on matching text corpora. English Stamou Constantina. 2008. Stylochronometry: Stylis- Language and Linguistics, 6: 105–131. tic Development, Sequence of Composition, and Relative Dating. Literary & Linguistic Computing, Mair Christian, Hundt Marianne, Leech Geoffrey and 23(2): 181–199. Smith Nicholas. 2002. Short term diachronic shifts in part-of-speech frequencies: a comparison of the Ule Louis. 1982. Recent progress in computer meth- tagged LOB and F-LOB corpora. International ods of authorship determination. Association of Lit- Journal of Corpus Linguistics, 7: 245–264. erary and Linguistic Computing Bulletin, 10(3): 73– 89. Mair Christian and Leech Geoffrey. 2006. Current change in English syntax. In: B. Aarts and A. Westin Ingrid and Geisler Christer. 2002. A multi- MacMahon, eds. The Handbook of English Linguis- dimensional study of diachronic variation in British tics, Ch.14. Oxford: Blackwell. newspaper editorials. ICAME Journal, 26: 133– 152. McCallum R. Douglas and Peterson L. James. 1982. Computer-based readability indexes. In Proceed- Westin Ingrid. 2002. Language Change in English ings of the ACM ’82 Conference: 44–48. New York, Newspaper Editorials. Amsterdam: Rodopi. NY. Samuelsson Christer and Voutilainen Atro. 1997. Comparing a linguistic and a stochastic tagger. Pro- ceedings of the 35th Annual Meeting of the Asso- ciation for Computational Linguistics and Eighth Conference of the European Chapter of the Asso- ciation for Computational Linguistics, (ACL ’98): 85 AVATecH: Audio/Video Technology for Humanities Research Sebastian Tschöpel Oliver Schreer Daniel Schneider Stefano Masneri Rolf Bardeli Fraunhofer HHI Fraunhofer IAIS http://www.iais.fraunhofer.de http://www.hhi.fraunhofer.de Peter Wittenburg Przemek Lenkiewicz Han Sloetjes Eric Auer MPI for Psycholinguistics MPI for Psycholinguistics http://www.mpi.nl/ http://www.mpi.nl/ Abstract The main challenge in the project is to tackle audio and video pattern recognition where In the AVATecH project the Max-Planck In- the standard methods based on stochastic engines stitute for Psycholinguistics (MPI) and the trained on large training sets cannot be applied to Fraunhofer institutes HHI and IAIS aim to noisy field and complex lab recordings because: significantly speed up the process of creating (1) there are only small training corpora availa- annotations of audio-visual data for humani- ble; (2) there are in general no models for the ties research. For this we integrate state-of-the- languages or visual setups in focus; (3) the re- art audio and video pattern recognition algo- rithms into the widely used ELAN annotation cordings are usually of limited quality, e.g., af- tool. To address the problem of heterogeneous fected by noise in the background or disadvanta- annotation tasks and recordings we provide geous lighting conditions; (4) there are no or on- modular components extended by adaptation ly few annotations that can be used to train a and feedback mechanisms to achieve competi- model. The central ideas in the AVATecH tive annotation quality within significantly less project are to adapt models to the given annota- annotation time. Currently we are designing a tion scenario and exploit iterative feedback from large-scale end-user evaluation of the project. the human annotator. 1 Introduction 2 System Landscape 1 The AVATecH project is a collaborative The system landscape of AVATecH is research project between the Max-Planck Insti- detailed in Figure 1. The Fraunhofer institutes tute for Psycholinguistics (MPI) on the one hand are technology providers delivering recognizers and the Fraunhofer Institutes HHI and IAIS on in form of executables. These recognizers are the other hand. The aim of the project is to ena- integrated into existing annotation tools using a ble researchers in the field of humanities to sig- common recognizer interface that is based on a nificantly speed up their annotation process. This derivate of the CMDI (Component Metadata process is inevitable, for example, for carrying Infrastructure) specification, developed within out deep linguistic studies (Wittenburg et al., the CLARIN research infrastructure project 2010; Masneri et al., 2010). (Váradi et al., 2008; Broeder et al., 2010). The To reach this goal the Fraunhofer institutes annotation tools are developed and maintained provide audio and video pattern recognition by the MPI. The interactive ELAN 2 tool is a technology for (semi-) automatic extraction of widely used, open source annotation tool with a content related annotations. By integrating them graphical frontend to annotate audiovisual into the common annotation process for linguis- content for linguistic research (Auer et al., 2010; tic research we expect a significant reduction of Wittenburg et al., 2006). ELAN is not only used the overall annotation time. High potential of by many of the MPI researchers but also by a lot such technologies and tools for increasing anno- of other researchers worldwide. The main areas tation speed has been shown in (Roy and Roy, of application include language documentation, 2009). 1 http://www.mpi.nl/avatech/ 2 http://www.lat-mpi.eu/tools/elan/ 86 Proceedings of Language Technologies for Digital Humanities and Cultural Heritage Workshop, pages 86–89, Hissar, Bulgaria, 16 September 2011. Figure 1. AVATecH System landscape sign language research and gesture research. An additional tool, ABAX, has been created in the 3.1 Annotation of field recordings AVATecH project. In contrast to ELAN it is Within the first scenario, researchers come used to perform a series of annotation tasks on back from an extensive field trip with tens of multiple files. ABAX provides a CMDI-interface hours of unstructured media data. We aim at as well. Researchers can use either ELAN or supporting annotation of arbitrary field record- ABAX to create enriched annotations using the ings with only little manual interaction. The re- recognizers provided by the Fraunhofer searchers provide their recordings to the analysis institutes. Researchers provide media files to be components via ELAN or ABAX in their usual annotated and, depending on the recognizer, working environment. If required, they can pro- existing annotations or additional feedback vide prior knowledge about the recordings, e.g., information, e.g., parameter settings, to optimize they can adjust analysis parameters or label a few the performance of the recognizers. segments for providing examples to a detection algorithm. After the analysis, they will obtain a 3 State of work pre-annotated set of field recordings, where they During the beginning of the AVATecH can quickly navigate to the portions of interest project, we carried out intensive corpora studies that requires more detailed manual annotation. to identify typical annotation scenarios and their We already integrated a number of recognizers to costs in terms of hours spend by the humanities address this task: researchers. The sample corpora provided by MPI consist of 38 sub-corpora with a total file Audio Recognizers • size of 730 GB and about 43,000 individual me- Audio Activity Detection • dia documents. We concluded that the material is Acoustic Segmentation highly varying in audio and video quality (from • Detection of Speech office or lab experiments with good recording • Speaker Diarization quality to field recordings in noisy environ- • Vowel and Pitch contour detection ments), in language, genre (from monologues to interviews and other discourse situations), and in Video Recognizers the amount of information that can be derived • Detection of Shot Cuts directly from the audio or video stream. This led • Extraction of Key frames to the conclusion that IAIS and HHI have to as- • Camera Motion / Motion Inside-the-Scene semble flexible solutions in order to cope with Detection the large variety of annotation problems. • Hands and Head Tracking In the first half of the project we mainly ad- dressed three types of annotation scenarios and 3.2 Annotation of interview recordings the utilization of user feedback. For a second scenario we exploit the resulting annotations of the first workflow described above. In this scenario, the researchers have a large set of interview recordings where they are 87 just interested in the responses of the intervie- researchers need to be able to rely on those au- wee. We add further prior knowledge in the form tomatic annotations that exclude data from the of speech examples of the interviewer, and create manual process. Hence, we investigate the poten- separate tiers for the interviewer and the subjects tial of each analysis component to support either of an interview situation. To address this task we an adaptation mechanism or a feedback-loop me- incorporate widely used state-of-the-art audio chanism or both. Furthermore, the graphical user analysis technology for the automatic detection interface will support fast correction of typical of specific speakers. To use this component the annotation errors produced by the recognizers. researcher must provide a few minutes of sam- By an adaptation mechanism we mean that the ples of the desired speaker. researchers provide examples of aspects they would like to detect, e.g., samples of a speaker 3.3 Annotation of sign language studio re- for automatic speaker detection. Alternatively, cordings they can choose from presets for typical acoustic In the third scenario, the researchers want to environments, e.g., different presets for the create gesture annotations based on a corpus acoustic segmentation of studio or field record- without any pre-annotations. The MPI-corpora ings. contain sign language studio recordings that can By a feedback-loop mechanism we mean be partitioned in two groups. The first group strategies where the user first runs a recognition consists of single person videos, where the sub- process, gives feedback about the quality of the ject can be filmed from several positions, e.g., result and then runs the process with the updated from above or facing the camera and at different information again. For example, for speaker camera distances. The other group consists of identification the user adapts the recognizer by interviews with two to four people in the scene, selecting some examples of the speaker, then none of them facing the camera. Resolution and runs the recognizer, and then verifies a number quality of the recordings vary heavily depending of segments. The recognizer uses this response to on the sub-corpus. adapt the algorithm before running the process Typical gesture annotations require the user to again. manually select the start and end point of each To support this, the user interface must sup- gesture as well as the appropriate descriptions. port various techniques to interact with the re- For each gesture, glosses for each hand can be cognizers, e.g., to quickly jump from segment to included, as well as mouth position and informa- segment, to allow a decision whether a segment tion about eye aperture, gaze direction, head is correctly labeled or not (feedback-loop), or to movements, etc. Accurate gesture analysis can be select segments that will be used for the adapta- an extremely time consuming task. In the project, tion mechanism. the videos are automatically prioritized, allowing User interaction is an essential part of our an- the researchers to decide which ones are worth notation workflows and it directly addresses the annotating without the necessity to view them in goal of reducing the overall annotation time advance. The aim is to provide the automatic since some of the originally unsupervised algo- extraction of low level features (like position of rithms will not be able to provide the necessary the hands during a movement, average speed of high-quality annotation. the hands, duration of a gesture), allowing the researchers to focus on higher level gesture anal- 4 Evaluation Phase ysis. We integrated a recognizer to automatically For the evaluation, we are interested in two sets estimate skin colour parameters and, building up of information: qualitative statements from the on this, a recognizer for automatic hands and human annotators that assess the annotation ex- head detection and tracking . The latter also de- perience when using the AVATecH system and a tects interaction between different body parts, for quantitative evaluation of the actual annotation example, when two hands join or an arm is over- speed-up compared to manual annotation. lapping the face. During the qualitative evaluation we are inter- 3.4 Using user-feedback for optimization ested how the work of annotators is supported by automatic analysis, and whether annotators think On the heterogeneous data in this project, that the system is beneficial. In the evaluation, some of the baseline recognizers perform poorly subjects annotate their own material with ELAN without additional adaptation. Moreover, in order supported by recognizers. We record their expe- to really speed up the annotation process, the 88 rience with a questionnaire. Within the qualita- nizers to finish processing. This can be achieved tive evaluation we aim to cover all three annota- with proper logging of the times when recogniz- tion tasks mentioned above. The questionnaire ers start and finish their execution. As population consists of 20 questions addressing the quality of we will incorporate five to ten advanced ELAN the ELAN interface, the productivity using re- users. cognizers during annotating, the quality of the delivered results and the overall experience. We 5 Future work aim to incorporate at least thirty MPI researchers After carrying out the study presented above we or students who want to annotate their data with plan to do iterations of recognizer advancements recognizer support. and subsequent user-reviews to further reduce Within quantitative evaluation, we want to the overall annotation time and to increase the measure the speed-up for annotation by using user satisfaction. Also we will add more recog- automatic tools. We ask a limited number of nizers for more specialized tasks such as lan- people to perform a specific annotation task for a guage-independent and -dependent forced align- subset of field recordings or studio recordings ment (already in an advanced stage of develop- from our corpus, both with and without support ment), acoustic and visual query-by-example. from automatic analysis. The annotation will Furthermore we expect advancements by careful- consist of labeling the speech of the interviewee, ly evaluating and implementing more ways of such that the researcher can quickly browse from adaption and user-feedback to overcome the dif- answer to answer (for interview recordings) and ficulties of heterogeneous and low-quality field labeling the gesture of the person in the video, so recordings. that the researcher can quickly see when a ges- ture begins, ends and what kind of motion is as- Acknowledgments sociated to it (for studio recordings). AVATecH is a joint project of Max Planck and To measure the annotation speed we have de- Fraunhofer, started in 2009 and funded by both fined a metric that can be used not only for our Max Planck Society and Fraunhofer Society. evaluation, but can give a good insight about the general annotation speed of a researcher. This is References not straightforward to assess, because researchers Stefano Masneri, Oliver Schreer, Daniel Schneider, et work with recordings of varying complexity (e.g. al. 2010. Towards semi-automatic annotations for having few or many relevant events per time video and audio corpora . Proc. CSLT Workshop, unit) and they are looking for different informa- LREC 2010. tion in them, hence creating very different anno- Peter Wittenburg, Eric Auer, Han Sloetjes, et al. 2010. tations (from very basic to multi-level, complex Automatic Annotation of Media Field Recordings . annotations with long descriptions). Proc. LaTECH, Workshop, ECAI 2010. Our general metric is based on two measures: Eric Auer, Albert Russel, Han Sloetjes, et al. 2010. 1) the number of created annotation blocks per ELAN as Flexible Annotation Framework for unit of time; 2) the length of the media file. Sound and Image Processing Detectors . Proc. These values considered together will allow as- LREC 2010. sessing the average annotation speed and com- Brandon C. Roy and Deb Roy. 2009. Fast plexity of annotation created for a given media Transcription of Unstructured Audio Recordings . file. Calculating these measures will be done by Proc. Interspeech 2009. extending ELAN to record the overall annotation Peter Wittenburg, Hennie Brugman, Albert Russel, et time from opening to closing the project file and al. 2006. ELAN: a Professional Framework for to log certain annotation events, e.g., “created a Multimodality Research . Proc. LREC 2006 new label” or “created a new segment”, with cor- Tamás Váradi, Steven Krauwer, Peter Wittenburg, et responding timestamps. However, if the same al. 2008. CLARIN: Common Language Resources data is annotated twice by the same subject, the and Technology Infrastructure . Proc. LREC 2008 second annotation will be biased as the subject Daan Broeder, Marc Kemps-Snijders, Dieter Van already knows the structure of the file. Therefore Uytvanck, et al. 2010. A Data Category Registry- we penalize the automatic annotation and do it and Component-based Metadata Framework . Proc. first. Moreover, we split the runs over two days LREC 2010 (1 stday: annotation with automatic tools, 2 nd day manual annotation). Also we measure active vs. passive annotation time, i.e., waiting for recog- 89 Handwritten Text Recognition for Historical Documents Veronica´ Romero, Nicolas´ Serrano, Alejandro H. Toselli, Joan Andreu Sanchez´ and Enrique Vidal ITI, Universitat Politecnica` de Valencia,` Spain vromero,nserrano,jandreu,atoselli,evidal @iti.upv.es { } Abstract HTR should not be confused with Optical Char- acter Recognition (OCR). Nowadays, OCR sys- The amount of digitized legacy documents tems are capable to recognizing text with a very has been rising dramatically over the last good accuracy (Breuel, 2008; Ratzlaff, 2003). years due mainly to the increasing num- However, OCR products are very far from offer- ber of on-line digital libraries publishing ing useful solutions to the HTR problem. They this kind of documents. The vast majority are simply not usable, since in the vast majority of them remain waiting to be transcribed of the handwritten documents, characters can by into a textual electronic format (such as no means be isolated automatically. HTR, spe- ASCII or PDF) that would provide histori- cially for historical documents, is a very difficult ans and other researchers new ways of in- task. To some extent HTR is comparable with dexing, consulting and querying them. In the task of recognizing continuous speech in a this work, the state-of-the-art Handwritten significantly degraded audio file. And, in fact, Text Recognition techniques are applied the nowadays prevalent technology for HTR bor- for the automatic transcription of these rows concepts and methods from the field of Auto- historical documents. We report results for matic Speech Recognition (ASR) (Rabiner, 1989) several ancient documents. as Hidden Markov Models (HMMs) (Bazzi et al., 1999) and n-Gram (Jelinek, 1998). The most im- 1 Introduction portant difference is that the input feature vector In the last years, huge amount of handwritten his- sequence of the HTR system represents a hand- torical documents residing in libraries, museums written text line image, rather than an acoustic and archives have been digitalized and have been speech signal. made available to the general public through spe- In this sense, the required technology should cialized web portals. The vast majority of these be able to recognize all the text elements (sen- documents, hundreds of terabytes worth of digital tences, words and characters) as a whole, with- image data, remain waiting to be transcribed into a out any prior segmentation of the image into textual electronic format that would provide histo- these elements. This technology is generally re- rians and other researchers new ways of indexing, ferred to as segmentation-free off-line Handwrit- consulting and querying them. ten Text Recognition (HTR) (Marti and Bunke, The automatic transcription of these ancient 2001; Toselli and others, 2004; Espana-Boquera˜ handwritten documents is still an incipient re- et al., 2011). search field that has been started to be explored Given that historical documents suffered from in recent years. For some time in the past decades, the typical degradation problems of this kind of the interest in Off-line Handwritten Text Recog- documents and, in order to obtain accurately tran- nition (HTR) was diminishing, under the assump- scription of them, different methods and tech- tion that modern computer technologies will soon niques of the document analysis and recognition make paper-based documents useless. However, field are needed. Among them are the layout anal- the increasing number of on-line digital libraries ysis and text line extraction methods, image pre- publishing large quantities of digitized legacy doc- processing techniques, lexical and language mod- uments has turned HTR up in an important re- eling and HMMs. In this paper we study the adap- search topic. tation/application of the above mentioned tech- 90 Proceedings of Language Technologies for Digital Humanities and Cultural Heritage Workshop, pages 90–96, Hissar, Bulgaria, 16 September 2011. niques on historical documents, testing the system where l is the word length of the sequence w and α on four sort of different ancient documents charac- and β are optimized for all the training sentences terized, among other things, by different handwrit- of the corpus. ten styles from diverse places and time periods. The HTR system used here follows the clas- This paper is divided as follows. First, the HTR sical architecture composed of three main mod- framework is introduced in section 2. Then, the ules: a document image preprocessing module, different corpora used in the experiments are de- in charge to filter out noise, recover handwritten scribed in subsection 3.1. The experiments and strokes from degraded images and reduce variabil- results are commented in subsection 3.2. Finally, ity of text styles; a line image feature extraction some conclusions are drawn in the section 4. module, where a feature vector sequence is ob- tained as the representation of a handwritten text 2 Handwritten Text Recognition line image; and finally a HMM training/decoding module, which obtains the most likely word se- The handwritten text recognition (HTR) prob- quence for the sequence of feature vectors (Bazzi lem can be formulated in a similar way to ASR, et al., 1999; Toselli and others, 2004). as the problem of finding the most likely word 2.1 Preprocessing sequence, w = (w1 w2 . . . wn), for a given handwritten sentence image represented by an ob- It is quite common for ancient documents to suffer servation sequence, x = (x1 x1 . . . xm), i.e., from degradation problems (Drida, 2006). Among w = argmax P (w x). Using the Bayes’ rule we these are the presence of smear, background of big w | can decompose this probability into two probabil- variations and uneven illumination, spots due to ities, P (x w) and P (w): the humidity or marks resulting from the ink that | goes through the paper (generally called bleed- wˆ = argmax P (w x) argmax P (x w)P (w) through). In addition, there are other kinds of dif- | ≈ | w w ficulties appearing in these pages as different font (1) types and sizes in the words, underlined and/or crossed-out words, etc. The combination of all P (x w) can be seen as a morphological-lexical | knowledge. It is the probability of the observa- these problems contributes to make the recogni- tion process difficult, and hence, the preprocessing tion sequence x given the word sequence w and is typically approximated by concatenated char- module quite essential. acter HMMs (Jelinek, 1998). On the other hand, The following steps take place in the prepro- P (w) represents a syntactic knowledge. It is the cessing module: first, the skew of each page is prior probability of the word sequence w and is corrected. We understand as “skew” the angle be- approximated by a word language model, usually tween the horizontal direction and the direction of n-grams (Jelinek, 1998). the lines on which the writer aligned the words. Then, a conventional noise reduction method is In practice, the simple multiplication of P (x | applied on the whole document image (Kaval- w) and P (w) needs to be modified in order to balance the absolute values of both probabilities. lieratou and Stamatatos, 2006), whose output is To this end a language model weight α (Gram- then fed to the text line extraction process which mar Scale Factor, GSF), which weights the influ- divides it into separate text lines images. The ence of the language model on the recognition re- method used for the latter case is based on the hor- sult, and an insertion penalty β (Word Insertion izontal projection profile of the input image. Local Penalty, WIP), which helps to control the word in- minimums in this projection are considered as po- sertion rate of the recognizer (Ogawa et al., 1998) tential cut-points located between consecutive text are used. In addition, log-probabilities are usually lines. When the minimum values are greater than used to avoid the numeric underflow problems that zero, no clear separation is possible. This prob- can appear using probabilities. So, Equation (1) lem has been solved using a method based in con- can be rewritten as: nected components (Marti and Bunke, 2001). Fi- nally, slant correction and size normalization are wˆ = argmax log P (x w) + α log P (w) + lβ applied on each separate line. More detailed de- w | scription can be found in (Toselli and others, 2004; (2) Romero et al., 2006). 91 2.2 Feature Extraction The feature extraction process approach used to obtain the feature vectors sequence follows similar ideas described in (Bazzi et al., 1999). First, a grid is applied to divide the text line image into M N × squared cells. M is chosen empirically and N is such that N/M equals the original line image as- pect ratio. Each cell is characterized by the follow- ing features: average gray level, horizontal gray level derivative and vertical gray level derivative. To obtain smoothed values of these features, an Figure 1: Example of feature-vector sequence and s s cell analysis window, centered at the cur- × HMM modeling of instances of the character “a” rent cell, is used in the computations (Toselli and within the Spanish word “cuarenta” (“forty”). others, 2004). The smoothed cell-averaged gray The model is shared among all instances of char- level is computed through convolution with two acters of the same class. The zones modeled by 1-d Gaussian filters. The smoothed horizontal each state show graphically subsequences of fea- derivative is calculated as the slope of the line ture vectors (see details in the magnifying-glass which best fits the horizontal function of column- view) compounded by stacking the normalized average gray level in the analysis window. The fit- grey level and its both derivatives features. ting criterion is the sum of squared errors weighted by a 1-d Gaussian filter which enhances the role of central pixels of the window under analysis. The handwritten text (without any kind of segmenta- vertical derivative is computed in a similar way. tion and represented by their respective observa- Columns of cells (also called frames) are pro- tion sequences) accompanied by the transcription cessed from left to right and a feature vector of these images into the corresponding sequence is constructed for each frame by stacking the of characters. This training process is carried three features computed in their constituent cells. out using a well known instance of the EM al- Hence, at the end of this process, a sequence of gorithm called forward-backward or Baum-Welch M 3 N-dimensional feature vectors (N normal- re-estimation (Jelinek, 1998). · ized gray-level components and N horizontal and Each lexical entry (word) is modeled by a vertical derivatives components) is obtained. Fig- stochastic finite-state automaton which represents ure 1 shows a representative visual example of all possible concatenations of individual charac- the feature vectors sequence for the Spanish word ters that may compose the word. By embedding “cuarenta” (“forty”) and how a continuous density the character HMMs into the edges of this automa- HMM models two feature vector subsequences ton, a lexical HMM is obtained. corresponding to the character “a”. Finally, the concatenation of words into text lines or sentences is usually modeled by a bi- 2.3 Recognition gram language model, with Kneser-Ney back-off smoothing (Katz, 1987; Kneser and Ney, 1995), Characters (or graphemes) are considered here as which uses the previous n 1 words to predict the the basic recognition units in the same way as − next one: phonemes in ASR, and therefore, they are mod- N eled by left-to-right HMMs. Each HMM state i 1 generates feature vectors following and adequate P (w) P (wi wi−n+1) (3) ≈ i=1 | − parametric probabilistic law; typically, a Gaussian Y Mixture. Thereby, the total amount of parameters This n-grams are estimated from the given tran- to be estimated depends on the number of states scriptions of the trained set. and their associated emission probability distribu- However, there are tasks in which the relation tions, which need to be empirically tuned to opti- of running words and vocabulary size is too low mize the overall performance on a given amount of causing that bi-gram language models hardly con- available training samples. As in ASR, character tributes to restrict the search space. This is the HMMs are trained from images of continuously case of one of the documents used in the experi- 92 ments reported in section 3.2 called “Index”. In the following subsection we describe the language model used for recognition in this specific task. Once all the character, word and language models are available, the recognition of new test sentences can be performed. Thanks to the ho- mogeneous finite-state (FS) nature of all these Figure 2: Language model for the Index task. models, they can be easily integrated into a sin- gle global (huge) FS model. Given an input se- quence of feature vectors, the output word se- corpora used in the experiments, as well as the dif- quence hypothesis corresponds to a path in the in- ferent measures and the obtained experimental re- tegrated network that produces the input sequence sults are summarized in the following subsections. with highest probability. This optimal path search is very efficiently carried out by the well known 3.1 Corpora and Transcription Tasks Viterbi algorithm (Jelinek, 1998). This technique allows for the integration to be performed “on the Four corpora with more or less similar HTR dif- fly” during the decoding process. ficulty were employed in the experiments. The first three corpora, CS (Romero et al., 2007), Ger- 2.3.1 “Index” Language Model mana (Perez´ et al., 2009) and Rodrigo (Serrano The Index task (see section 3.2) is related to the and Juan, 2010), consist of cursive handwritten transcription of a marriage register book and cor- page images in old Spanish from 16th and 19th responds to the transcription of the index at the be- century. The last corpus: Index (Romero et al., ginning of one of these books. This index registers 2011), was compiled from the index at the be- the page in which each marriage record is located. ginning of a legacy handwritten marriage register These marriage register books were usually used book. Figure 3 shows examples of each of them. for centuries to register marriages in ecclesiastical institutions and have been used recently for mi- Cristo-Salvador This corpus was compiled gratory studies. Their transcription is considered from the legacy handwriting document identified an interesting problem (Esteve et al., 2009). These as “Cristo-Salvador”, which was kindly provided 1 index pages have some regularities and a very easy by the Biblioteca Valenciana (BIVAL) . syntactic structure. The lines of the index pages It is composed of 53 text page images, written used in this study have first a man surname, then by only one writer and scanned at 300dpi. As has the word “ab” (that in old Catalan means “with”), been explained in section 2, the page images have then a woman surname and finally the page num- been preprocessed and divided into lines, resulting ber in which that marriage record was registered. in a data-set of 1, 172 text line images. The tran- In this work, in order to improve the accuracy scriptions corresponding to each line image are and speed up the transcription process of this doc- also available, containing 10, 911 running words ument, we have defined a very simple language with a vocabulary of 3, 408 different words. model that strictly accounts for the easy syntactic Two different partitions were defined for this structure of the lines. Figure 2 shows a graphical data-set. In this work we are going to use the par- representation of this language model. First a sur- tition called hard (Romero et al., 2007), where the name must be recognized, then the word “ab”, and test set is composed by 497 line samples belong- then another surname that can be preceded by the ing to the last 20 document pages, whereas the re- word “V.”. This letter means that the woman was maining 675 were assigned to the training set. widow and she was using her previous husband surname. Finally a page number or the quotation Germana The GERMANA corpus is the re- marks symbol must be recognized. sult of digitizing and annotating the Spanish manuscript “Noticias y documentos relativos a 3 Experimental Results Dona˜ Germana de Foix, ultima´ Reina de Aragon”´ written in 1891. It is a single-author book and a In order to assess the effectiveness of the above- limited-domain topic, and the original manuscript presented off-line HTR system on legacy docu- ments, different experiments were carried out. The 1http://bv2.gva.es 93 was well-preserved (Perez´ et al., 2009). It is com- posed of 764 pages, with approximately 21k lines. The page images were preprocessed and divided into lines 2. These lines have been transcribed by paleography experts, resulting in a data-set of 217k running words with a vocabulary of 30k words. To carry out the experiments, we have used the same partition described in (Perez´ et al., 2009), that only uses the first 180 pages of the corpus. Rodrigo The Rodrigo database corresponds to a manuscript from 1545 entitled ”Historia de Espana˜ del arc¸obispo Don Rodrigo“, and written in old Spanish by a single author. It is 853-page bound volume divided into 307 chapters. The manuscript was carefully digitized by ex- perts at 300 dpi and annotated in a procedure very similar to the one used for the Germana database. The complete annotation of Rodrigo comprises about 20K text lines and 231K running words form a lexicon of 17K words. In this work, the experiments have been carried out using the same partition described in (Serrano and Juan, 2010) Index This corpus was compiled from the in- dex at the beginning of a legacy handwritten mar- riage register book. This book was kindly pro- vided by the Centre d’Estudis Demografics` (CED) of the Universitat Autonoma` de Barcelona. As previously said, the lines in these pages have some syntactic regularities that could be used to reduce the human effort needed to carry out the transcrip- tion (Romero et al., 2011). The Index corpus was written by only one writer and scanned at 300 dpi. It was composed by 29 text pages. For each page, the GIDOC (Serrano et al., ) prototype was used to perform text block layout, line segmentation, and transcription. The results were visually inspected and the few line- separation errors were manually corrected, result- ing in a data-set of 1, 563 text line images, contain- ing 6, 534 running words from a lexicon of 1, 725 different words. Four different partitions were de- fined for cross-validation. 3.2 Results Figure 3: From top to bottom: Single-Writer Manuscripts from the XIX Century (CS and Ger- The quality of the transcriptions obtained with mana), Single-Writer Spanish manuscript from the off-line HTR system is given by the well- XVI century (Rodrigo) and index page of a mar- known Word Error Rate (WER). It is widely riage register book (Index). used in HTR (Toselli and others, 2004; Toselli et al., 2010; Espana-Boquera˜ et al., 2011) and in ASR (Jelinek, 1998). It is defined as the mini- 94 mum number of words that need to be substituted, field of Automatic Speech Recognition technol- deleted or inserted to convert a sentence recog- ogy, has been tested for historical documents. nized by the system into the corresponding refer- This HTR technology is based on Hidden Markov ence transcription, divided by the total number of Models using Gaussians as state emission proba- words in the reference transcription. bility function. The HMM-based HTR has a hi- The corresponding morphological (HMMs) and erarchical structure with character HMMs mod- language models (the different bi-grams and the elling the basic recognition units. These mod- special language model for the Index task) asso- els are concatenated forming word models, and ciated with each corpus were trained from their these in turn concatenated forming sentence mod- respective training images and transcriptions. Be- els. The HMM used in this work was furthermore sides, all results reported in Table 1 have been ob- enhanced by a language model incorporating lin- tained after optimizing the parameters values cor- guistic information beyond the word level. responding to the preprocessing, feature extraction Several tasks have been considered to assess and modeling processes for each corpus. this HTR approach. Considering all the difficul- Concerning to the CS corpus, the obtained ties involving the old handwritten documents used WER (%) results was 33.5 using in this case in the experiments, although the results achieved a closed-vocabulary. For the Germana cor- are not perfect they are really encouraging. In pus, the best WER achieved were around 8.9% addition, as previously commented, this accuracy and 26.9% using closed-vocabulary and open- could be enough for tasks such as document index- vocabulary respectively. Regarding the out-of- ing and searching or even could be used to derive vocabulary (OOV) words, it becomes clear that adequate metadata that describes roughly the con- a considerable fraction of transcription errors is tent of documents. Moreover, other applications due to the occurrence of unseen words in the test such as word-spotting can be easily implemented partition. More precisely, unseen words account using this segmentation-free HTR technology. In here for approximately 50% of transcription er- this sense, results are expected to be much more rors. Although comparable in size to GERMANA, precise than using the popular approaches which RODRIGO comes from a much older manuscript do not take advantage of the context of spotted (from 1545), where the typical difficult character- words. istics of historical documents are more evident. Finally, to obtain perfect transcriptions, instead The best WER figure achieved in this corpus un- of the heavy human-expert “post-editing” work, til the moment is around 36.5%, where most of that generally results inefficient and uncomfort- the errors are also caused by the occurrence of able to the user and also it is hardly accepted by OOV words. Respect to the Index corpus, in expert transcribers, computer assisted interactive which the transcription process used a specific lan- predictive solutions (Toselli et al., 2010) can be guage model, WER of 28.6% and 40.3% were ob- used. These solutions offer promising significant tained for closed-vocabulary and open-vocabulary improvements in practice and user acceptance. In respectively. these approaches, the user and the system work From the results we can see that current state- interactively in tight mutual cooperation to obtain of-the-art segmentation-free “off-line HTR” ap- the final perfect transcription of the given text im- proach produces word error rates as high as 9- ages. 40% with handwritten old documents, depending whether open or closed vocabulary is used. These results are still far from offering perfect solutions Acknowledgments to the transcription problem. However, this ac- curacy could be enough for indexing and search- Work supported by the Spanish Government ing tasks or even to derive adequate metadata to (MICINN and “Plan E”) under the MITTRAL roughly describe the ancient document contents. (TIN2009-14633-C03-01) research project and under the research programme Consolider Ingenio 4 Conclusions 2010: MIPRCV (CSD2007-00018), the Generali- tat Valenciana under grant Prometeo/2009/014 and In this paper the nowadays technology of HTR, FPU AP2007-02867. which borrows concepts and methods from the 95 Table 1: Basic statistics information from each corpus along with the WER(%) obtained using the segmentation-free off-line HTR system. Corpus CS GERMANA RODRIGO INDEX Language 19th C Sp. 19th C Sp. 16th C Sp. Old Catalan Lexicon 2 277 7 477 17 300 1 725 Lan. Model Train. Ratio 2.8 4.5 12.5 3.8 Characters 78 82 115 68 HMMs Train. Ratio 460 2 309 7 930 453 Open Vocabulary N N Y Y N Y WER (%) 33.5 8.9 26.9 36.5 28.6 40.3 References Daniel Perez,´ Lionel Tarazon,´ Nicolas´ Serrano, Francisco-Manuel Castro, Oriol Ramos-Terrades, I. Bazzi, R. Schwartz, and J. Makhoul. 1999. An and Alfons Juan. 2009. The germana database. Omnifont Open-Vocabulary OCR System for En- In Proceedings of the ICDAR’09, pages 301–305, glish and Arabic. IEEE Transactions on PAMI, Barcelona (Spain), July. IEEE Computer Society. 21(6):495–504. L. Rabiner. 1989. A Tutorial of Hidden Markov Mod- Thomas M. Breuel. 2008. The ocropus open source els and Selected Application in Speech Recognition. ocr system. In DRR, page 68150. Proceedings IEEE, 77:257–286. F. Drida. 2006. Towards restoring historic documents degraded over time. In Proceedings of the DIAL’06, E. H. Ratzlaff. 2003. Methods, Report and Survey IEEE Computer Society, pages 350–357. Washing- for the Comparison of Diverse Isolated Character ton, DC, USA. Recognition Results on the UNIPEN Database. In Proceedings of ICDAR ’03, volume 1, pages 623– S. Espana-Boquera,˜ M.J. Castro-Bleda, J. Gorbe- 628, Edinburgh, Scotland, August. Moya, and F. Zamora-Mart´ınez. 2011. Improv- ing offline handwriting text recognition with hybrid V. Romero, M. Pastor, A. H. Toselli, and E. Vidal. hmm/ann models. IEEE Transactions on PAMI, 2006. Criteria for handwritten off-line text size nor- 33(4):767–779. malization. In Proceedings of the VIIP 06, Palma de Mallorca, Spain, August. A. Esteve, C. Cortina, and A. Cabre.´ 2009. Long term trends in marital age homogamy patterns: Spain, V. Romero, A. H. Toselli, L. Rodr´ıguez, and E. Vidal. 1992-2006. Population, 64(1):173–202. 2007. Computer Assisted Transcription for Ancient Text Images. In Proocedings of the ICIAR 2007, F. Jelinek. 1998. Statistical Methods for Speech volume 4633 of LNCS, pages 1182–1193. Springer- Recognition. MIT Press. Verlag, Montreal (Canada), August. S. M. Katz. 1987. Estimation of Probabilities from V. Romero, Joan Andreu Sanchez,´ Nicolas´ Serrano, Sparse Data for the Language Model Component of and E. Vidal. 2011. Handwritten text recognition a Speech Recognizer. IEEE Transactions on Acous- for marriage register books. In Proceedings of the tics, Speech and Signal Processing, ASSP-35:400– 11th ICDAR, IEEE Computer Society. To be pub- 401, March. lished, September. E. Kavallieratou and E. Stamatatos. 2006. Improving Nicolas´ Serrano and Alfons Juan. 2010. The rodrigo the quality of degraded document images. In Pro- database. In Proceedings of the LREC 2010, Malta, ceedings of the DIAL ’06, pages 340–349, Washing- May 19-21. ton, DC, USA. IEEE Computer Society. N. Serrano, L. Tarazon,´ D. Perez,´ O. Ramos-Terrades, R. Kneser and H. Ney. 1995. Improved backing-off and A. Juan. The GIDOC prototype. In Proceed- for m-gram language modeling. volume 1, pages ings of the 10th PRIS 2010, pages 82–89, Funchal 181–184, Los Alamitos, CA, USA. IEEE Computer (Portugal). Society. A. H. Toselli et al. 2004. Integrated Handwriting U.-V. Marti and H. Bunke. 2001. Using a Statisti- Recognition and Interpretation using Finite-State cal Language Model to improve the preformance of Models. IJPRAI, 18(4):519–539. an HMM-Based Cursive Handwriting Recognition System. IJPRAI, 15(1):65–90. A.H. Toselli, V. Romero, M. Pastor, and E. Vidal. 2010. Multimodal interactive transcription of text A. Ogawa, K. Takeda, and F. Itakura. 1998. Balanc- images. Pattern Recognition, 43(5):1824–1825. ing acoustic and linguistic probabilites. In Proceed- ing IEEE CASSP, volume 1, pages 181–184, Seattle, WA, USAR, May. 96 Reducing OCR Errors in Gothic-Script Documents Lenz Furrer Martin Volk University of Zurich University of Zurich [email protected] [email protected] Abstract which already lead to a lower accuracy compared to antiqua texts – but also particular words, phrases In order to improve OCR quality in texts and even whole paragraphs are printed in anti- originally typeset in Gothic script, we qua font (cf. figure 1). Although we are lucky to have built an automated correction system have an OCR engine capable of processing mixed which is highly specialized for the given Gothic and antiqua texts, the alternation of the two text. Our approach includes external dic- fonts still has an impairing effect on the text qual- tionary resources as well as information ity. Since the interspersed antiqua tokens can be derived from the text itself. The focus lies very short (e. g. the abbreviation Dr.), their divert- on testing and improving different meth- ing script is sometimes not recognized by the en- ods for classifying words as correct or er- gine. This leads to heavily misrecognized words roneous. Also, different techniques are ap- due to the different shapes of the typefaces; for plied to find and rate correction candidates. example antiqua Landrecht (Engl.: “citizenship”) In addition, we are working on a web ap- is rendered as completely illegible I>aii 97 Proceedings of Language Technologies for Digital Humanities and Cultural Heritage Workshop, pages 97–103, Hissar, Bulgaria, 16 September 2011. Figure 2: Output XML of Abbyy Recognition Server 3.0, representing the word “Aktum”. errors. Section 4 is concerned with a web applica- serifProbability rates its probability of tion. having serifs on a scale from 0 to 255. The fourth character u shows an additional attribute 2 OCR system suspicious, which only appears if it is set to true. It marks characters that are likely to be We use Abbyy’s Recognition Server 3.0 for OCR. misrecognized (which is not the case here). Our main reason to decide in favour of this product We are pleased with the rich output of the OCR was its capability of converting text with both anti- system. Still, we can think of features that could qua and Gothic script. Another great advantage in make a happy user even happier. For example, comparison to Abbyy’s FineReader and other com- post-correction systems could gain precision if the mercial OCR software is its detailed XML output alternative recognition hypotheses made during file, which contains a lot of information about the OCR were accessible.1 Some of the information recognized text. provided is not very reliable, such as the attribute Figure 2 shows an excerpt of the XML output. “wordFromDictionary” that indicates if a word The fragment contains information about the was found in the internal dictionary: on the one five letters of the word Aktum (here meaning hand, a lot of vocabulary is not covered, whereas “governmental session”), which corresponds to on the other, there are misrecognized words that the first word in figure 1. The first two XML are found in the dictionary (e. g. Bandirektion, tags, par and line, hold the contents and further which is an OCR error for German Baudirektion, information of a paragraph and a line respectively, Engl.: “building ministry”). While the XML at- e. g. we can see that this paragraph’s alignment tribute “suspicious” (designating spots with low is centered. Each character is embraced by a recognition confidence) can be useful, the “char- charParams tag that has a series of attributes. Confidence” value does not help a lot with locating The graphical location and dimensions of a letter erroneous word tokens. are described by the four pixel positions l, t, r An irritating aspect is, that we found the dehy- and b indicating the left, top, right and bottom phenation to be done worse than in the previous borders. The following boolean-value attributes engine for Gothic OCR (namely FineReader XIX). specifie if a character is at a left word boundaries Words hyphenated at a line break (e. g. fakul- tative (wordStart), if the word was found in the in- on the fourth line in figure 1) are often split into ternal dictionary (wordFromDictionary) and two halves that are no proper words, thus look- if it is a regular word or rather a number or a ing up the word parts in its dictionary does not punctuation token (wordNormal, wordNumeric, help the OCR engine with deciding for the cor- wordIdentifier). charConfidence has a value between 0 and 100 indicating the recog- 1Abbyy’s Fine Reader Software Developer Kit (SDK) is nition confidence for this character, while capable of this. 98 rect conversion of the characters. Since Abbyy’s an automated approach to the post-correction. The systems mark hyphenation with a special charac- great homogeneity of the texts concerning the lay- ter when it is recognised, one can easily determine out as well as the limited variation in orthography the recall by looking at the output. We encoun- are a good premise to achieve remarkable improve- tered the Recognition Server’s recall to be signif- ments. Having in mind the genre of our texts, we icantly lower than that of FineReader XIX, which followed the idea of generating useful documents is regrettable since it goes along with a higher error ready for reading and resigned from manually cor- rate in hyphenated words. recting or even re-keying the complete texts, as did The Recognition Server provides dictionaries e. g. the project Deutsches Textarchiv3, which has for various languages, including so-called “Old a very literary, almost artistic standard in digitizing German”.2 In a series of tests the “Old Ger- first-edition books from 1780 to 1900. man” dictionary has shown slightly better results The task resembles that of a spell-checker as than the Standard German dictionary for our texts. found in modern text processing applications, with Some, but not all of the regular spelling varia- two major differences. First, the scale of our text tions compared to modern orthography are cov- data – with approximately 11 million words we ex- ered by this historical dictionary, e. g. old Ger- pect more than 300,000 erroneous word forms – man Urtheil (in contrast to modern German Urteil, does not allow for an interactive approach, asking Engl.: “jugdement”) is found in Abbyy’s internal a human user for feedback on every occurrence dictionary, whereas reduzirt (modern German re- of a suspicious word form. Therefore we need duziert, Engl.: “reduced”) is not. It is also possi- an automatic system that can account for correc- ble to include a user-defined list of words to im- tions with high reliability. Second, dating from prove the recognition accuracy. However, when the late 19th century, the texts show historic or- we added a list of geographical terms to the sys- thography, which differs in many ways from the tem and compared the output of before and after, spelling encountered in modern dictionaries (e. g. there was hardly any improvement. Although the historic Mittheilung vs. modern Mitteilung, Engl.: words from our list were recognized more accu- “message”). This means that using modern dictio- rately, the overall number of misrecognized words nary resources directly cannot lead to satisfactory was almost completely compensated by new OCR results. errors unrelated to the words added. Additionally, the governmental resolutions con- All in all, Abbyy’s Recognition Server 3.0 tain, typically, many toponymical references, i. e. serves our purposes of digitizing well, especially geographical terms such as Steinach (a stream or in respect of handling large amounts of data and place name) or Schwarzenburg (a place name). for pipeline processing. Many of these are not covered by general- vocabulary dictionaries. We also find regional pe- 3 Automated Post-Correction culiarities, such as pronunciation variants or words that are not common elsewhere in the German spo- The main emphasis of our project is on the post- ken area, e. g. Hülfstrupp vs. standard German correction of recognition errors. Our evaluation of Hilfstruppe, Engl.: “rescue team”. Of course there a limited number of text samples yielded a word is also a great amount of genre-specific vocabu- accuracy of 96.7 %, which means that one word lary, i. e. administrative and legal jargon. We are out of 30 contains an error (as e. g. Regieruug in- hence challenged to build a fully-automated cor- stead of correct Regierung, Engl.: “government”). rection system with high precision that is aware of We aim to significantly reduce the number of mis- historic spelling and regional variants and contains recognized word tokens by identifying them in geographical and governmental language. the OCR output and determining the originally in- tended word with its correct spelling. 3.1 Classification In order to correct a maximum of errors in the The core task of our correction system with respect corpus with the given resources, we decided for to its precision is the classification routine, that 2The term has not to be confused with the linguistic epoch determines the correctness of every word. This Old High German, which refers to texts from the first mille- part is evidently the basis for all correcting steps. nium A. D. Abbyy’s “Old German” seems to cover some or- thographical variation from the 19th, maybe the 18thcentury. 3see www.deutschestextarchiv.de 99 Misclassifications are detrimental, as they can lead events in the administered area) and the recogni- to false corrections. At the same time it is also tion confidence values of the OCR engine. Every the hardest task, as there are no formal criteria word is either judged as correct or erroneous ac- that universally describe orthography. Reynaert cording to the information we gather from the var- (2008) suggests building a corpus-derived lexicon ious resources. that is mainly based on word frequencies and the The historic and regional spelling deviations are distribution of similarly spelled words. However, modelled with a set of handwritten rules describ- this is sometimes misleading. For example, saum- ing the regular differences. For example, with a selig (Engl.: “dilatory”) is a rare but correct word, rule stating that the sequence th corresponds to t in which is found only once in the whole corpus, modern spelling, the old form Mittheilung can be whereas Liegenschast is not correct (in fact, it is turned into the standard form Mitteilung. While misrecognized for Liegenschaft, Engl.: “real es- the former word is not found in the dictionary, the tate”), but it is found sixteen times in this erroneous derived one is, which allows for the assumption form. This shows that the frequency of word forms that Mittheilung is a correctly spelled word. alone is not a satisfactory indicator to distinguish The classification method is applied to single correct words from OCR errors; other information words, which are thus considered as a correct or has to be used as well. erroneous form separately and without their origi- An interesting technique for reducing OCR er- nal context. This reduces the number of correction rors is comparing the output of two or more differ- candidates significantly, as not every word is to be ent OCR systems. The fact that different systems handled, but only the set of the distinct word forms. make different errors is used to localize and cor- However, this limits the correction to non-word er- rect OCR errors. For example, Volk et al. (2010) rors, i. e. misrecognized tokens that cannot be in- achieved improvements in OCR accuracy by com- terpreted as a proper word. For example, Fcuer is bining the output of Abbyy FineReader 7 and Om- clearly a non-sense word, which should be spelled nipage 17. However, this approach is not applica- Feuer (Engl.: “fire”). In contrast, the two word ble to our project for the simple reason that there forms Absatze and Absätze (Engl.: “paragraph”) is to our knowledge, for the time being, only one are two correct morphological variants of the same state-of-the-art OCR system that recognizes mixed word, which are often confused during OCR. To antiqua / Gothic script text. decide which one of the two variants is correct in The complex problem of correcting historical a particular occurrence, the word has to be judged documents with erroneous tokens is addressed by in its original context, which is outside the scope Reffle (2011) with a two-channel model, which in- of our project. tegrates spelling variation and error correction into The decision for the correctness of a word is one algorithm. To cover the unpredictable num- primarily based on the output of the German ber of spelling variants of texts with historical or- morphology system Gertwol by Lingsoft, in that thography (or rather, depending on their age, texts we check every word for its analyzability. Al- without orthography), the use of static dictionaries though the analyses of Gertwol are reliable in most is not reasonable as they would require an enor- cases, there are two kinds of exceptions that can mous size. Luckily, the orthography of our corpus lead to false classifications. First, there are cor- is comparatively modern and homogenous, which rect German words that are unknown to Gertwol, means that there is a manageable number of devi- such as Latin words like recte (Engl.: “right”) or ations in comparison to modern spelling and be- proper names. Second, sometimes Gertwol finds tween different parts of the corpus. analyses for misrecognized words, e. g. correct In our approach, we use a combination of vari- Regierungsrat (Engl.: “member of the govern- ous resources for this task, such as a large dictio- ment”) is often rendered Negierungsrat, which is nary system for German that covers morphological then analysed as a compound of Negierung (Engl.: variation and compounding (such as ging, a past “negation”) and Rat (Engl.: “councillor”). To form of gehen, Engl.: “to go”, or German Nieder- avoid these misclassifications we apply heuristics druckdampfsystem, a compound of four segments which include word lists for known frequent is- meaning “low-pressure steam system”), a list of sues, the frequency of the word form or the feed- local toponyms (since our texts deal mostly with back values of the OCR system concerning the 100 recognition confidence. rank frequency {correct}–{recognized} 1 49 {u}–{n} 3.2 Correction 2 38 {n}–{u} The set of all words recognized as correct words 3 14 {a}–{ä} plus their frequency can now serve as a lexicon 4 14 {}–{.} for correcting erroneous word tokens. This corpus- 5 13 {d}–{b} derived lexicon is by nature highly genre-specific, 6 11 {s}–{f} which is desirable. On the other hand, rare words … … … might remain uncorrected if all of their few occur- 140 1 {o}–{p} rences happen to be misspelled, in which case there 141 1 {r}–{?} will be no correction candidate in the lexicon. Due 142 1 {§}–{8} to the repetitive character of the texts – administra- 143 1 {ä}–{ö} tive work involves many recurrent issues – there is also a lot of repetition in the vocabulary across the Table 1: Either ends of a ranked list, showing char- corpus. This increases the chance that a word mis- acter errors and their frequency. recognized in one place has a correct occurrence in another. The procedure of finding correction candidates is algorithmically complex, since the orthographi- Figure 4: Gothic characters of the Schwabacher cal similarity of words is not easily described in a Fraktur that are frequently confused by the OCR formal way. There is no linear ordering that would system. From left to right, the characters are: s (in group together similar words. In the simplest ap- its long form), f, u, n, u or n, B, V, R, N. proach to searching for similar words in a large list, every word has to be compared to every other word, which has quadratic complexity regarding ically recognized text, one will inevitably notice the number of words. In our system we avoid this that a small set of errors accounts for the majority with two different approaches that are applied sub- of misrecognized characters. At character level, sequently. In the first step, only a limited set of OCR errors can be described as substitutions of character substitutions is performed, whereas the single characters. Thus, by ranking these error second step allows for a broader range of devia- types by their frequency, it can be shown that al- tions between an erroneous form and its correction. ready a small sample of OCR errors resembles Zip- fian distribution,4 as is demonstrated in figure 3. For example, the 20 most frequent types of errors, which is less than 14 % of the 143 types encoun- tered, sum up to 50 % of all occurrences of char- acter errors. Table 1 shows the head of this error list as well as its tail. Among the top 6 we find pairs like u and n that are also often confounded when recognizing antiqua text as well as typical Gothic-script confusions like d–b or s–f. Figure 4 illustrates the optical similarity of certain charac- ters that can also challenge the human eye. For ex- ample, due to its improper printing, the fifth letter cannot be determined clearly as u or n. Figure 3: OCR errors ranked by their frequency, For our correction system we use a manually from a sample text of approx. 50,000 characters edited substitution table that is based on these ob- length. servations. The substitution pairs are inverted and used as replacement operations that are applied to the misspelled word forms. The spelling vari- 3.2.1 Substitution of similar characters 4Zipf’s Law states that, given a large corpus of natural lan- In this approach we attempt to undo the most com- guage data, the rank of each word is inversely proportional to mon OCR errors. When working with automat- its frequency. 101 ants produced are then searched for correct words with ersten, but only one trigram (ten) is shared that can be found in our dictionary. For example, with warten. given an erroneous token sprechcn and a substitu- The correction candidates found with this tion pair e, c stating that recognized c can represent method are further challenged, e. g. by using the actual e, the system produces the variants spree- Levenshtein distance as a similarity measure and hcn, sprechen and spreehen. Consulting the dictio- by rejecting corrections that affect the ending of a nary finally uncovers sprechen (Engl.: “to speak”) word.6 as a correct word, suggesting it as a correction for garbled sprechcn. 3.3 Evaluation In order to measure the effect of the correction 3.2.2 Searching shared trigrams system, we manually corrected a random sample In a second step, we look for similar words more of 100 resolutions (approximately 35,000 word to- generally. This is applied to the erroneous words kens). Using the ISRI OCR-Evaluation Frontiers that could not be corrected by the method de- Toolkit (Rice, 1996), we determined the recogni- scribed above. There are various techniques to tion accuracy before and after the correction step. efficiently find similar words in large mounds of As can be seen in table 2, the text quality in terms data, such as Reynaert’s (2005) anagram hashing of word accuracy could be improved from 96.72 % method, that treats words as multisets of words to 98.36 %, which means that the total number of and models edit operations as arithmetic functions. misrecognized words was reduced by half. Mihov and Schulz (2004) combine finite-state au- tomata with filtering methods that include a-tergo 4 Crowd Correction dictionaries. Inspired by the Australian Newspaper Digitisation In our approach, we use character n-grams to Program (Holley, 2009) and their online crowd- find similar correct words for each of the erroneous correction system, we are setting up an online ap- words. Since many of the corrections with a close plication. We are working on a tool that allows for editing distance have already been found in the browsing through the corpus and reading the text in previous step, we have to look for corrections in a a synoptical view of both the recognized plain text wider spectrum now. In order to reduce the search and a scan image of the original printed page. Our space, we create an index of trigrams that points online readers will have the possibility to immedi- to the lexicon entries. Every word in the lexicon ately correct errors in the recognized text by click- is split into overlapping sequences of three charac- ing a word token for editing. The original word in ters, which are then stored in a hashed data struc- the scan image is highlighted for convenience of ture. For example, from ersten (Engl.: “first”) we the user, which permits fast comparison of image get the four trigrams ers, rst, ste and ten. Each and text, so the intended spelling of a word can be trigram is used as a key that points to a list of all verified quickly. words containing this three-letter sequence, so ten Crowd correction combines easy access of the not only leads to ersten, but also to warten (Engl.: historic documents with manual correcting. Simi- “wait”) and many other words with the substring lar to the concept of publicly maintained services ten. as Wikipedia and its followers, information is pro- Likewise, all trigrams are derived from each er- vided free of charge. At the same time, the user roneous word form. All lexicon entries that share has the possibility to give something back by im- at least one trigram with the word in question can thus be accessed by the trigram index. The lexicon 6Editing operations affecting a word’s ending are a deli- cate issue in an inflecting language like German, since it is entries found are now ranked by the number of tri- likely that we are dealing with morphological variation in- grams shared with the error word. To stay with stead of an OCR error. Data sparseness of rare words can the previous example, a misrecognized word form lead to unfortunate constellations. For example, the adjective 5 form innere (Engl.: “inner”) is present in the lexicon, while rrsten has three trigrams (rst, ste, ten) in common the inflectional variant innern is lacking. However, the latter form is indeed present in the corpus, but only in the misrecog- 5The correction pair ersten–rrsten is not found in the first nized form iunern. As innere is the only valuable correction step although its editing distance is 1 and thus minimal. But candidate in this situation, the option would be changing mis- since the error {e}–{r} has a low frequency, it is not con- spelled iunern into grammatically inapt innere. For the sake tained in the substitution table, which makes this correction of legibility, an unorthographical word is preferred to an un- invisible for the common-error routine. grammatical form. 102 plain (errors) corrected (errors) improvement character accuracy 99.09 % (2428) 99.39 % (1622) 33.2 % word accuracy 96.72 % (1165) 98.36 % (584) 49.9 % Table 2: Evaluation results of the automated correction system. proving the quality of the corpus. It is important Stephen V. Rice. 1996. Measuring the Accuracy of to keep the technical barrier low, so that new users Page-Reading Systems. Ph. D. dissertation, Univer- can start correcting straight away without needing sity of Nevada, Las Vegas, NV. to learn a new formalism. With our click-and-type Martin Volk, Torsten Marek, and Rico Sennrich. 2010. tool the access is easy and intuitive. However, in Reducing OCR Errors by Combining Two OCR Sys- consequence, the scope of the editing operations tems. ECAI 2010 Workshop on Language Technol- ogy for Cultural Heritage, Social Sciences, and Hu- is limited to word level, it does not, for example, manities (LaTeCH 2010), Lisbon, Portugal, 16 Au- allow for rewriting whole passages. gust 2010 – 16 August 2010, 61–65. 5 Conclusion With this project we demonstrate how to build a highly specialized correction system for a spe- cific text collection. We are using both general and specific resources, while the approach as a whole is widely generalizable. Of course, work- ing with older texts with less standardized orthog- raphy than late 19thcentury governmental resolu- tions may lead to a lower improvement, but the techniques we applied are not bound to a cer- tain epoch. In the crowd correction approach we see the possibility to further improve OCR output in combination with online access of the historic texts. References Rose Holley. 2009. How Good Can It Get? Analysing and Improving OCR Accuracy in Large Scale His- toric Newspaper Digitisation Programs. D-Lib Magazine, 15(3/4). Stoyan Mihov and Klaus U. Schulz. 2004. Fast Ap- proximate Search in Large Dictionaries. Computa- tional Linguistics 30(4):451–477. Ulrich Reffle. 2011. Efficiently generating cor- rection suggestions for garbled tokens of histor- ical language. Natural Language Engineering 17(2):265–282. Martin Reynaert. 2005. Text-Induced Spelling Correc- tion. Ph. D. thesis, Tilburg Universtiy, Tilburg, NL. Martin Reynaert. 2008. Non-Interactive OCR Post- Correction for Giga-Scale Digitization Projects. Proceedings of the Computational Linguistics and Intelligent Text Processing 9th International Confer- ence, CICLing 2008 Berlin, 617–630. 103 Author Index Agre, Gennady, 51 Schreer, Oliver, 86 Auer, Eric, 19, 86 Serrano, Nicolas, 90 Simov, Kiril, 51 Bardeli, Rolf, 86 Sloetjes, Han, 86 Bernardi, Raffaella, 11 Stajner,ˇ Sanja, 78 Birnbaum, David, 57 Staykova, Kamenka, 51 Bollmann, Marcel, 34 Stehouwer, Herman, 19 Chechev, Milen,3 Toselli, Alejandro H., 90 Tschopel,¨ Sebastian, 86 Damova, Mariana,3 Dannells,´ Dana,3 Vald, Ed, 11 Dimitrova, Ludmila, 43 Vidal, Enrique, 90 Dimitrova, Tsvetana, 57 Volk, Martin, 97 Dipper, Stefanie, 34 Dutsova, Ralitsa, 43 Wittenburg, Peter, 86 Eckhoff, Hanne Martine, 57 Enache, Ramona,3 Furrer, Lenz, 97 Galic Kakkonen, Gordana, 62 Gruszczynski,´ Włodzimierz, 27 Kakkonen, Tuomo, 62 Kokkinakis, Dimitrios, 70 Le, Dieu-Thu, 11 Lenkiewicz, Przemek, 86 Malm, Mats, 70 Masneri, Stefano, 86 Miltenova, Anissava, 57 Mitkov, Ruslan, 78 Ogrodniczuk, Maciej, 27 Osenova, Petya, 51 Panova, Rumiana, 43 Petran, Florian, 34 Prosz´ eky,´ Gabor,´ 1 Romero, Veronica, 90 Sanchez, Joan Andreu, 90 Schneider, Daniel, 86 105не мо’га да ~м на очи’те си
hierarchy of nodes and each node is an object in