Language Resources, Taxonomies and Metadata Lothar Lemnitzer12 and Erhard Hinrichs1 and Andreas Witt3 1 Seminar für Sprachwissenschaft, Universität Tübingen, Germany 2 Berlin-Brandenburgische Akademie der Wissenschaften, Berlin, Germany 3 Institut für Deutsche Sprache, Mannheim, Germany Abstract. In this paper we present an approach to faceted search in large language resource repositories. This kind of search which enables users to browse through the repository by choosing their personal se- quence of facets heavily relies on the availability of descriptive metadata for the objects in the repository. This approach therefore informs the col- lection of a minimal set of metatdata for language resources. The work described in this paper has been funded by the EC within the ESFRI infrastructure project CLARIN4. 1 Introduction The ultimate objective of the ESFRI infrastructure project CLARIN is to create a European federation of existing digital repositories that include language-based data, to provide uniform access to the data, wherever it be stored, and to pro- vide existing language and speech technology tools as web services to retrieve, manipulate, enhance, explore and exploit the data. The primary target audience is researchers in the humanities and social sciences and the aim is to cover all languages relevant for the user community. The objective of the current CLARIN Preparatory Phase Project (2008-2010) is to lay the technical, linguistic and or- ganisational foundations, to provide and validate specications for all aspects of the infrastructure (including standards, usage, IPR) and to secure sustainable support from the funding bodies in the (now 23) participating countries for the subsequent construction and exploitation phases beyond 2010. One of the major tasks of the CLARIN project is to produce a broad and detailed survey of language resources and tools. The rationale for this task is threefold: 1. it serves as the empirical basis for the prototypical integration of resources and tools into an emerging web service infrastructure; 2. it serves as the empirical basis for the development of a Basic Language Resource tool Kit (BLARK), that primarily serves the research needs of humanities and social science scholars; 4 Grant agreement no. 212230; we are grateful to the EC for the generous funding without which the reported work would not have been possible. We also would like to thank Peter Wittenburg and two anonymous reviewers for their helpful remarks on earlier versions of this paper. 3. it serves as the empirical basis for assessing to what extent existing language resources conform to existing standards for language resources and metadata Besides these three project-internal reasons the data collected in this survey will of course constitute a valuable, freely accessible resource for any potential users of LRs and tools, especially for humanities and social science researchers. The purpose of this paper is to specify the requirements for user-friendly ways of accessing information about sets of language resources that are of interest to humanities and social science scholars. It surveys existing web-based front-ends, such as registries and catalogues, and suggests ways of enhancing the existing functionalities of such front-ends. Such functionalities are intimately tied to the types of metadata that are available for individual language resources. So as to not overburden resource providers with unnecessarily complex and detailed questionnaires for metadata collection, a minimum set of metadata needs to be dened. The specication of this set will be partly informed by the predened categories and hierarchies and will otherwise conform to metadata standards such as TEI header (Text Encoding Initiative, cf. www.tei-c.org), Dublin Core cf. http://dublincore.org/) and IMDI (ISLE Metadata Initiative, cf. http: //www.mpi.nl/IMDI/). This paper is related to the eorts on the development of the CLARIN reg- istry infrastructure. It therefore addresses the eorts towards an interoperable language resource and metadata federation from the point of view of the lan- guage resource infrastructure building. The purpose of a metadata federation infrastructure is to provide information about and access to available language resources and services in a systematic fashion. The development of such a fed- eration, thus, represents an important prerequisite for the interoperable infra- structure of language tools and resources that CLARIN aims to develop. It is important for the target users to be able to navigate, i.e. browse and query, a complex repository of tools and resources. This presupposes that the tools and resources are categorized in a uniform manner and that the categories provide the basis of conceptual hierarchies that best reect the information and research needs of the potential users of the CLARIN infrastructure. This nat- urally leads to the idea of providing dierent views or facets of the available tools and resources along dierent dimensions. Such facets are likely to include the dierent languages, modalities of language (spoken, written, multimodal), intended user groups (e.g. linguists, historians, social scientists) and contexts of use (e.g. information retrieval, machine translation, and speech recognition). Our work will be informed by existing eorts and initiatives for cataloguing and categorizing language tools and resources. Therefore, in section 2 we will present and review prominent examples of existing registries (DFKI, ACL) and of existing catalogues (ELRA) as well as the IMDI metadata browser developed by the Max Planck Institute for Psycholinguistics in Nijmegen. In section 3, we will present a detailed proposal of a set of hierarchically structured views of tools and resources. At the moment, several initiatives are actively working on registry structures. The important work in this eld is carried out by the ISO TC37 and by the Max Planck Institute for Psycholinguistics in Nijmegen. Some relevant background information on the state of the ISO approach can be found in: ISO TC37/SC4: Infrastructure Note on Registry Databases5; Data Category Registry DCR2 Requirements6. The Infrastructure Note on Registry Databases introduces a separation of reg- istries into registries of the resources and relation registries. Given this separa- tion, the views of tools and resources would be found in one or more relation registries, whereas the registry of resources contains all the metadata directly associated to the resources. 2 What web-based front-ends are available? In the eld of computational linguistics (CL) a wide variety of tools and frame- works for processing natural language have been and will be developed. Since an inuential branch in CL also deals with the development and the application of statistical methods, there is also a need for tools to access and process large quantities of data. Moreover, language resources, especially corpora and lexica, play an important role in linguistic research. In response to a rising need to get an overview of existing methods, tools, resources and frameworks, some institu- tions and organisations gathered information on tools and resources in registries and made it available on the web. These registries should be considered as build- ing blocks or reference models for the CLARIN registry. The main purpose of the existing registries is to support humans in nding what they are looking for. This will also be an important use case of the search and browsing facilities on federated metadata provided by CLARIN. The CLARIN infrastructure will also support the searching for resources by non-human agents, e.g. automatic processes. One feature that all the registries we investigated share is that they either do not provide multiple views on the same resources or provide them only in a very limited way. This means, that the tools and resources catalogued in the existing registries are not browsable according to several dierent perspec- tives arising from dierent information needs. The infrastructure we are going to establish will not only allow the presentation of the resources along multiple views. It should rather be possible for users to browse the repository by drilling into several category trees to arrive at a level of specity which they consider to be sucient for their information needs. 2.1 DFKI software registry The Natural Language Software Registry (NLSR, cf. http://registry.dfki. de/ ) was developed and set up by the German Research Institute on Arti- cial Intelligence (DFKI), which is also a CLARIN partner. The DFKI registry 5 cf. [4]. 6 cf. [3]. presents information on natural language processing software. It lists academic as well as commercial software. The NLSR can be seen as a taxonomy, hence its structure is a tree. At the leaves, the products are listed. Each product is associated with a set of information, amongst them the languages for which the software can be used, the terms on which it can be acquired, its price and the name of a contact person. In addition to the NLP software, NLSR also lists some other Natural Language Resources, e.g. corpora, but only to a limited extent. The NSLR does not consider it to be in their focus to present an exhaustive overview of resources. The users who are interested in these resources are referred to other institutions, especially the ELRA (see section 2.3). On the left-hand side of the start page and on all the other pages of this website a navigation panel is shown (see g 1). It allows the user to navigate to several areas. The user can submit new resources (item 4 in the navigation panel) or search for resources (item 3). The taxonomical structure is accessible through item 2. Fig. 1. The DFKI Software Registry The tools are grouped into eight categories, e.g. tools for (manually) anno- tating resources, tools for processing of spoken or written language data, evalu- ation tools etc. The groups Annotation tools and Multimedia do not contain any sub-groups. Evaluation tools is subdivided in three categories, i.e. Evaluation of Machine Translation, Evaluation of Parsers and Evaluation of Speech Synthe- sis.