Unlocking the Semantics of Multimedia Presentations in the Web with the Multimedia Metadata Ontology
Total Page:16
File Type:pdf, Size:1020Kb
Unlocking the Semantics of Multimedia Presentations in the Web with the Multimedia Metadata Ontology Carsten Saathoff Ansgar Scherp Nr. 19/2009 Arbeitsberichte aus dem Fachbereich Informatik Die Arbeitsberichte aus dem Fachbereich Informatik dienen der Darstellung vorläufiger Ergebnisse, die in der Regel noch für spätere Veröffentlichungen überarbeitet werden. Die Autoren sind deshalb für kritische Hinweise dankbar. Alle Rechte vorbehalten, insbesondere die der Übersetzung, des Nachdruckes, des Vortrags, der Entnahme von Abbildungen und Tabellen – auch bei nur auszugsweiser Verwertung. The “Arbeitsberichte aus dem Fachbereich Informatik“ comprise preliminary results which will usually be revised for subsequent publication. Critical comments are appreciated by the authors. All rights reserved. No part of this report may be reproduced by any means or translated. Arbeitsberichte des Fachbereichs Informatik ISSN (Print): 1864-0346 ISSN (Online): 1864-0850 Herausgeber / Edited by: Der Dekan: Prof. Dr. Zöbel Die Professoren des Fachbereichs: Prof. Dr. Bátori, Prof. Dr. Beckert, Prof. Dr. Burkhardt, Prof. Dr. Diller, Prof. Dr. Ebert, Prof. Dr. Furbach, Prof. Dr. Grimm, Prof. Dr. Hampe, Prof. Dr. Harbusch, Prof. Dr. Sure, Prof. Dr. Lämmel, Prof. Dr. Lautenbach, Prof. Dr. Müller, Prof. Dr. Oppermann, Prof. Dr. Paulus, Prof. Dr. Priese, Prof. Dr. Rosendahl, Prof. Dr. Schubert, Prof. Dr. Staab, Prof. Dr. Steigner, Prof. Dr. Troitzsch, Prof. Dr. von Kortzfleisch, Prof. Dr. Walsh, Prof. Dr. Wimmer, Prof. Dr. Zöbel Kontaktdaten der Verfasser Carsten Saathoff, Ansgar Scherp Institut WeST – Web Science and Technologies Fachbereich Informatik Universität Koblenz-Landau Universitätsstraße 1 D-56070 Koblenz EMail: [email protected], [email protected] Unlocking the Semantics of Multimedia Presentations in the Web with the Multimedia Metadata Ontology, Fachbereich Informatik Nr. 19/2009 Unlocking the Semantics of Multimedia Presentations in the Web with the Multimedia Metadata Ontology Carsten Saathoff Ansgar Scherp WeST, University of Koblenz-Landau WeST, University of Koblenz-Landau http://west.uni-koblenz.de/ http://west.uni-koblenz.de/ [email protected] [email protected] ABSTRACT like audio and video and one discrete media asset such as The semantics of rich multimedia presentations in the web text and image [32]. The media assets are arranged in time such as SMIL, SVG and Flash cannot or only to a very and space into a coherent multimedia presentation such as limited extend be understood by search engines today. This SMIL [35], SVG [36], and Flash [1]. Annotation of such rich, hampers the retrieval of such presentations and makes their structured multimedia content is the association of metadata archival and management a difficult task. Existing metadata to the structured content and its media assets [24]. models and metadata standards are either conceptually too In the web of today, rich, structured multimedia presen- narrow, focus on a specific media type only, cannot be used tations constitute a \black box". They cannot or only to a and combined together, or are not practically applicable for limited extend be understood by search engines. Multimedia the semantic description of rich multimedia presentations. formats such as the W3C standards SMIL and SVG forsee In this paper, we propose the Multimedia Metadata On- the use of Semantic Web technologies for annotating the con- tology (M3O) for annotating rich, structured multimedia tent using the Resource Description Framework (RDF) [34]. presentations. The M3O provides a generic modeling frame- However, there is currently no appropriate model provided work for representing sophisticated multimedia metadata. It or best practice available that explains how to describe and allows for integrating the features provided by the existing annotate such rich, structured multimedia content in the metadata models and metadata standards. Our approach web. The existing metadata models such as [17, 3, 23, 15, bases on Semantic Web technologies and can be easily inte- 16] and metadata standards like [21, 2, 18, 25] are either grated with multimedia formats such as the W3C standards conceptually too narrow, semantically ambiguous, focus on a SMIL and SVG. With the M3O, we unlock the semantics of specific media type only, cannot be used and combined with rich multimedia presentations in the web by making the se- each other, or are not practically applicable for the seman- mantics machine-readable and machine-understandable. The tic description of rich multimedia presentations in the web. M3O is used with our SemanticMM4U framework for the For example, image descriptions using EXIF [21] cannot be multi-channel generation of semantically-rich multimedia pre- combined with MPEG-7 [25] descriptors. In IPTC [18], the sentations. location fields are defined to contain the locations the con- tent is \focusing on". However, it remains unclear what \fo- cusing on" actually means. For instance, consider an image Categories and Subject Descriptors from the atomic bombing of the city of Nagasaki1 in Japan in E.4 [Data]: Coding and Information Theory; H.1.m [Infor- 1945. This image is about the city of Nagasaki since it docu- mation Systems]: Miscellaneous ments an event taking place in that city. But it is also about the world as a whole since the atomic bombing of the city of Nagasaki is of global importance. Distinguishing these General Terms different roles a location can play is impossible with IPTC Design, Languages, Management and others. Here, support for semantic annotations with for- mally defined background knowledge needs to be provided. Keywords However, this is hardly found in the existing models. In ad- dition, none of the existing models and standards explicitly rich multimedia presentations, multimedia metadata, seman- support the distinction between information objects and in- tic annotation formation realizations [8]. An information object such as an image is often available in different formats and resolutions, 1. INTRODUCTION i.e., in different information realizations. However, this fea- Multimedia metadata and semantic annotation of multi- ture is often requested by conceptual multimedia metadata media is a key-enabler for improved services on multimedia models [20, 15]. In addition, most metadata models focus content. If there is no or only limited metadata and anno- on a single media type only. Thus, they ignore the type's tations provided, the archival, retrieval, and management relation to other media types and the media assets' context of multimedia content becomes very hard if not practicably within a rich, structured multimedia presentation. These infeasible. Rich, structured multimedia content is encoded metadata models are not designed to be combined with each by the combination of at least one continuous media asset 1http://commons.wikimedia.org/wiki/File: Nagasakibomb.jpg, from Wikimedia Commons. The . image is in the public domain. 3 Unlocking the Semantics of Multimedia Presentations in the Web with the Multimedia Metadata Ontology, Fachbereich Informatik Nr. 19/2009 other. This results in a disconnectedness of today's models. of the existing models, the M3O aims at integrating and However, this combination is required by multimedia appli- representing the metadata and data structures that underlie cations that integrate, e.g., image data and video data, in the existing approaches. This is achieved by the formal na- particular in the open world of the web. In addition, most ture of the M3O and by following a pattern-oriented design existing models do not support representing both high-level approach. The problem of annotating rich, structured mul- semantic annotation with background knowledge as well as timedia presentations such as SMIL, SVG, and Flash is not the annotation with low-level features extracted from the new. However, until today it remains an unsolved problem multimedia content. and there is no appropriate model or best practice available This situation is very unfortunate as the authoring of rich, that can be used to describe and annotate structured con- structured multimedia content can be quite expensive and tent in the web. With the M3O, we aim at providing a model providing support for annotating rich content not only im- for annotating structured multimedia content and to unlock proves archival, retrieval, and management of the content, its semantics for a better archiving, retrieval, and manage- but also allows for better reuse. With the Multimedia Meta- ment of the content. The M3O continues our prior work on data Ontology (M3O), we propose an approach for anno- multimedia annotation and bases on the experiences gained tating rich, structured multimedia content in the web and with developing the Core Ontology on Multimedia [3]. unlocking its semantics by making it machine-readable and The remainder of the paper is organized as follows: In the machine-understandable. The M3O allows for a sophisti- next section, we motivate the need for the M3O by a con- cated semantic description of rich, structured multimedia crete scenario. The requirements to the M3O are presented content. It provides a generic modeling framework that in Section 3 and related work is reviewed in Section 4. The can accomodate and integrate the features provided by the ontology design patterns of the M3O are introduced in Sec- different multimedia metadata models and metadata stan- tion 5. In Section 6, we discuss the scenario from Section 2 dards we find today. The M3O bases on Semantic