Journal of Ambient Intelligence and Smart Environments 0 (2010) 1 1 IOS Press A semantic web annotation tool for a web-based audio sequencer Luca Restagno a, Vincent Akkermans b, Giuseppe Rizzo a and Antonio Servetti a a Dipartimento di Automatica e Informatica (DAUIN), Politecnico di Torino, Corso Duca degli Abruzzi, 24, 10129 Torino, Italy E-mail: [email protected], {giuseppe.rizzo|antonio.servetti}@polito.it b Music Technology Group, Universitat Pompeu Fabra, Barcelona, Spain E-mail: [email protected] Abstract. In this work we describe an ontology driven resource annotation tool based on semantic web technology and web standards. It can load any OWL ontology and guide the end user in the annotation process with a simple graphical user interface. Furthermore, a collaborative web-based audio sequencer was developed as a test case for the annotation tool. With this content authoring tool, users can remix materials from the Freesound website and share their creations with others. Because of the integration of these two components, users are able to annotate the sonic materials used, the intrinsic structure of the artistic work, and the reasoning behind their actions during the creative process. We believe this approach will provide several novel ways of making not only the final product, but the creative or production process first class citizens of the semantic web. Keywords: ontology driven annotation tool, semantic audio annotation, web-based audio sequencer, semantic web sequencer, semantic web 1. Introduction annotation, which is ready to be stored to a server so that the annotations are available to everyone. A key focus of the Semantic Web [2] is the pro- Furthermore the annotation tool has been com- cess of combining semantic information from several pletely developed using web standard languages, in sources in order to more easily understand and explore such a way it can be easily used inside other web appli- that information programmatically. Currently, a large cations. Moreover it provides a userfriendly web front- number of web communities allow their members to end to make the semantic annotation, driven by an on- tag and annotate the content and thereby crowdsource tology, an easy task. In order to accomplish this goal the content classification. the tool has been developed to load all the classes from The ontology driven annotation tool detailed in this the ontology and their attributes; this way the classes work was developed to allow users to annotate any are easily browsable through the tools graphical user kind of resource,but using conceptsfrom a specific do- interface. For each attribute of the class, the tool loads main, as opposed to annotating with textual tags. The the right user interface widget, that lets the user to main goal of the tool is the capability to load from specify a value. the web any ontology developed with the Web Ontol- The annotation tool has been extended to provide a ogy Language (OWL) [8]. The annotation tool permits specific user interface for the annotation of audio con- tent. For this purpose, it allows the user to load and to select an ontology from the web, to choose the re- annotate sounds from the Freesound1 website. source to annotate and to create annotations linking concepts from the ontology to the resource. In addi- tion, it creates an RDF/XML document [1] for each 1http://www.freesound.org 1876-1364/10/$17.00 c 2010 – IOS Press and the authors. All rights reserved 2 L. Restagno et al. / A semantic web annotation tool for a web-based audio sequencer The practice of collaboratively tagging media con- The remainder of this paper is organized as follows: tent, known as folksonomy, has become widespread a review of the current state of the art is presented in on the Internet. Users annotate and categorize content Section 2, key ideas of our approach are introduced in by providing tags describing any and all possible as- Section 3 and the description of the annotation Tool is pects. One of the challenges in the area of annotation showed in Section 4. A contextualization of the tool, is generating consistency across annotations in terms by means of a use case is described in Section 5 fol- of both the vocabulary used and the way it is used. A lowed by conclusions and future works in Section 6. possible solution is to let users annotate only with a certain set of concepts defined in an ontology. An ob- vious disadvantage is the incompleteness of this ap- 2. Related Work proach. However, with several well designed ontolo- gies knowledegeable users can be guided to deliver The Internet has become a large repository of re- higher quality annotations. Developing a front-end ap- sources of any type. As the amount of digital content plication, we built a dynamic environment which per- grows it becomes increasingly important to improve mits to reuse the knowledge contained into ontologies, methods for description and retrieval. An often used according to the Linked Data principle [9]. approach is annotating information with metadata in To asses our annotationtool we developeda web-based order to more easily retrieve the information later. In audio sequencer with which users can mix sounds particular, annotation is often required to refine and available on the web by simply using their URLs. The improve data descriptions also when automatic feature two applications have been integrated, realizing a com- extraction tools are employed. plete tool for composing and annotating sounds. In this context we present an open source web tool that Music and sound generally have a rich semantic can augment the user experience in the process of se- structure. They communicate a message, designed by mantic annotation because it allows to easily perform the composer or sound designer, which ranges in its in- an ontology driven annotation during the creative pro- tention from nonsensical or abstract to symbolic (e.g. cess of audio authoring by means of a web sequencer. a piece of film music supporting a clear narrative). A Previous works have already addressed the problem, majority of media production nowadays is done with but with different approaches. software tools, which give rise to various new opportu- In the LabelMe Project [12], Russell et al. pro- nities to monitor the production process. In this work duced a web-based image tool useful to identify ob- we focused on the combination on the annotation tool jects. Their goal was to provide a dynamic dataset that and the audio sequencer, both described earlier, to in- would lead to new researchin the areasof object recog- vestigate the implications of this idea. Take for exam- nition and computer graphics. Although this research ple video games, which generally have a non linear was looking into tagging analysis, they did not fo- narrative that is often supported by affective music. As cus on annotation itself. Additionally, they used free the game world can give rise to a variety situations the form textual tags, so the annotations were affected by music should be able to adapt. If the composer is able the typical “folksonomy” problems, like polysemy and to formally describe pieces of his non linear work, and synonymy. In our approach we try to solve these prob- the game designer is as well, the game would be able lems using a controlled vocabulary, an ontology which to generate new music with the musical material pro- includes hierarchy of concepts from a specific knowl- vided by matching the formalized intentions in both edge domain. domains. Another example is music education, where Indeed, in the M-OntoMat-Annotizer [10], Petridis providing insight and understanding into the music is et al. have explored the usage of controlled vocabu- the primary concern. A piece of music, whose struc- laries to improve semantic annotation of images. They ture and different aspects have been formally anno- implemented different ontologies based on the MPEG- tated, could be represented in different ways and so 7 standard, to let user to associate visual descriptors to give students views on the work of art that match their contents. They permitted to select a portion of an im- capabilities. Possibly with future work the artist can, age or a portion of a frame and to associate a concept while working, develop his own ontology. This ontol- retrieved from the furnished ontology. Based on this ogy would describe the themes of the work as seen by work, we take advantage of multiple formalized ontol- the creator and allow himself and others to reflect and ogy domains to extend the descriptive possibilities of learn from this. an annotator. L. Restagno et al. / A semantic web annotation tool for a web-based audio sequencer 3 In [13], Wang et al. faceted the annotation problem by means of a set of ontologies, which are linked us- ing a bridge ontology. This work overcame the prob- lem of ontology reuse and prevented unnecessary on- tology extension and integration. Although this idea is promising, it presents only a general idea of how to link ontologies, without proposing a clear method. However, G. Kobilarov et al., following the previ- ous idea, used a bridge or hub ontology for catego- rizing multimedia documents located within the BBC archives [6] and linking them to DBPedia [3]. By means of this approach, they exploited object persis- Fig. 1. The annotation tool acts as a knowledge aggregator on the tence in the BBC categorization system (CIS) and Web. It permits to link pieces of knowledge extracted from online they mapped the resources according to DBPedia ref- distributed repositories, to digital resources on the web, in order to describe their contents in a semantic way.