Advene as a Tailorable Hypervideo Authoring Tool: a Case Study Olivier Aubert Yannick Prié Daniel Schmitt Université de Lyon, CNRS Université de Lyon, CNRS Université de Strasbourg, Université Lyon 1, LIRIS, Université Lyon 1, LIRIS, LISEC EA-2310, France UMR5205, France UMR5205, France [email protected] [email protected] [email protected] ABSTRACT Many video annotation tools are often strongly tied to a Audiovisual documents provide a great primary material specific application domain. This allows them to offer a cus- for analysis in multiple domains, such as sociology or in- tomized experience, with dedicated tools and interfaces, at teraction studies. Video annotation tools offer new ways of the expense of some difficulty to be used outside of intended analysing these documents, beyond the conventional tran- uses. For instance, a tool like ELAN [8] does not allow over- scription. However, these tools are often dedicated to spe- lapping annotations, which may make sense in the linguis- cific domains, putting constraints on the data model or in- tic context but may not be convenient for other practices. terfaces that may not be convenient for alternative uses. Moreover, most specialized tools being aimed at analysis, Moreover, most tools serve as exploratory and analysis in- they do not particularly insist on the final phase of the pro- struments only, not proposing export formats suitable for cess: publishing documents. Tools like Anvil [4] or Elan [8] publication. propose multiple predefined structured export formats but We describe in this paper a usage of the Advene software, do not allow users to define their own visualizations. On the a versatile video annotation tool that can be tailored for var- other hand, rendering environments like [5] do not focus on ious kinds of analyses: users can define their own analysis the analysis side. structure and visualizations, and share their analyses either The Advene project and its main application Advene [2] as structured annotations with visualization templates, or are trying to reach a larger usability spectrum by offering published on the Web as hypervideo documents. We explain a number of generic tools and interfaces, that can be user- how users can customize the software through the definition customized to fit the application domain. Users can also de- of their own data structures and visualizations. We illus- fine their own visualizations, in order to accommodate their trate this adaptability through an actual usage for interview specific needs and possibly to generate publishable docu- analysis. ments. We claim that it is necessary that tools adapt to some extent to the user's needs, especially in such an ex- ploratory activity. Furthermore, this flexibility must always Categories and Subject Descriptors be effective, in order to accompany the continous evolution H.5.4 [Hypertext/Hypermedia]: User Issues,Navigation; of the user's ideas and needs. H.5.1 [Multimedia Information Systems]: Video In this article, we first present how Advene has been con- ceived to provide generic but customizable tools in order to Keywords accommodate a variety of usages. We briefly highlight the main features that provide this flexibility. Then we illus- Hypervideo, Annotation, Video Analysis, Mediafragments, trate these capabilities by detailing a real use example and Active Reading how Advene features were used to adapt to this particular situation. 1. INTRODUCTION As described in [1], video analysis is often carried out 2. ADVENE PRINCIPLES through active reading, an iterative activity based on the creation of structured metadata and the usage of various vi- The Advene software covers the whole range of active sualizations for this metadata, in an intermediary stage for reading activities [1]: 1/ it allows to create annotations, video navigation or as a final publishing stage. Documents either through dedicated interfaces, by importing data or produced by this activity can take the form of hypervideos. through some feature extraction algorithms. Annotations can be structured through user-defined schemas. They also can be linked through n-ary relations, also user-defined, in order to express any kind of relationship. 2/ Once annota- tions are present, they can be used as a reference to navigate the audiovisual document, in order to investigate the mate- ACM, 2012. This is the authors version of the work. It is posted here rial and possibly to update or extend the annotations. 3/ by permission of ACM for your personal use. Not for redistribution. The Annotations and possibly pieces from the video document definitive version was published in Proceedings of the 12th ACM sympo- can be used to generate various visualizations, as custom in- sium on Document engineering, 2012 DocEng’12, September 4–7, 2012, Paris, France. terfaces or as documents that may be published. The appli- Copyright 2012 ACM 978-1-4503-1116-8/12/09 ...$10.00. cation offers a number of customizable interfaces for various actions (annotation edition, visualization or search). It inte- one annotation to another one), or trigger different actions grates a template-based document generation system, allow- such as using text-to-speech or a Braille table [3] to render ing users to specify various renderings for their data. Even- annotation data while the video is playing. Users can define tually, the application can be extended by developers either dynamic views through rulesets, comparable to the filtering through the use of plugins, for dedicated developments, or rules that can be found in e-mail client software. by contributions, Advene being free software. Static views are visualizations specified by XML-based The principles and architecture of Advene have been de- templates, combining information from the annotation struc- scribed with more details in previous articles [2]. We will ture and from the video. Using an XML-compatible tem- focus here briefly on the main aspects of Advene that allow plate [9], users can define any kind of visualization accord- more specifically its flexibility. ing to their needs. The use of a template language is in- tended to lower the barrier to conceiving such visualizations. 2.1 Data model The Advene application embeds a webserver that dynami- As described in [2], the Advene/Cinelab1 data model fea- cally serves the rendered templates to any web browser, or tures explicitly structured annotation data, as well as the through an embedded HTML component. Links into the definition of queries and visualization templates. In our au- video control the Advene video player, offering the ability diovisual context, we define an annotation as a piece of data to directly play fragments related to annotations. This gives of any kind associated with a video fragment, i.e. with two users the possibility to modify the underlying metadata (an- timecodes designating beginning and ending instants in a notations) and get instant updates of the generated docu- given video. ments. 2.2.1 Hypervideo publishing However, this approach implies that the Advene software has to be used to access visualizations and to fully experience their hypervideo nature by synchronizing with the Advene video player. In order to provide a publishing functionality, Figure 1: Global representation of the Ad- we integrated the possibility to statically export a set of vene/Cinelab data model defined visualizations. The web export feature is basically a specialized version The annotation structure, summarized in figure 1, consists of a website copier, that walks through the defined visual- in user-defined annotation types and relation types that can izations (which are most often HTML documents with some themselves be grouped into schemas that represent a given specific links to control the Advene video player), captures point of view on the analysis. Views are used to transform the HTML content and injects javascript code to embed an annotations, along with the video, into visualizations appro- HTML video player. Links to the Advene video player are priate for the user's task. converted into corresponding Media Fragment [7] URLs, to The model has been defined to be as expressive and flex- be able to address the embedded HTML video player. ible as possible. The definition of annotation types gives a Some features can be activated in the exported version, way to categorize annotations, which is useful both for the through the definition of HTML classes in the template. user and for the computer. Relations, classified by relation For instance, specifying the transcriptHighlight class will types, may be used to express any graph structure, or more highlight elements of the class transcript synchronously basically to link annotations according to the user's needs. with the video player. The annotation structure (annotation types and relation We have seen that in addition to the annotation creation types) is defined by the user to fit his or her needs. It can and navigation tools, user-definable visualizations allow Ad- be defined iteratively, and interactively, in order to foster vene to cover the whole annotation lifecycle, from annotation experimentation with various analysis structures. New an- creation to publishing. Besides, user-defined data structure notation types can be created on the fly, and annotations and views offer a high level of tailorability. Their defini- copied into this new type, seamlessly. tions represent the specialization of the tool, and they can 2.2 Visualization - hypervideo be themselves shared and reused. We will now present how these features have been used through a real use case. Advene provides three main types of visualizations: ad- hoc views, dynamic views and static views. Ad-hoc views are graphical components present in the soft- 3. USE CASE ware, such as a timeline or a tabular view. Such components provide great interactivity, and can be customized to some 3.1 Context extent by users. For instance, users can define which anno- Advene is used by Daniel Schmitt for his analysis of the tation types are displayed in a timeline, and can save this visitors perception of museum exhibitions2.