D2.1 State of the Art on Multimedia Search Engines

D2.1 State of the Art on Multimedia Search Engines

Chorus D2.1 D2.1 State of the Art on Multimedia Search Engines Deliverable Type * : : PU Nature of Deliverable ** : R Version : Released Created : 23-11-2007 Contributing Workpackages : WP2 Editor : Nozha Boujemaa Contributors/Author(s) : Nozha Boujemaa, Ramon Compano, Christoph Doch, Joost Geurts, Yiannis Kampatsiaris, Jussi Karlgren, Paul King, Joachim Koehler, Jean-Yves Le Moine, Robert Ortgies, Jean-Charles Point, Boris Rotenberg, Asa Rudstrom, Nicu Sebe. * Deliverable type: PU = Public, RE = Restricted to a group of the specified Consortium, PP = Restricted to other program participants (including Commission Services), CO= Confidential, only for members of the CHORUS Consortium (including the Commission Services) ** Nature of Deliverable : P= Prototype, R= Report, S= Specification, T= Tool, O = Other. Version : Preliminary, Draft 1, Draft 2,…, Released Abstract: Based on the information provided by European projects and national initiatives related to multimedia search as well as domains experts that participated in the CHORUS Think-thanks and workshops, this document reports on the state of the art related to multimedia content search from, a technical, and socio-economic perspective. The technical perspective includes an up to date view on content based indexing and retrieval technologies, multimedia search in the context of mobile devices and peer-to-peer networks, and an overview of current evaluation and benchmark inititiatives to measure the performance of multimedia search engines. From a socio-economic perspective we inventorize the impact and legal consequences of these technical advances and point out future directions of research. Keyword List: multimedia, search, content based indexing, benchmarking, mobility, peer to peer, use cases, socio-economic aspects, legal aspects The CHORUS Project Consortium groups the following Organizations: 1 Chorus D2.1 JCP-Consult JCP F Institut National de Recherche en Informatique et Automatique INRIA F Institut fûr Rundfunktechnik GmbH IRT GmbH D Swedish Institute of Computer Science AB SICS SE Joint Research Centre JRC B Universiteit van Amsterdam UVA NL Centre for Research and Technology - Hellas CERTH GR Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e. V. FHG/IAIS D Thomson R&D France THO F France Telecom FT F Circom Regional CR B Exalead S. A. Exalead F Fast Search & Transfer ASA FAST NO Philips Electronics Nederland B.V. PHILIPS NL 2 Chorus D2.1 Table of Contents TABLE OF CONTENTS ................................................................................................................................................. 3 1. INTRODUCTION................................................................................................................................................... 6 2. USER INTERACTION .......................................................................................................................................... 7 2.1. SUMMARY ........................................................................................................................................................ 7 2.2. USER INVOLVEMENT IN THE SYSTEM DEVELOPMENT PROCESS ......................................................................... 7 2.2.1. User centered design .................................................................................................................................. 8 2.2.2. The HCI perspective ................................................................................................................................... 8 2.3. USER GENERATED CONTENT............................................................................................................................. 9 2.4. USER CENTERED DESIGN OF MULTI -MEDIAL INFORMATION ACCESS SERVICES ................................................ 10 2.4.1. Beyond "relevance" as a target notion - How can we formalise our understanding of what users are up to? 10 2.4.2. Use cases as a vehicle............................................................................................................................... 11 2.5. USE CASES IN CURRENT RESEARCH PROJECTS ............................................................................................... 12 2.6. USE -CASES AND SERVICES ............................................................................................................................. 14 2.7. OVERVIEW : THE CHORUS ANALYSIS OF USE CASES .................................................................................... 14 2.8. PARTICIPATING PROJECTS .............................................................................................................................. 15 2.9. DIMENSIONS OF DATA ANALYSIS ................................................................................................................... 15 2.10. DATA ANALYSIS ............................................................................................................................................ 15 2.10.1. Actions.................................................................................................................................................. 15 2.10.2. Corpora................................................................................................................................................ 17 2.10.3. Methodologies (Technical Requirements)............................................................................................ 19 2.10.4. Products ............................................................................................................................................... 21 2.10.5. Users .................................................................................................................................................... 24 2.10.6. User Classes......................................................................................................................................... 26 2.11. CONCLUSION AND FUTURE PROSPECTS .......................................................................................................... 27 2.12. REFERENCES ................................................................................................................................................... 27 3. STATE OF THE ART IN AUDIO-VISUAL CONTENT INDEXING AND RETRIEVAL TECHNOLOGIES.......................................................................................................................................................... 29 3.1. INTRODUCTION ............................................................................................................................................... 29 3.2. RECENT WORK ............................................................................................................................................... 30 3.2.1. Learning and Semantics............................................................................................................................ 31 3.2.2. New Features & Similarity Measures....................................................................................................... 33 3.2.3. 3D Retrieval.............................................................................................................................................. 35 3.2.4. Browsing and Summarization................................................................................................................... 36 3.2.5. High Performance Indexing...................................................................................................................... 36 3.3. SUMMARY OF MULTIMEDIA ANALYSIS IN EUROPEAN RESEARCH .................................................................. 37 3.3.1. Multimedia Analysis in European Projects............................................................................................... 37 3.3.2. Multimedia Analysis in National Initiative ............................................................................................... 38 3.3.3. State-of-the Art in European Research ..................................................................................................... 39 3.4. FUTURE DIRECTIONS ...................................................................................................................................... 43 3.5. REFERENCES ................................................................................................................................................... 44 ANNEX A: OVERVIEW OF THE 9 IST PROJECTS ............................................................................................................. 51 MODULES ON SPEECH /A UDIO INDEXING AND RETRIEVAL ............................................................................................ 61 MODULES ON IMAGE INDEXING AND RETRIEVAL ......................................................................................................... 64 MODULES ON 3D INDEXING AND RETRIEVAL ............................................................................................................... 65 MODULES ON VIDEO INDEXING AND RETRIEVAL ......................................................................................................... 65 MODULES ON TEXT INDEXING AND RETRIEVAL ........................................................................................................... 72 ANNEX B: OVERVIEW OF THE NATIONAL RESEARCH PROJECTS ............................................................ 74 4. SOA OF

View Full Text

Details

  • File Type
    pdf
  • Upload Time
    -
  • Content Languages
    English
  • Upload User
    Anonymous/Not logged-in
  • File Pages
    219 Page
  • File Size
    -

Download

Channel Download Status
Express Download Enable

Copyright

We respect the copyrights and intellectual property rights of all users. All uploaded documents are either original works of the uploader or authorized works of the rightful owners.

  • Not to be reproduced or distributed without explicit permission.
  • Not used for commercial purposes outside of approved use cases.
  • Not used to infringe on the rights of the original creators.
  • If you believe any content infringes your copyright, please contact us immediately.

Support

For help with questions, suggestions, or problems, please contact us