Framework Programme 7
“Future Media Internet Coordination action”
Contract no.: 249065
Deliverable D2.3 Report on current research and business targets V3.0
ICT Project Number FP7- 249065 Acronym nextMEDIA Reference nextMEDIA D2.3_SYN_V0.1_20110310 Project URL www.fi-nextmedia.eu EU Project officer Mr. Georgios Kaiafas Code name D2.3 Status draft final Nature Prototype Report Specification Tool Other Distribution Type (*) PU PP RE CO Th. Zahariadis (Synelixis) Authors (Partner) P. Daras (CERTH), T. Piatrik (QMUL), O. Mayora (Create-net), F. Alvarez (UPM) Guillermo Cisneros Contact Person Email [email protected] Phone +34-91 336-7344 Fax +34-91 336-7350 The deliverable provides a study of the research covered by the running and Abstract newly started projects of the unit, the relevant state-of the art and the business (for dissemination) sectors they cover, in relation with the FIA activities. Keywords Version Date Version (0.1) Change Author (s) Revised Outline Listed above Contractual Date of February 2011 Delivery Quality assurance Federico Alvarez Tomas Piatrik reders result Ok Ok ______(*) Distribution Type / Security: PU – Public, PP - Restricted to other program participants (including the Commission Services), RE - Restricted to a group specified by the consortium (including the Commission Services), CO - Confidential, only for members of the consortium (including the Commission Services) (**) Version: See Document History This document is produced under the EC contract FP7 ICT-249065. This document is property of the nextMEDIA consortium and shall not be distributed or reproduced without the written approval of the nextMEDIA consortium
Deliberadle D2.3 “Report on current research and business targets V3.0”
Table of Contents
EXECUTIVE SUMMARY ...... 3 Acronyms ...... 4 1 Introduction ...... 5 1.1 What is new in D2.3 ...... 5 2 Current Research Challenges...... 7 2.1 Research Challenges in Media Delivery ...... 7 2.2 Research Challenges in 3D Media ...... 16 2.3 Search and retrieval of multimedia content ...... 22 2.4 Research Challenges in User Centric Media/Social Networks ...... 28 3 Market Assessment and Business Targets ...... 37 3.1 Delivery of multimedia content ...... 37 3.2 3D Media Internet Technologies ...... 42 3.3 Networked Search...... 49 3.4 Social Network Internet Services ...... 53 4 Results from the Concertation Meetings ...... 58 4.1 Spring 2010 Concertation Questionnaire for Research Challenges ...... 58 4.2 Fall 2010 Concertation Questionnaire for Standardisation ...... 60 5 Conclusions ...... 63 6 References ...... 65 7 ANNEX I: Spring 2010 Concertation Meeting ...... 74 7.1 Questionnaire ...... 74 7.2 Detailed Results ...... 78 8 ANNEX II: Fall 2010 Concertation Meeting ...... 79 8.1 Questionnaire ...... 79 8.1 Detailed Results ...... 81 8.2 IETF statistics ...... 82 9 ANNEX II: FISA Standardisation Support Group ...... 84 9.1 Existing pre-standardisation initiatives related to future internet ...... 84
nextMEDIA D2.3_FF_20110325 Page 2 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
EXECUTIVE SUMMARY The nextMEDIA Coordination Action aims to coordinate the efforts in Europe towards producing a clear situation of the Future Media Internet (FMI). Especially in the Networked Media area, the expected results are a) to inform the existing and evolving project and project clusters about the FIA related activities, and to increase the efficiency of the cross-clusters collaboration in FIA related issues. This deliverable is the 3rd outcome of the WP2 ―Coordination of the activities of the cluster towards the FIA‖. The current version of the deliverable D2.3 is an updated version of D2.1 and D2.2. It is still a draft version, as the final will be delivered after the Networked Media spring 2011 concertation meeting on 14-15 April 2011. As compared to the previous versions (2.1 and D2.2), this deliverable includes the results of the questionnaires and additional information about running projects. In order to collect information about the running and new projects, a questionnaire will be distributed on the 28th of November 2010, during the project presentation at the concertation plenary session.
Disclaimer
This document contains material, which is the copyright of certain nextMEDIA contractors, and may not be reproduced or copied without permission. All nextMEDIA consortium partners have agreed to the full publication of this document. The commercial use of any information contained in this document may require a license from the proprietor of that information. The nextMEDIA Consortium consists of the following companies:
No Participant name Participant Country Countr short name y 1 Universidad Politécnica de Madrid UPM Coordinator Spain 2 ATOS Origin ATOS Contractor Spain 3 Centre for Research and Technology Hellas CERTH Contractor Greece 4 CREATE-NET CREATE Contractor Italy 5 Queen Mary University of London QMUL Contractor UK 6 Synelixis Solutions Ltd Synelixis Contractor Greece
Table 1. Partners list The information in this document is provided ―as is‖ and no guarantee or warranty is given that the information is fit for any particular purpose. The user thereof uses the information at its sole risk and liability.
nextMEDIA D2.3_FF_20110325 Page 3 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Acronyms AQoS Adaptation QoS BOF Birds Of a Feather session (IETF) CAS Content Aware Storage CBIR Content-based image retrieval CBVIR Content-Based Visual Information Retrieval CDN Content Delivery Network DPI Deep Packet Inspection ETSI European Telecommunications Standards Institute FCN Future Content Networks FIA Future Internet Assembly FIArch Future Internet Architecture Group FISO Future Internet Service Offerings FMI Future Media Internet FMIA-TT Future Media Internet Architecture Think Tank FMN Future Media Networks cluster IETF Internet Engineering Task Force MANA Management and self-Aware Architectures MDC Multi Description Coding MVC Multi-View Video Coding MVV Multi-View Video Plus Depth NQoS Network QoS OQoS Objective QoS P2P Peer to peer PQoS Perceived QoS RFC Request for Comments QBIC Query By Image Content QoS Quality of Service SQoS Subjective QoS SVC Scalable Video Coding UGC User Generated Content VpD Video Plus Depth
nextMEDIA D2.3_FF_20110325 Page 4 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
1 Introduction The nextMEDIA Coordination Action aims to coordinate the efforts in Europe towards producing a clear situation of the Future Media Internet (FMI), create common pillars (e.g. the Future Media Internet reference architecture for Europe) and support the results of the developments of the European projects. In order to achieve these goals, nextMEDIA realises the following actions: Coordinate the research collaboration in networked media scientific fields, especially in Future Media Internet (with strong orientation to the combination of 3D Internet Media and Content-aware Networks/Network-aware applications) Support the cooperation and future research of the European projects in the field of Networked Media by means of driving Task Forces and propose and coordinate new ones (―Future Media Internet Architecture‖). Identify the position and clear opportunities in the areas where the European research in Future Media Internet is more competitive world-wide and develop studies and roadmaps by means of analysing the situation, technology and patents in the most developed countries in relevant research areas. Create the necessary dissemination means to advertise and spread excellence of the European research in Future Media Internet world-wide and attract the prominent scientists and company managers to the events. With respect to the abovementioned activities, the expected results are: To inform the existing and evolving clusters about the FIA related activities, in order to achieve their role in sharing the gained knowledge between the participants. To increase the efficiency of the cross-clusters collaboration in FIA related issues, via presentations, emails and off-line discussion, in the timeframe between the Concertation meetings. Increase the collaboration of the EU-funded projects with similar or complementary National and International initiatives in areas related to FIA.
1.1 What is new in D2.3 This deliverable is the 3rd outcome of WP2 ―Coordination of the activities of the cluster towards the FIA‖. It is based on D2.1 and D2.2 yet its content has been updated in various sections. First of all the structure has been changed and all references to relevant nation and EU projects have been removed. Interested audience may refer to [123]. New sections related to social networks and business targets have been added. Moreover, a new chapter has been added analysing the results of the Networked Media Spring 2010 and Fall 2010 concertation meetings. As the Fall 2010 concertation meeting was focused on the standardization efforts, a detailed analysis is provided. The results of the Spring 2011 (which will take place on 14-15 April 2011), concertation meeting will be added in a new version of the deliverable. This deliverable is organized as follows. Chapter 2 provides for each cluster, the relevant state of the art and the research challenges. Chapter 3 provides the business nextMEDIA D2.3_FF_20110325 Page 5 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” orientations and the targeted market sectors. Chapter 4 provides the results of the concertation meetings questionnaires and analysis of Europe‘s contribution in IETF. Annex I and II provide the questionnaires, the results and a relevant analysis.
nextMEDIA D2.3_FF_20110325 Page 6 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” 2 Current Research Challenges Internet is today the most important information exchange mean. It is providing to the society the mechanisms to create new forms of social, political and economical intercourse. With the evolving role(s) of digital communication, a cognitive society goes beyond information and content accumulation and consumerism by involving conscious intellectual activity (as thinking, learning, reasoning, or remembering). For this purpose, the Internet should support mechanisms for knowledge dissemination both at local and global level. Knowledge and culture must be diffused worldwide to breakdown barriers and to promote dissemination and learning. In practice, this means that as digitalization of data progresses, it is expected that the majority of new media will arrive in digital form, with the analogue form being the exception. For instance, digital videos will not only increase in number, but also in size, due to increases in resolution and the ease of creation and manipulation. Progress in network multimedia communication is also leading to 3D videos. Several means accelerate sharing of these new forms of digital media continue to appear (YouTube for asynchronous video content being one of the well known pioneers in this space). Digital TV channels are also progressively penetrating the Internet space, Zattoo and Joost, for real time/streaming video content being pioneers in this space. As such the Internet plays a crucial role in the ability of humans to communicate, but at the same time opens new challenging problems. As the current Internet grows beyond its original expectations (a result of increasing demand for performance, availability, security, and reliability) and beyond its original design objectives, it progressively reaches a set of fundamental technological limits and is impacted by operational limitations imposed by its architecture. In this chapter, we aim to collect and categorize the research challenges based on the different Networked Media Unit clusters. In each cluster, we provide a short description of the research in EC projects, then we provide a current state of the art section and we conclude with the near and forthcoming research challenges.
2.1 Research Challenges in Media Delivery The term ―Networked Media‖ implies that all kinds of media including text, image, 3D graphics, audio and video are produced, distributed, shared, managed and consumed on- line through various networks, like the Internet, Fiber, WiFi, WiMAX, GPRS, 3G and so on, in a convergent manner. In the above context, Media Delivery Platform (MDP) cluster aims to cover the Networked challenges of the Networked Media in the transition to the Future of the Internet. Projects that fall in this cluster are more networked oriented, that content creation oriented. The main drivers in MDP are the use of the Internet as a common infrastructure for interconnecting more than computing machines. While the focus of today's Internet is mainly on elastic traffic (e.g. mainly Web/HTTP applications), bulk data transfers (e.g. peer-to-peer applications) and multimedia streaming (e.g. Internet TV, gaming), it is also expected that new applications will demand for new capabilities from the networks (Figure 1). For instance, strict real-time communication, and reliable connectivity with no information loss during failure scenarios are examples of high interest especially for industrial applications. These place new demands on the network and transport layers requiring new approaches and designs.
nextMEDIA D2.3_FF_20110325 Page 7 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Figure 1: Future Internet Traffic Properties [1] However, as encoding, adaptation and streaming are associated, this cluster covers research challenges not only in the area of “networking”, but also content encoding and adaptation. It should also be mentioned that MDP was renamed during the 5th FP7 Spring concertation meeting to Future Media Networks (FMN) to better represent the actual activities of the cluster.
References to relevant nation and EU projects in this topic may be found in [123].
2.1.1 Current State of the Art Trying to abstract and group the research areas that the MDP/FMN projects aim to advance, we may come up to three main subjects: the area of content-centric networks, the area of multimedia encoding and the area of video adaptation. The state of the art in these areas is summarised in the following sub-sections.
2.1.1.1 P2P and Content-Centric Networks The technology in the field of distributed overlay streaming is evolving at a very rapid pace. The potential advantages in terms of aggregated bandwidth, robustness and scalability have spurred a significant body of R&D activities. The first generation of P2P and overlay streaming systems includes several protocols that, in different amounts, are directly derived from P2P file sharing protocols e.g. Narada builds a mesh overlay topology that connects peers based on round-trip-time estimates; SplitStream improves bandwidth usage employing multiple trees, allowing leaf nodes to contribute their upstream bandwidth; a similar approach is taken by VidTorrent; Coolstreaming is also largely based on file sharing protocols, and implements an algorithm that chooses to download first the rarest chunks; finally popular large-scale systems such as SopCast and PPLive are based on mesh overlays. Some projects like SEA and P2PNext have introduced new multi-tree and mesh P2P protocols respectively.
nextMEDIA D2.3_FF_20110325 Page 8 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
The last couple of years however, many research efforts and activities have already targeted the approach of moving intelligence into the network. One of the rather short- term initiatives is the ETSI TISPAN NGN (Next Generation Network). The NGN approach includes a session control core that is based on the IMS and the SIP that allows a horizontal integration of services. However, NGN efforts have been mainly focused on the networking aspects needed to provide telephony, high-speed Internet access and television and does not consider requirements related to content in a more general way. Moreover, in NGNs, the roles of the network operators, content providers, and end users are considered to be fairly static and not to leave enough room for tussle. Moving towards modern content-aware networking activities, we can highlight the following approaches: Active Networks can be viewed as an effort to augment data packets with code fragments containing specialised processing logic for handling the packet itself. While active networks are a very powerful paradigm, there have been several drawbacks that have hindered their acceptance and wide spread deployment. The primary concern is the security against active routers running malicious code hidden in active packets. An equally important issue is that network operators prefer to have complete control over their network. In DONA (Data-Oriented Network Architecture) users can request named data from the network by using the FIND primitive, while content providers can publish a data object, which will be served to the users by using the REGISTER primitive. To support these two primitives, DONA introduces resolution handlers, which forward content to the users in an overlay manner. Similar to DONA, Siena (Scalable Internet Event Notification Architectures) [1] features a generic scalable publish/subscribe event- notification service. Siena formulates a general model of content-based addressing and routing to maximize both expressiveness and scalability. PARC [5] launches a research program named Assurable Global Networks (AGNs), where they focus on the point-to- multiparty or multiparty-to-multiparty information dissemination rather than traditional point-to-point conversations. The main feature of content-centric AGNs is that the security will reside in the data itself, not in the network channel as in today‘s Internet. The network only concerns how to distribute the data and the publishers control the security of the data. As a consequence, the content-centric network will be a huge storage of authenticated data. OpenCDN [6]constructs an application level tree via relay nodes that distribute the multimedia content. To coordinate relay nodes, OpenCDN collects client-related information from relay nodes and decides the best relay node for a newly joining client. Oscar [7]collects sampling information of key distribution during the P2P overlay construction and uses that information to choose routes based on small- world graphs. COCONET [8] aims to utilize semantic data tagging to provide content level information for data streams flowing through a network. Last but not least, Akamai's EdgePlatform [9] is a network of more than 40,000 secure servers with proprietary software, aiming to optimize routes and replicate data dynamically to deliver content and applications more quickly, reliably, and securely. Akamai‘s approach is to eliminate long routes, by replicating and delivering content and applications from servers close to end users.
2.1.1.2 Video Encoding In recent years, there have been increasing developments in technologies for transmission of multimedia content over Internet. When video is delivered to the user it usually needs to traverse network paths with very different traffic capacities: from very high bandwidth on dedicated glass fibre connections to very low bit-rate connectivity for wireless transmissions. Furthermore, the same content needs to be accessible from a nextMEDIA D2.3_FF_20110325 Page 9 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” variety of devices at the user side, which have different displaying and computational capabilities [10]. To tackle this challenge, the compression technology used in the transmission system should ideally provide two important features; first, it has to be highly efficient in terms of compression and second, it has to provide flexible, low- complexity, real-time content adaptation to the network and user‘s device properties. Scalable Video Coding (SVC) [11][12][10] is a relatively recent approach to video coding, which enables encoding of video content to produce a single scalable bit-stream that can be seamlessly adapted to network or terminal properties while providing high compression efficiency. A bit-stream is scalable if it is composed of a hierarchically embedded family of content instances at different resolution levels. The underlying multiresolution family of content instances should be embedded in the sense that when looking at two different instances of the same content at two different resolution levels, the two representations must match exactly over the extent of the lowest one. Any instance of the content embraces all representations for lower resolutions. The representation gets more accurate as the resolution increases. This allows very low complexity of adaptation as lower resolutions can be extracted from a higher resolution directly in the compressed domain, thus without performing computationally expensive transcoding.
SVC: Three Scalability Dimensions
Spatial Scaling Spatial
Combined spatial and temporal scalingspatial temporal and Combined
Scaling Te
mporal mporal
Figure 2: Scalability dimensions in SVC
During the last two decades a significant amount of research has been dedicated to SVC with the aim of developing the technology that would provide a low-complexity video adaptation, but retain the comparable compression efficiency and decoding complexity to those of conventional (non-scalable) video coding systems. This research evolved from two main branches of conventional video coding: 3D wavelet and hybrid video coding techniques. Although a hybrid based technology was chosen for standardisation within MPEG, a great amount of research continued also on Wavelet-based Scalable Video Coding (W-SVC). Several recent W-SVC systems have shown a very good performance in different types of application scenarios, especially when fine granular quality scalability is required. The approach presented in [11], performs a joint optimization of a wavelet-based SVC (W-SVC) and a Forward Error Correction method (FEC) based on Turbo Codes (TC) to provide a smooth delivery of video over Internet. The JSCC scheme [13][14] minimizes the reconstructed video distortion at the decoder subject to a constraint on the overall transmission bit-rate budget. The minimization is achieved by exploiting the source Rate- Distortion (RD) characteristics and the statistics nextMEDIA D2.3_FF_20110325 Page 10 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” of the available codes. Here, the critical problem of estimating the Bit Error Rate (BER) probability in error prone applications is also discussed. Regarding the error rate statistics, not only the channel coding rate, but also the interleaver and packet size for TCs are considered in the proposed approach. The aim is to improve the overall performance of the underlying JSCC. In [15], a perceptually adaptive loop filtering method is proposed for wavelet based SVC. Specifically, a new structure of in-loop filtering is designed for SVC considering update step in MCTF. In [16], a novel method for bitstream allocation is introduced. This method assumes that minimum R–D slope of the same fractional bit-plane within the same bit-plane across different subbands is higher than or equal to the maximum R–D slope of the next fractional bitplane across different subbands. An analysis of different down-sampling filters in popular wavelet- based scalable video coding schemes is presented in [17]. In addition to transmission over internet, SVC is apt for an event driven application that may be potential application for future internet. Apart from SVC, Multi-View Video Coding (MVC) [20] has recently attracted a lot of research. Compressing multi-view sequences independently is not efficient since the redundancy between the closer cameras is not exploited. MPEG and VCEG groups jointly created an ad-hoc group 3DAV [18], which received several contributions for Multi-View coding. A good review on the proposed algorithms can be found in [19]. As an output of this work, Multi-View Video Coding (MVC) is generated as an amendment to H.264/AVC, exploiting temporal and inter-view redundancy by interleaving camera views and coding in a hierarchical manner. The multi-view video codec based on H.264/AVC exploiting the correlation between cameras in a backward compatible way is proposed in [21]. Several prediction structures are proposed with the signalling in the bitstream. Codec is based on baseline profile and using only P pictures. It showed superior performance for dense cameras. First version of MVC extension of H.264/AVC can be used for some applications such as real time video communication. MVC is one of the first standards towards formal 3D encoding. In 2008, Nokia released a baseline version of MVC, which is very similar to the codec with the adapted syntax changes. Recently, Blu-ray Disc Association (BDA) announced the ―Blu-ray 3DTM‖ specification, which calls for encoding 3D video using the MVC codec to be supported by all Blu-ray Disc players. Usage of MVC will be more popular by upcoming new specifications and streaming and error resilient tools based on MVC will be required for transmission mediums. MPEG-C Part-3 [22] is the current standard to encode Video Plus Depth (VpD) data by individually compressing video and depth data using standard H.264/AVC [23]. Since depth data is much easier to encode rather than another view, compression efficiency of VpD is better than MVC coded stereoscopic videos, however image-based-rendered other view have artifacts due to occlusion. Another advantage of VpD representation is to enable changing the distance between the cameras while rendering other view. In order to handle 45-view displays, Multi-View Video Plus Depth (MVV) and VpD are not adequate. The solution to feed such systems is to decode a limited number of views using MVC and then generate artificial views using interpolation techniques [24]. There are two important reasons for this approach. The first practical reason is the problem of data acquisition. It is difficult to set up a mobile recording system that is composed of fifty high definition cameras that are calibrated. The second reason is the problem of data transmission. It is difficult to transmit even encoded data for fifty views over the Internet. MVC provides significant compression gain but the result is still linear with the number of views. Multi-view streaming systems use video+depth format
nextMEDIA D2.3_FF_20110325 Page 11 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” to drive displays with high number of views [25]. Future display systems will use M video signal and N depth maps. MPEG vision is a new 3D Video (3DV) format that goes beyond the capabilities of existing standards to enable both advanced stereoscopic display processing and improved support for auto-stereoscopic N-view displays, while enabling interoperable 3D services. This is an ongoing task between MPEG working groups.
2.1.1.3 QoS and Video Adaptation Video adaptation is the third area of research today, related to the MDP/FMN projects. Offering QoS-based services involves interactions, not only among a number of entities along the content/service delivery chain, but also across different layers. To coordinate effective adaptation and mapping of QoS parameters at service, application and network layers, cross-layer interactions are required. In order to measure and evaluate the QoS, a number of objective and subjective methods have been proposed. A subjective evaluation of the quality of the content is difficult due to various factors including time, cost and human perception. Strictly speaking, subjective tests require a large number of tests operated under controlled psychometric experimental conditions, to obtain statistically meaningful Mean Opinion Scores (MOS), summarizing the Perceived QoS (PQoS). Obviously this is not a good or even possible solution for real-time audiovisual services. As alternatives, objective measurements are used by analyzing the signals in both compressed (e.g. MPEG- 4/H.264 compressed video stream) and non-compressed (e.g. reconstructed RGB video as the output) formats. As illustrated in Figure 3, from the end users‘ perceptual experience to adaptation decision, a series of QoS are involved: Subjective QoS, Objective QoS, Network QoS (NQoS), and Adaptation QoS (AQoS). However, more types of QoS (and PQoS) may be defined in an end-to-end environment.
Figure 3: Subjective, Objective, Application, Network QoS In a layered architecture, such as today‘s typical TCP/IP model, each layer has a set of distinct mechanisms and associated parameters to fulfil its functionality. Table 2 classifies useful parameters and adaptation mechanisms by their respective layers. Parameters presented in this table include both tunable and read-only parameters.
nextMEDIA D2.3_FF_20110325 Page 12 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Layers Mechanisms (to optimize) Parameters Terminal characteristics, objective quality metrics User User priority selection (e.g. PSNR, VQM, SSIM), Subjective quality metrics (e.g. MOS, SAMVIQ, DSIS) Transrating, Transcoding, Forward Error Correction (FEC), Automatic Application Rate, Codec, Protection level Repeat Request (ARQ), Adaptive encoding/decoding
TCP Congestion Control, UDP, Header Packet loss information, receiver window, Transport Compression congestion window, retransmission timer
IP packet size, DiffServ Code Point, Handoff Network Packetization, DiffServ, TE information Retransmission attempts, Error rate, retry limit, MAC Protocols, Radio resource control, Data Link RTS/CTS, Handoff, Traffic classes, TDMA time FEC, ARQ, Framing slots, OFDM carriers BER, signal strength, transmission power, Physical Channel modulation and coding capability profile Context Dynamic Voltage Scaling, Scheduling Battery status, Architectural capability profile Information
Table 2. Mechanisms and parameters at different layers
During the last couple of years however, it has been shown that adaptation techniques limited to adaptation within a single layer are deficient in providing global optimal setting for the system. In contrast, cross-layer approach has been extensively discussed in recent research literature for its viability for providing better performance than traditional layered architecture. Although cross-layer design emerged as a by-product of recent proliferation of wireless networks having totally different properties from wired networks, it offers various opportunities for heterogeneous environment, where a variety of application types, network technologies and terminal capabilities are utilised. A comprehensive framework that deals with content delivery and adaptation issues is MPEG-21 [26]. All parts of MPEG-21 address a distinctive set of requirements, which allow implementers of the standard to design and implement a system or application that goes beyond simple multimedia content delivery in an interoperable way.
Though MPEG-21 describes a complete content delivery and adaptation framework, it has turned out that only relatively small portions of the whole MPEG-21 framework have been adopted by industry so far. This ultimately leads to the question whether MPEG has addressed the requirements in a vital way and what needs to be done to foster adoption of the MPEG-21 concepts on a broader scale. New approaches based on SDP signalling have been proposed [27][28] to provide the required adaptation in the network, without the need for the MPEG-32 signalling and communication overheads.
2.1.2 Research Challenges Table 3 shows the running and newly started projects against the research areas that are related to the FMI and are currently state of the art. Most running and newly started projects of the Future Media Networks cluster (as is the new name of the Media Delivery Platforms cluster) have in common that they use advanced networking technologies including P2P, content/information overlays or content centric networking technologies for optimal content adaptation, delivery and streaming.
nextMEDIA D2.3_FF_20110325 Page 13 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
t
Home Home
Content Content
P2P TV
protocol Caching
Protocol
Network
Location
Awareness
Adaptation
Networking
Architecture
New routing New
Environmen
Access /Core Access
New Network New
New Transport New
New Signalling New
P2P Topologies Content Centric ADAMANTIUM OPTIMIX P2P Next SEA NAPA-WINE iNEM4U ALICANTE COAST COMET CNG ENVISION
Table 3. Comparison of Media Delivery Platforms Cluster projects Looking into the future, research challenges that we may highlight are: Ever increasing Bandwidth. The Internet is growing and advancing in several dimensions: the number of users, the amount and size of the content, the new media and new services‘ traffic requirements. Thus, more bandwidth is needed in the end- to-end path (from the server side, through the core, metro, access and body networks, to the final user terminal) and new (e.g. P2P network-friendly) delivery methods are required. Content adaptation & personalisation. The highly heterogeneous environment in terms of diversity of user devices, (mobile) networks and user preferences will remain. To ensure a real seamless access to new immersive applications, it is desirable that the network itself and the services could automatically realise content adaptation and enrichment inside the network. Moreover, mainly from the business domain, substantial demand and significant challenge for the FMI will be required for adaptation of encrypted and trusted content. Content Centric networks. Content-aware real-time transmission of future media means that the relative importance of each packet towards increasing the end-to-end utility function is established [3]. That is, the more important packets should be better protected (by allocating appropriately network resources) or should be transmitted first in a scheduling scenario. Content/Information driven routing. Internet routing system shall be capable to consider associated routing information (e.g. meta-data) and metrics for path calculation such as the link quality, security level, energy consumption, priorities or location. Additionally, an efficient way of addressing groups of "users" by pointing at (groups of) terminals/devices is required in many applications that rely on a one- to-many data exchanges. New architectures and overlay networks for content distribution. The main research challenge related to new architecture is the (dynamic, autonomicity and self- organising) creation of overlay network infrastructures to support the provisioning of media services to end-user communities. Some of the issues related to overlay
nextMEDIA D2.3_FF_20110325 Page 14 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
networks have a wider impact and span, in fact, multiple areas. For instance the specification and measurement of QoS parameters and other metrics that can be used to assess the underlying communication technologies, achieve network friendliness (via local caches), suggest most suitable service instances for the end-user. A possible approach to deal with these cross-layer issues is to gather information from the underlying networks and combine it with the higher level quality assessment and requirements of applications to adjust the overlay networks. Quality of Experience. Quantification of QoE using objective and subjective measures, remains a challenging research problem. While measures to predict and evaluate the visual quality of uncompressed 3D video is itself an unsolved problem, developing such measures in the presence of packet losses and other compression artifacts is even more challenging and dependent on the specific display technology. Furthermore, the relationship between network level QoS measures and overall QoE must be studied. Identity, Trust, Privacy and Security. The content that is being produced and distributed is increasing rapidly. Users expect to be able to take advantage of the future widespread availability of multimedia content and access to virtual worlds. At the same time, they need to feel confident that their security and privacy is being protected. The increasing complexity and scale of future media systems will make the problems of Identity, Trust, Privacy and Security harder to solve. Content Encoding. Multi-layered Scalable Video Coding (SVC) offers temporal, spatial and quality scalability; Multi-view point Video Coding (MVC) allows for different views of video streaming without drastically increasing the data rate; Multiple Description Coding (MDC) offers an inherited resiliency mechanism with improved PQoS when different sub-streams are received from independent physical or logical paths. However, new media formats and encoding methods (also network coding methods) to offer high definition (HD) selectable free-viewpoint content coding and delivery, considering the evolution from H.264 2D SVC/MVC to scalable HD 3D, Multiview Video plus Depth (MVD) and selectable free -viewpoint video with interactive virtual panning/zooming. Moreover, new media formats that go beyond video and sound to even other senses e.g. feeling, touching, sensing. In-network content enrichment. Novel methods for in-network content enrichment and cross-network adaptation will be needed to allow for optimal use of available resources and enriched QoE. By dynamically combining the inherited content scalability (SVC different content layers, MVC different content views and MDC different content descriptions) of the same resource (video stream), transmitted from multiple sources (different servers or peers in case of P2P streaming) and/or received over multiple diverse paths or networks (utilise the MDC features), on-the fly content adaptation, inherited resiliency and enriched QoE may be achieved. Reconstruction of the content segments may take place either within the network or at the edge of the network (at content aware edge routers) offering transparent streaming to low- end terminals or at the terminal side in case multi-network connectivity is available. Cross-network adaptation and in-network content enrichment especially in P2P overlay topologies, will offer traffic adaptation (load balancing to avoid network flooding), optimal use of available resources (bandwidth), and enriched QoE.
nextMEDIA D2.3_FF_20110325 Page 15 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
2.2 Research Challenges in 3D Media After having been confined to stand alone applications for long time, the use of 3D visualization technologies in the representation of reality is now mature enough to merge with internet applications. This trend can already be observed in some popular applications such as Second Life or World of Warcraft and it is progressively changing the way in which people will experience the future Internet. Furthermore, recent technologies such as networks of embedded sensors and actuators immersed in the physical environment appear now mature enough to overcome another limit to the widespread acceptance of 3D Internet in the everyday life, which has been the lack of suitable, non invasive interfaces letting the user to interact with the virtual world as if she was immersed in the physical environment. For these reasons, although the use of 3D is currently limited to a few applications, it can be expected that most future internet applications, such as e-mail, web browsing, VoIP, virtual shops, file sharing, social networking etc… will rely on this technology. In this section FP6 - FP7 funded and national projects related to 3D Media are analysed. This is followed by the revision of the State of the Art in 3D Media.
References to relevant nation and EU projects in this topic may be found in [123].
2.2.1 Current State of the Art In this section, we provide the current state of the art in 3D Caching and coding and in interaction techniques and User Interfaces. MVV and VpD have already been mentioned and will not be repeated in this section.
2.2.1.1 3D Cashing and Coding The goal of 3D streaming is to deliver 3D content in real-time, so that immediate interactions with the virtual world are possible [29]. Although several factors all affect the streaming quality, the most important limitation has been the bandwidth. Model simplifications and progressive transmissions have thus been used to deal with the limited bandwidth [30]. Even though content streaming requires additional bandwidth, it becomes less of a problem with better broadband networks. Prefetching techniques [31] additional could reduce the potential delay users‘ experience. In fact, VE‘s based on 3D streaming have already appeared, where terabytes of content are navigable via progressive transmissions. In the long run, install-then-play may no longer be practical or feasible, when the content becomes truly massive and dynamic or when VE‘s become as numerous as the websites today. Current 3D streaming scheme may be classified into four types: object streaming, scene streaming, visualisation streaming and image – based streaming [32]. For scene streaming, as the users‘ field of view may not cover the entire VE, downloading the entire scene is not necessary. Hesina and Schmalstieg [33] proposed that user can use a circular AOI, and keep consistency of only the objects with in AOI. Additionally, if users just rotate views but do not move, the objects within the AOI need not be re-acquired. Some streaming may roughly be divided into two stages: object determination and object transmission [34]. The first stage determines which objects are visible given a user‘s view. The second stage decides the transmission order of objects based on visibility or the importance of the objects. One of the recent trends in contemporary computer graphics application is the application of more and more polygons in order to increase image (or object) realism [35]. This trend is particularly fuelled by recent developments in graphics hardware, particularly the appearance of the GPU on low-end display adaptors. This means that nextMEDIA D2.3_FF_20110325 Page 16 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” graphics adaptor not only performs scan conversion but also the 3D geometric projections and shading operations. While rendering a typical 3D mesh on a per-triangle basis, each vertex participates in six triangles on an average. Processing a cached vertex can be significantly faster than processing an uncached vertex. Thus, to maximise benefit from the cache, the mesh triangles and associated vertices must be rendered in an order, which somehow preserves locality. This ordering is called mesh rendering sequence. A good rendering sequence will minimize the average number of cache misses per triangle. 3D meshes are usually specified, for example in the ASCII VRML 2.0 file format, as a list of triangle in an arbitrary order, where each triangle is specified as three indices into a list of vertices. Simple minded renderers send these triangles to the graphics pipeline in the order specified in the file, hence achieve mediocre rendering performance. More sophisticated renderers use the trianalge strips technique, which renders the triangle mesh using a FIFO vertex cache of size 2, which is a standard part of legacy 3D hardware. Algorithms to generate triangle strips were described by Xiang et al [36] and Stewart et al [37]. However, due to the limited size of the cache, it is provably not possible to reduce the error rate below 1.0. Deering [38] first proposed a hardware model where a larger vertex cache is allowed, which the author termed as generalised triangle meshes, but did not supply algorithms to generate the appropriate rendering sequences. Chow later provided algorithm, along with Bar-Yehuda and Gotsman [39] and Lin and Yu [40]. If a client can provide unbounded disk storage and wait for a possibly very long preloading time, it is possible to transmit all virtual objects in the environment to the client before starting the interaction in the VE [41]. This approach is adopted by some existing distributed walkthrough systems. However, a more realistic situation is that the available cache storage and preloading time are limited. Furthermore, preloading a large section of a database would saturate the network with unnecessary traffic, depriving other clients of their service. To avoid this bandwidth over-utilisation, CyberWalk [41]. employs cache replacement policy to retain only frequently accessed objects in the cache and prefetching mechanisms to prefetch only potentially visible objects in order to reduce access and rendering latency. Various cache replacement policies have been proposed and their suitabilities in a conventional database system have been examined. Policies such as Least Recently Used (LRU) and Least Reference Density (LRD) are being used widely. These policies are derived from their counterpart in operating system. In the context of databases, the Most Recently Used (MRU) policy is also occasionally adopted to cater for cyclic data access behaviour. These policies are all page – based, due to the logical mapping made by the database or operating system to the physical storage. In general, the performance of individual replacement policies is sensitive to the characteristics of queries initiated. A general conclusion on the performance of the replacement policies cannot be made. In practice, replacement policy is often approximated by the LRU policy in conventional caching [42]. In [43], authors argue that LRU policy is not appropriate in a context where objects accessed by a client might change over time. Rather, the semantics of data access is more important in defining the replacement policy.
2.2.1.2 Interaction techniques and User Interfaces Although many interactive computer graphics systems are now able to render high quality shaded 3D models in real time, there remains a problem of how to interact with virtual environments in natural and error-free manner. The 3D user interface, the add-on application that would give users access to the technology‘s benefits, is a critical part of the 3D Web. A number of 3D interaction techniques have been developed by various nextMEDIA D2.3_FF_20110325 Page 17 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” researchers during the past decade. A comprehensive and recent overview is given by Bowman in [44]. Currently user interaction (UI) techniques in virtual environments are usually implemented on the application level. Given the possibility to create complex tracking behaviour directly on the tracking middle level, the idea to implement 3D user interface techniques on that higher level becomes quite obvious. A uniform approach for specifying mixed reality interfaces, including 3D UI techniques was recently published by Figueroa et al. [45]. In this ambitious attempt to describe components of 3D user interfaces, a formal specification model and a corresponding XML based description language including pseudo code were introduced. This Interface Component Description Language (ICDL) seems not to be suitable for direct implementation. The current focus of development on ICDL appears rather to be on theoretical specification than on practical implementation. VRPN (Virtual-Reality Private Network) [46] is one of the most well known and popular device-independent and network transparent frameworks for peripheral devices used in MR systems. It supports a wide variety of input devices and different types of data such as 6DOF pose data, button states, analogue values, incremental rotations and more. OpenTracker [47] is an open software architecture that provides another framework for the different tasks involving tracking input devices in MR applications. It is an appropriate framework to integrate 3D User Interface techniques directly in middleware. Tabletop displays have the potential to bring the advantages of electronic media to the type of interactions that occur over traditional desktops, and software environments like ―BumpTop‖ [48] greatly enhance the reality of interaction through physics modelling. Almost all windowing interfaces currently use shallow 3D effects to support interaction. The layering and shadowing effects both enhance the visual appeal of the interfaces and provide a natural metaphor for switching documents and workspaces into and out of focus. Some commercial interfaces further extend the 3D effects, using animations to clarify feedback effects such as distorting windows and icons to show the relationship between pre- and post-action states (Mac OS®X). Researchers are also investigating problems and solutions that arise from moving between layers on the desktop. Dragicevic [49] describes 3D visuals of dog-ears, folding and shuffling to make working with overlapping windows more intuitive. Agarawala and Balakrishnan‘s ―BumpTop‖ [48] adopts the emulation of reality on the desktop, using both rich 3D visuals and physics modelling to enrich interaction so that objects can be piled on top of one another, flipped onto their backs or thrown at others, and the visual effects of collisions depends on their mass and velocity. Stahl and Lundberg‘s [50] tabletop 3D virtual pond floats items in use to the surface and allows items to sink when they are no longer in active use. The Lumisight table [51] and Nakashima et al.‘s 3D table [52] provide up to four users with a coherent view of a 3D image in the centre of the display. While these systems are capable of rich 3D visuals in a collaborative setting, they do not fully address the interaction with these 3D models. Furthermore, these systems require a very large tabletop to achieve a small central 3D display. More efficient interaction techniques that provide control of all types of 3D rotation coupled with translation (6DOF) on a direct-touch tabletop display are presented in [53].
2.2.1.3 Virtual Reality/Virtual Worlds Virtual reality (VR, also called virtual environments, VEs) is an important technology that invites 3D interaction because of its inherently spatial nature. Initially, many of the interfaces to VR applications were designed to be ―natural‖ – e.g. to view a virtual room the user walked around it or to examine virtual objects the user picked up directly with a hand. Recent 3D internet browser-based interfaces create an instant virtual places from nextMEDIA D2.3_FF_20110325 Page 18 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” web pages, creating a new universe of inter-connected worlds for users to visit, explore, and meet people (Figure 4) [54][55]. Only in recent years have research groups been attempting to connect Brain-Computer Interfaces (BCIs) and virtual worlds. However, several impressive prototypes already exist that enable users to navigate in virtual scenes or manipulate virtual objects solely by means of their cerebral activity, recorded on the scalp via electroencephalography (EEG) electrodes. Meanwhile, VR technologies provide motivating, safe, and controlled conditions that enable improvement of BCI learning as well as the investigation of the brain responses and neural processes involved.
Figure 4: 3D browser-based virtual world Interactions with VE can be decomposed into elementary tasks [44] such as navigating to change the viewpoint or selection and manipulation of virtual objects [56]. In virtual worlds, current BCI systems can let users change the camera position in a VE toward the left or right by using two different brain signals, such as left- or right-hand motor imagery (MI) or two steadystate visual-evoked potentials (SSVEPs) at different frequencies [57]. MI-based BCIs have also been used to explore a virtual bar [70] or move along a virtual street [58]. The connection between BCIs, videogames, and VR technologies offers a promising research area for the next user interaction systems. A relevant aspect related to the adoption of 3D media applications regards their usability and the quality of experience perceived by users. Indeed the quality of experience when interacting with such kind of environments depends strongly on the effort that users make to engage in the applications due to the required cognitive load to process ―unnatural‖ 3D interfaces. In fact different approaches for minimizing the cognitive load are under study such as the use of motion constraints in 3D environments for providing navigation guidance [59]. This approach in addition to taking away some of the bewildering freedom that causes the high cognitive load in the first place, also has the benefit of supporting the user maintaining an accurate cognitive map if the path is designed to visit all of the important landmarks of the environment. Other approaches based on use of table-top augmented reality environments obtained relevant results for evaluating the ability to complete tasks in 3D when compared to 2D. In general, it was identified that that 3D objects contain more information about themselves and their relative position in space than 2D objects, although users take longer to process this information they are more accurate when searching and retrieving 3D objects [60]. Besides navigation and task execution, recent research has focused on the QoE of interaction for manipulation of 3D environments [61]. Previous studies have identified that in general a change in the input devices for manipulating 3D environments influenced perceived control but not the perception of the actual environments [62]. In this sense, recent research is exploring different input/output modalities within 3D
nextMEDIA D2.3_FF_20110325 Page 19 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” environments including tangibles [63], mobile [64] or even combination of modalities to provide research interactions or enable impaired users [65].
2.2.2 Research Challenges Table 4 shows the running ands newly started projects against the research areas that are related to the FMI and are currently state of the art. Most running and newly started projects of the 3D Media cluster have in common that they use 3D Display, modeling, capturing and delivering issues, targeting application such as 3D TV, Games and Immersive Environments/Virtual worlds.
QoS
3D
Format
Cinema
3D
Networking Formats
Caching
3D TV &
Capturing
Mobile3D
3DDisplay
New Media New
Multimodal
Technology
3DDelivery 3DNetwork
Environment
3D Immersive 3DImmersive
Teleconference
3D
Films & Games Films& Virtual Worlds 3DTV HOLOVISION MUTED PlayMacer VirtualLife MOBILE3DTV 3D4YOU Digital holography HELIUM3D 3DPRESENCE 2020 3D Media i3DPost 3DLife 3DPHONE Real3D 3D VIVANT DIOMEDES FascinatE MUSCADE SkyMedia
Table 4. Comparison of 3D Media Cluster projects Looking into the future, research challenges that we may highlight are: 3D Content Capturing. For 3D content capturing, 3D capture techniques (including multi-view capture) are still being developed, and more work is needed both on the capture methods themselves, but also on the methods of data representation and compression to allow transmission over different kinds of networks and reproduction on different display devices. In parallel, optimum methods of surround sound capture need to be determined, to produce a sound field well-matched to the immersive
nextMEDIA D2.3_FF_20110325 Page 20 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
visual experience. Research is also needed to explore the benefit that may be achieved by adding new modalities such as smell and haptics. 3D Content creation/rendering. Research is needed on the engineering methods and tools to support fast and easy 3D content creation, along with optimum methods for capturing the visual appearance of the real world, including 3D/multiview, high dynamic range and high frame rate. For the 3D content creation, one of the recent trends is the application of more and more polygons in order to increase image (or object) realism via contemporary computer graphics applications. However, better rendering systems are needed or better rendering sequences in order to minimize the average number of cache misses per triangle. 3D Content compression & coding. The access to the FMI content should be supported anywhere, anytime and using any terminal. Thus, the 2D/3D media content should be extensible and scalable, in other terms it should be adaptable to various usage environments and available to a wide range of users. Similarly, 3D audio content should also be rendered in the best possible way to give the user the sensation of 3D immersive experience although his/her terminal may not be capable of rendering the 3D content. Therefore, it is important to generate the FMI content in a scalable manner during either its capture or compression stage. Cross-modality (e.g., video, audio, text etc) and multi-sensory (i.e., new modalities for sensory, olfactory information) aspects of the original content should also be provided interchangeably, so as to make the FMI services extensible to various network conditions, terminal features, user preferences etc, and thus available to all. 3D User Interfaces. Future 3D rich multimedia interfaces will be needed to support the novel input modalities and the envisioned interactive applications. As we witness a shift from graphical interfaces to multimedia interfaces, developing richer and more intelligent interfaces is a key challenge to the adoption of the envisioned applications. Such interfaces will give more flexibility to the use by offering a choice of modalities depending on the nature of the information conveyed. Finally, multimedia user interfaces provide the adaptability needed to effectively operate in changing operating conditions. 3D navigation with physical and emotional involvement of the user. As an extension to the previous challenge, novel 3D navigation methods will be needed able to take into account not only the audiovisual, but also new types of haptic feedback and user‘s emotional involvement. During real world navigation the user is primarily guided by the audio and visual channel, but in the future the advancement in portable haptic interfaces will allow receiving new types of feedbacks. The interaction of the user with the navigation support will become hands free, with the possibility of tracking the gestures performed around the user's space, customizing or modifying the navigation process. Emotional involvement will imply the reconsideration of the 3D rendering process, which will integrate realism with non- real rendering of objects, users cues, and actions, in order to enhance the expressive and emotional communication within a community sharing the virtual/mixed environment. Virtual Worlds/Virtual Characters creation. The creation of virtual worlds and virtual characters with emotion and personality will increase the realism and quality of interaction such as in games, story-telling systems, interactive dramas, training systems and therapy systems. Emotion is another major component of personification since in real life emotions affect all a person‘s cognitive processes, their perceptions, beliefs and the way they behave. nextMEDIA D2.3_FF_20110325 Page 21 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
2.3 Search and retrieval of multimedia content Retrieving multimedia networked content (audio, images, 3D and videos) from the web is becoming a major use of Internet these last few years. This is confirmed by the growing popularity of social networks, content sharing and custom user generated content. However, the multimedia retrieval challenge is not efficiently addressed and the relevance of returned results is not satisfying. The main reason to explain the latter is in the algorithms used to perform multimedia search. In fact, most existing systems for retrieving images, music, videos, 3D, etc. rely on text-based retrieval algorithms applied on textual description of the considered multimedia content. In the web, this textual description is often built from a window of k words preceding and following the multimedia content in the html code of the container. Most of images and videos search engines (Google image, Exalead Image and Video, Yahoo image, etc) are based on this technique. The main limitation of this technique is that the textual description found in the web for an image or a video is not necessarily informative about the effective content. In addition to this, multimedia content has specific features that cannot be represented by a set or bag of words (texture for image, descriptor of videos, etc). In this section FP6 - FP7 funded and national search related projects are analysed. Then, a revision of the SoA in multimedia search is given. Finally this section concludes with the provisional research challenges (for a complete analysis of the search related projects can be found at www.ist-chorus.org/).
References to relevant nation and EU projects in this topic may be found in [123].
2.3.1 Current State of the Art Content-based image retrieval (CBIR), also known as query by image content (QBIC) and content-based visual information retrieval (CBVIR) is the computer vision problem for image retrieval, that is, the searching of digital images in large databases based on their content information. The underlying search algorithms vary depending on the application, but result images should all share common elements with the provided example, which can be either a pre-existing image supplied by the user, or a rough approximation (sketched) of the image. Many algorithms have been presented in the past based on different image criteria (e.g. texture, contour [73] and colour [74]) depending on the application and the dominant content of the image database. Recently, the research community has focused on the combination of the existing low-level approaches with semantic information in order to increase the retrieval accuracy. Metadata semantics have been proven to be a very effective and supportive tool for traditional and image-based search engines. Regarding Content-Based Video Retrieval, if we look at TRECVID benchmark as reference, we can see that most of the techniques are generally based on frame descriptions techniques to extract visual concepts. Some recent scalable techniques based on temporal and dynamic information are also adapted to videos: Key-frames strategies. In [75], two types of elliptical affine co-variant regions are used to describe the image contents of key-frames. Each region is represented by a 128-dimensional vector using the SIFT descriptor developed by Lowe [76]. These descriptors are post-processed in order to build a ―visual vocabulary‖. The objective of this post-processing is to vector quantize the descriptors into clusters representing the visual words as for text retrieval. Each descriptor for a new frame of the movie is
nextMEDIA D2.3_FF_20110325 Page 22 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
associated to visual words. The final representation of video is a set of key frames, and each key frame is represented by the visual words and their position. This strategy allows retrieving objects in videos. This type of technologies could be efficient to track objects, sets and people in a video or a set of videos. In [77] the used local features are based on an improved version of the Harris point detector and a set of local jets computed in the neighbourhood of each interest point. This type of descriptors has a limited robustness for scale changes and is not invariant to rotation. However, they have the advantage to be very discriminative, robust to localization errors and require a low computational cost. This makes them very efficient for content-based copy detection tasks. The technique fits well for hundreds of thousands hours of videos and is then very adapted for the scalability issue. Spatio-temporal interest points. Instead of detecting points of interest in key- frames, Ivan Laptev and Tony Lindeberg have extended the concept of points of interest in the spatiotemporal space These spatiotemporal points of interest have been combined with different types of descriptors and have been used to recognize types of activities [78] and applied to video copy detections [79]. Spatiotemporal points have the advantage to characterize very salient patterns and are thus adapted to describe temporal events. Moreover, this saliency is very efficient for the video copy detection. However, such method makes difficult the process of very large catalogues of videos, because the computational costs are very high. Interest points trajectories. New descriptions of video sequences, based on the dynamic content have been proposed. In [80] an equivalent indexing strategy is proposed, but using two different kinds of local features (Local jets around Harris points and Symmetry points) and an asymmetric feature extraction strategy. Interest point trajectories are efficient to describe the dynamic contents of the video sequences. Moreover, the dynamic information is estimated with the trajectories of the interest points that allow compressing temporal information, contrary to the method founded on the dense optic flow. However, this strategy requires computing the interest points on each frame of video sequences, which induces high computational costs. This could be a bottleneck when the aim is to process a very large catalogue of videos. 3D object search and retrieval is a relatively new and very challenging research field and a major effort of the research community has been devoted to the formulation of accurate and efficient 3D object search and retrieval algorithms. The existing 3D object retrieval methods can be classified into four main categories: histogram-based, transform-based, graph-based, view-based and, finally, combinations of the above. Despite the numerous advantages and shortcomings of each category, it has been proven experimentally that the best retrieval performance can be achieved through the transform-based or the view-based approaches. Transform-based methods are employed either on the surface [81][82][83] or on the volume [84][85][86] of a 3D model. The outcome is usually a multidimensional descriptor vector representing the global shape characteristics of the 3D object. In order to address partial shape retrieval problems, a move from the global shape description to a local representation of the shape characteristics is needed. In this case, graph-based solutions are preferred [87], which are able to encode geometrical and topological shape properties in a more faithful and intuitive manner. The drawbacks of the graph-based methods are that it is difficult to implement them, they do not generalize easily to all 3D shape representation formats and they require dedicated matching schemes. A more comprehensive solution is given by 2D view-based methods, which can adequately deal
nextMEDIA D2.3_FF_20110325 Page 23 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” with partial shape retrieval, while at the same time they achieve high retrieval accuracy in global shape retrieval tasks[88]. A great interest for Music Information Retrieval (MIR) emerged in recent years. The very successful series of International Conferences on Music Information Retrieval, coordinated by ISMIR (http://www.ismir.net), bears witness of such interest from both industry and academia. Different important research challenges are addressed in this community, from ―query by humming‖ to the classification of audio based on musical genres. As for identification of audio features contributing to explain music content, we may identify timbral related features, rhythmic related features, and pitch related features. Audio data that are to be classified cannot be represented as raw audio sample data, therefore some sort of parameterisation is required, based on audio analysis. Several methods such as Fourier transform, wavelet transform, statistical methods, etc. are available in the literature. Examples of features based on timbre are spectral centroid, spectral roll off, and time domain zero crossing, which measure the spectral shape, the changes in spectral shape and the noisiness of a signal respectively. Other features, such as Mel-Frequency Cepstral Coefficients (MFCC), also based on the STFT and borrowed from speech processing, are typically used. Beat features extraction is another important component. Overviews on beat tracking methods are available in the literature (e.g. Dixon). Another important category of musical audio features is related to the gesture and movement which is alluded in the music signal: early research results are available in the literature (see for example the ―danceability‖ measure of a music signal developed at MTG-PFU, Barcelona by the group of Xavier Serra). For a complete state-of-the art please visit [89].
2.3.1.1 Search and retrieval of multimedia using real-world data The discovery and utilization of sensors technology have made possible the efficient collection of real time data. The availability of this data adds a new dimension to a wide range of applications. A quite promising aspect of taking advantage of available sensor data is semantic annotation of multimedia (image, video, sound) with context metadata. Context metadata include spatial and temporal information such as the spatial region and temporal interval contained within a video. Space and time metadata can be provided by associating each video with the spatial and temporal coordinates of the corresponding sensor at a particular location and time. For example, spatial and temporal information such as the spatial region and temporal interval related to a video could be used for a more efficient discovery and retrieval of video files within a specific time interval or geographic location. Space and time metadata can be provided by associating each video with the spatial and temporal coordinates of the corresponding sensor at a particular location and time. An approach of annotating video sensor data with spatial, temporal, and thematic semantic metadata is presented in [90]. The developed prototype provides an environment for the discovery and retrieval of videos by semantic temporal concepts such as within, contains, or overlaps when querying with an interval of time. It incorporates the standardization efforts of the OGC and W3C. In [91] is studied how a sensor-rich world can be exploited by digital recording devices such as cameras and camcorders to improve a user‘s ability to search through a large repository of image and video files. The developed ―SEVA‖ system captures a stream of sensor data and a video stream and fuses them together in a series of stages. Augmented recording is a foundational component for the UCLA Hypermedia Studio's research into the use of sensor networks in film and video production. During a filming of a scene, sensor data such as light intensity, color temperature and location are nextMEDIA D2.3_FF_20110325 Page 24 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” collected and synchronized with each film or video frame. Later, editors, graphics artists and programmers can view this data in synchronization with film and video playback. ContextCam [92], is a prototype of a consumer video camera that provides point of capture annotation of time, location, person presence and event information associated to recorded video. Both low- and high-level metadata are discovered via a variety of sensing and active tagging techniques, as well as through the application of machine learning techniques that use past annotations to suggest metadata for the current recordings. As far as the environmental monitoring is concerned, many types of sensors have been developed and used for a plethora of purposes. An initial categorization of sensor types could be physical, biological and chemical [93] .
2.3.1.2 Meta and multi-domain search Merging, integrating and unifying search results from several data sources are far from being novel. Many systems and approaches have been proposed in early 90‘s. One major approach was the development of meta-search bots on top of existing search engines and on-line databases. The main idea was to build a unique interface that takes a user query as input and submits it to the integrated search engines. The results are collected, processed and re-ranked to present the most ―relevant‖ hits. Duplicate hits and dead links are discarded from the returned results to the user query. Meta-search systems do not store, index and cache documents and no feedback is given to the integrated search engines. MetaCrawler [94][95], Harvest [96], SoftBot [97], SavvySearch [98] and MetaSpider [99] are the main academic solutions based on the meta-search paradigm. Meta-search engines rely on the HTML interface or the dedicated search API (when this exists) of the integrated search services. They are therefore independent from internal implementation of the integrated search services (index structures, formats, etc.). However, a meta-search engine is strongly depending on the API of the integrated search services. Integrating a new service implies the development of new custom ad- hoc communication components like HTML parsers to extract hits or custom code to address APIs of search services. These components have to be updated, usually by hand, at every search service API or interface changes. This is a very limiting factor to build robust search applications. On the other hand, the approach that consists on unifying results coming from different search engines is not as promising as one could expect. Indeed, since the meta-search engines do not index the data themselves, they are lacking of a set of useful information that allows them to produce a relevant ranking, like the content of the hits, the web graph surrounding the hits, the context of the user query, etc. The merging process relevance is then often lowered by the lowest quality of the integrated search services.
2.3.2 Research challenges Growth of popularity of media is not accompanied by the rapid development of networked media search technologies. The most popular media services in the Web are typically limited to textual search. This method of searching is not entirely natural for humans. If we are performing a search in the real life, we do not only query by providing a spoken or textual description of the searched item or service. We provide examples: ―Do you have a jacket, like this one, but dark green, rather than orange?‖; ―Could you please give me a haircut, like the one in the catalogue?‖ This allows us to omit long and unnecessary description of the searched item. Such services are nowadays available in the Web only in form of low scale client-server test-beds and are not
nextMEDIA D2.3_FF_20110325 Page 25 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” present in P2P1.
tics
Media
-
3D
Text /
Video
Image
Mobile
Indexing Searches
Networks
Searching
Seman
Distributed
Personalised
Cross
P2PSearching Speech / Audio DIVAS PHAROS RUSHES SAPIR SEMEDIA TRIPOD VICTORY VIDI-VIDEO VITALAS PetaMedia i-Search LivingKnowledge COAST Glocal
Table 5. Overview of multimedia search engines in 9 FP6-IST projects Taking into account the past and current research in the EC along with the current state- of-the art, we can identify the following research challenges: New ways of information representation including multimedia & things & context & device type & user‘s intention. This means that a convergence provisioned in the context of FI with respect to Internet of Content (IoC), Internet of Things (IoT) and Internet of Services (IoS) should be also applied to search creating completely novel applications like search in Massively Multiplayer Online Role-Playing Games (MMORPG), in multimedia social networks, in 3D virtual worlds and so on. Extraction (in real time) and interpretation of multimedia and multiparty communication (speech, posture, gesture). The latter implies the need for novel methods for efficient interaction with content which in turn could lead the research into: o Novel interoperable interfaces able to describe and enable easy retrieval and access by users to multimedia content under a context-aware and user-centric framework. o Analysis of the users behaviour with respect to contextual information, in order to optimise the search and query techniques, taking into consideration customisable UIs, and retrieve the most appropriate multimedia for the specific user, with main emphasis on location-based data acquisition Automatic, content-based large scale multimedia indexing, which implies the need for new algorithms able to automatically extract, understand, classify and store discriminative features in a centralised or distributed manner
1 Please note that project COAST objectives fit in more than one clusters nextMEDIA D2.3_FF_20110325 Page 26 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Unification of multimedia content search combining real world information and social descriptions into one single descriptor (mixed multimedia search) able to integrate: o intrinsic properties of the content (static features such as shape, colour, texture, dimension, etc.); o dynamic properties (temporal descriptors, how it behaves, in which activities it is normally used, who uses it, etc.); o non-verbal expressive and emotional descriptors; o social descriptors (how content is related to users, social/collaborative use of the content) The above mentioned research challenges could lead to a new era in search and retrieval applications, however, the plethora of search services should intelligently managed by adopting methodologies from the IoS with respect to automatic service discovery, assessment and use following a ―white box‖ approach. Last but not least, of paramount importance is the need for the creation and use of search benchmarking systems, which would cover all critical aspects, including the user perceived quality. .
nextMEDIA D2.3_FF_20110325 Page 27 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
2.4 Research Challenges in User Centric Media/Social Networks In the last years, the role of end users in multimedia landscape has been becoming more and more active. All began with the explosion of the user generated content in the World Wide Web, which meant a great deal of opportunities for end users to cooperate in the development of Internet: forums to participate in, blogs to create in everyone‘s own way, content being uploaded to video portals, games to play with people from over the world, are only some examples of the great deal of possibilities of the User Centric Media over the Internet. One of the names used was ―Web 2.0‖, which meant a change in philosophy on Internet and a new focus on end users, who get not only a major participation, but also they became the main role. Focusing on media, this has generated the phenomenon called User Centric Media. User control has great advantages for end users, since it allows them to have a much more personalized access to the multimedia information. The great variety of user genated content open need research needs on different domains. For example, user‘s preferences to select the more relevant pieces of user generated content according to their profiles. Moreover, it also poses some relevant challenges for the Media industry when having to adapt the Media value chain to the new user control and the new role of users as content creators. This fact involves new business models far from the traditional broadcasting approaches [100].
References to relevant nation and EU projects in this topic may be found in [123].
2.4.1 Current State of the Art As it can be seen, projects related to User Centric Media have some properties in common. Some of them deal with 3D technologies or new networking structures, whereas others are more focused on the content itself and the ways in which distribute it. The specific parts to describe the current state of the art (not covered by other sections of the document) are those related to three main subjects: User Preferences & Profiles, Collaborative User Generated Content and User Generated Virtual Worlds.
2.4.1.1 User Preferences & Profiles Due to the change of the user role in the Internet, their preferences are one of the most worthy information to be used to provide new media tailored contents and services. In order to select which resources are needed to build the user‘s profile which will determine the actions that a user-focused system should perform. To elaborate the user profile, the most convenient way is to analyze the user behaviour when consuming and interacting with content. The automatic electronic data gathering and analysis, has been used in Internet environments normally for Internet usage measurement. This is not directly applicable but useful for the statistical modelling of the user behaviour data. In [110] the authors describe some methods for Internet user behaviour analysis based on access traces and its application to discover communities based on a self-similarity model. Other authors [111][112] extract the audience information and user‘s interest from the routine visits and web log data. An interesting method is in [114] for inferring identity from user behaviour using Bayesian statistics applicable to TV program consumption. Other interesting methods are those using classifying algorithms, such as binary classifiers
nextMEDIA D2.3_FF_20110325 Page 28 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
[102], or Bayesian classifiers, which allow to create a user‘s profile through feedback analyzing the user‘s behaviour [103]. Another widely deployed systems are the recommendation systems for context-aware content [115] which are based on content filtering tools. Techniques such as content- based filtering, collaborative filtering and hybrid methods use the information on content consumption as the input and feedback for their system characterisation. Collaborative filtering is interesting for its applicability to IPTV scenarios and media delivery over the Internet. In collaborative filtering two kinds of algorithms are normally used: memory-based approaches [116] and model-based approaches. In memory-based approaches the prediction of the rating is made on the basis of the ratings of other users with similar interests. This can be applied to consumption in communities or for community discovery. Model-based collaborative filtering techniques [117] first learn a statistical model and then predict the ratings based on the model learned. Hybrid methods have been proposed [118] using the content assets to improve the rating prediction, and traditionally have been used mainly for dealing with textual information. However, hybrids methods using visual information are just starting but with very promising results [115]. Other source of information about user preferences are the queries in search engines bars. Users‘ profile can be created from the information extracted from those queries, as well as the links which the user visits afterwards, since each of them represents a subject in which the user is interested. This has been used in some applications, for example to distinguish the response to the same query to be given to different users requiring a different response according to their profile [101]. Now, with the increase of amount of multimedia information needed to organise and classify, it is going to emerge many new applications of this, such as preference of the user in TV at choosing TV programmes or movies on TV.
2.4.1.2 Collaborative User Generated Content The term User Generated Content refers to the type of media content produced by end users. Nevertheless, there is not an official definition for this term, so the Organisation for Economic Co-operation and Development (OECD) has proposed three main characteristics [104]: Publication requirement: UGC has to be published in some context, such as a website or a social networking. Creative effort: it is important that the information is provided from personal knowledge of the user. Creation outside of professional routines and practices: usually UGC is generated without expectation of remuneration. There is a wide range of applications, mainly containing text or images such as blogs, wikis, discussion boards, social networks, photo sharing sites, location systems, trip planners, etc. but perhaps in media the videos exchanged in web-sites are the most popular. With the advent of Web2.0, user generated content is shared in an increasing number of ways. The collaboration in generating content started from communities writing blog describing their daily lives with text and photos. But day by day sharing personal photos and videos, some of them user generated, some of them as a result of the professional videos edited by a user, is becoming more and more popular on the Internet. As an
nextMEDIA D2.3_FF_20110325 Page 29 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” example, with the convergence of mobile, blogging and geolocational technologies, the mobile applications of user generated content are increasing [105].
2.4.1.3 User Generated Virtual Worlds As a more advanced and sophisticated subject than the previous one, the term User Generated Virtual Worlds refers to the creation of not only sites for sharing knowledge, but also a completely parallel world in which users can interact and cooperate with other members of it. There is a great deal of applications increasing day by day, such as entertainment, education, training, e-commerce or business. Virtual worlds are a paradigm of development of media centred on users, since a virtual environment is mostly designed for satisfying the preferences of a specific user. In these systems, users are completely involved, physically and emotionally. The professional generated virtual worlds are normally computer generated or reconstructing scenes with digital cameras and scanners. To generate user virtual worlds, it is needed to use low-cost cameras (e.g. web cameras or cameras mounted on mobile phones). For example, Nokia's image space (research.nokia.com/research/imagespace) which needs a specific handheld with a geo-localization module and a compass, making possible to add the context to the user generated content. Microsoft's Photosynth (photosynth.net/about.aspx) allows the user to ―take a bunch of photos of the same scene or object and automatically stitch them all together into one big interactive 3D viewing experience‖ that the user can share on the web. Projects like MagicBook and HistoryCity use virtual world for entertainment, specifically for children. MagicBook [106] consists of an interactive virtual book in which the reader can become in another character of the story and move around its pages. Real books often serve as the focus for face-to-face collaboration and in a similar way multiple people can use the MagicBook interface at the same time. Several readers can look at the same book and share the story together. If they‘re using the augmented reality displays, they can each see the virtual models from their own viewpoint. Since they can see each other at the same time as the virtual models, they can easily communicate using normal face-to-face conversation cues. HistoryCity [107] is a virtual world developed for children as a collaborative, distributed environment that can support 500 simultaneous users. Residents of HistoryCity explore, and live in, a virtual Singapore of 1870, complete with historical buildings, costumes, and objects. At present there are 24 communities in HistoryCity, each of them have several clubhouses. When users first join HistoryCity, they select avatars as their representations and these become the ―bodies‖ that other users see. Currently HistoryCity provides around 200 different avatars. In the user centric media cluster, VirtualLife (www.ict-virtuallife.eu) is showing some examples of these collaborative environments.
2.4.1.4 Social Networks Internet Services It should be noticed that the purpose of this chapter is not to give an exhaustive and thorough state-of-the art in each and every area. Other areas should not be excluded. Yet, we highlight the areas that have been selected as the most important ones and give the interested readers ‗the first step‘ for their further research.
Social graphs. A graph is a mathematical abstraction for modelling relationships between things. A graph is constructed from nodes (the things) and edges (the relationships). This mathematical tool that can model natural and artificial systems nextMEDIA D2.3_FF_20110325 Page 30 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
such as economy, deceases, power grids, etc. has been used by the anthropologists, sociologists and other humanities oriented academics. However, graph analysis and social network analysis are also valuable tools for studying the web and human behaviours of the web users. Social network analysis may be applied in any web field where a graph may be constructed. From the appearance of social networking sites, users were forming graphs with their friends and this was the ideal source of fresh data to apply social network analysis. One of the most prominent issues in social networks is the formation or the identification of a network of nodes based on real world knowledge (school friends, colleagues etc.) or web extracted knowledge (they are part of the same online community, they like the same movies, etc.). Social network analysis applied in the web by utilising the interconnected Web 2.0 blogs and their comments. Backlinks of posts and the blogroll (list of other blogs) of each blog constructed a graph that could provide some information. This structure was difficult to update, error prone (copy paste links, write urls, etc.) and the users had to have a web page or blog of their own. Social networking sites created the tool that made relations easier to track and build. Now every user that has an account in a SNS can ―tag‖ information and propagate it to that network. ―Likes‖, ―tweets‖, ―diggs‖, etc. are one button actions that users perform while surfing the web in order to post a piece of information without leaving from the current web page. Facebook Open Graph2 provides an interface for interconnecting web pages with the Facebook social graph. The most common practice is to add a ―Like‖ button near a media object in a web page and let users share their ―Likes‖. When a user clicks the ―Like‖ button outside the Facebook platform, a new connection is formed in the user‘s profile. This simple API has significant impact on the generated content in the Facebook platform considering the 500 million active Facebook users that surf the web and collect ―Likes‖.
Microblogging. Microblogging websites are services, which enable their users to post small text messages that will update their personal profile and will also be delivered to their list of friends. Microblogging became extremely popular due to twitter, however other microblogging sites exist as well with similar functionalities. Twitter currently has approximately 190 million users and growing. After its extreme popularity and due to its flexibility and integration in several platforms (mobile phones, desktop apps, email alerts etc.) marketing experts invented a new tool for fast and targeted advertising. The statistics provided from pearAnalytis3 study reveal that almost 44% of the posts are spam and pointless, about 6% are personal or product advertising, while 3.6% are news and 37.6% are conversational posts. However, there are many other microblogging services available. Tumblr4 provides similar functionalities, however more focus on the design and styling is given. A more flexible microblogging solution is posterous5 that provides a unique way to post small messages to various destinations. Posterous enables a user to connect his/her social networking accounts to the posterous platform and to deliver the posts also to other social networking sites or email addresses of choice. One very powerful feature that is relatively new for microblogging as well as for the social networking platforms is the ability for the user to post a message only to a selected
2 http://developers.facebook.com/docs/opengraph 3 http://www.pearanalytics.com/blog/wp-content/uploads/2010/05/Twitter-Study-August-2009.pdf 4 http://www.tumblr.com/ 5 https://posterous.com/ nextMEDIA D2.3_FF_20110325 Page 31 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
group of friends or other recipients and not necessarily for the whole list of friends. A new mobile service that extends this feature and works on the boundaries of microblogging is GroupMe6. Users of GroupMe compile groups of their contacts and use a new unique telephone number to reach the group. Identity algorithms and APIs. Identity algorithms are used to avoid the pollution of the social graph or provide easy access to potential users. In the process of opening their platforms to the outside world, SNSs embraced or build several APIs. The aims of these APIs are to engage more users, get more content from the web or ease the access of users and interlink their content in various sites. The following paragraphs of the subsection refer to such APIs and protocols that are used in most of the known SNSs or sites that provide third party social networking functionalities. OAuth provides a method for clients to access server resources on behalf of a resource owner (such as a different client or an end-user). It also provides a process for end-users to authorise third-party access to their server resources without sharing their credentials (typically, a username and password pair), using user-agent redirections 7 . Facebook platform uses OAuth 2.0 for authentication and authorisation in desktop as well as in mobile applications. OpenID is a protocol that provides easy sign up or sign in functionality for the users. Most of the widely known web and social networking sites use openID to provide their users with single sign in username and password. Sites like Google, Yahoo!, Face-book, MySpace8, flickr9, wordpress10 and many others use it in order to allow users to enter without the discouraging registration forms and with a unique login for every service. By registering once, a user is able to access any service that supports openID with the same username-password pair that s/he used for the first service. Except for the above mentioned services that act also as openID providers, there are also dedicated openID providers such as claimID11, myOpenID12, etc. Social media search. Since the domination of social networking sites as the primary channel of communicating ideas and sharing media, new social search engines emerged. However, these search engines crawl the social networks and index the available content based only on text. Some of these keyword-based social search engines are: Spy13, SamePoint14, SocialMention15, WhosTalkin16, wikio.com.17 Spy is a web application that is updated in real time and provides the user with the ability to watch what is being said in a certain topic in specific social networking sites and blogs. SamePoint provides an easy interface for the user to select in which of the social networks to search for a keyword or topic. SocialMention works like Google alerts18 but for social media. WhosTalkin is a social media search tool that allows users to search for conversations surrounding the topics that they care about
6 http://groupme.com/ 7 http://oauth.net/ (accessed Nov 2010) 8 http://www.myspace.com/ 9 http://www.flickr.com/ 10 http://wordpress.com/ 11 http://claimid.com/ 12 http://openid.net/get-an-openid/ (accessed Nov 2010) 13 http://spy.appspot.com/ 14 http://www.samepoint.com 15 http://socialmention.com/ 16 http://www.whostalkin.com/ 17 http://www.wikio.com/ 18 http://www.google.com/alerts/ nextMEDIA D2.3_FF_20110325 Page 32 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
the most. Wikio is a personalisable news page featuring a news search engine that searches media sites, blogs and the contributions of Wikio members.
Content management in large scale. In the scale that most of the prime social networks operate, even the most common operations are not trivial. The most powerful example is Facebook that has to handle almost 500 Million active users that share more that 3 billion photos per month and its servers should serve about 1.2 million photos per second19. For such volumes of content management becomes a very crucial issue. Here we refer to some technologies and tools that most of the social networks use in order to survive the torrents of queries. Memcached20 is a distributed caching system that caches database queries in order to minimise the relatively slow database access. Memcached started from LiveJournal blogging and social networking site and released as open source. At this time Facebook runs thousands Memcached servers with tens of terabytes of cached data. Cassandra21 is a distributed storage system with no single point of failure. It‘s one of the poster children for the NoSQL movement (others are MongoDB, Redis etc.) and has been made open source (it becomes an Apache project). Cassandra is in use at Digg22, Facebook, Twitter, Reddit 23 , Rackspace 24 , Cloudkick 25 , Cisco 26 , SimpleGeo 27 , Ooyala 28, OpenX29, and more companies that have large, active data sets. The largest production cluster has over than 100 TB of data in over than 150 machines.
Human powered and community question answering. Human powered systems emerged from the social networks, which provided the ability to the user to contribute with web content. Since artificial intelligence and computer vision problems were consistent, the researchers envisioned that the solution to unsolved problems was to harness the human intelligence. However, to engage users to answer questions, annotate image or proofread OCR extracted text for free had to have something as a reward. Towards this end the ―games with a purpose‖ (GWAP) appeared. In a GWAP the user answers or solves difficult for a computer but easy for a human problems while s/he plays an online game. In the same track, online community question answering sites provide a place that everyone can contribute by answering questions from other members. The answers are validated by a ―start-based‖ system where the end user gives feedback whether the answer was helpful or not. Some of the well known community questions answering systems are yahoo! Answers30 for general questions, stackoverflow31 for questions on programming, serverFault 32 for server administrators and IT professionals or ―Seasoned Advice‖33 for cooking professionals and many others.
19 http://royal.pingdom.com/2010/06/18/the-software-behind-facebook/ (accessed Nov 2010) 20 http://memcached.org/ 21 http://cassandra.apache.org/ 22 http://digg.com/news 23 http://www.reddit.com/ 24 http://www.rackspace.com/ 25 https://www.cloudkick.com/ 26 http://www.cisco.com/ 27 http://simplegeo.com/ 28 http://www.ooyala.com/ 29 http://www.openx.org/ 30 http://answers.yahoo.com/ 31 http://stackoverflow.com/ 32 http://serverfault.com/ 33 http://cooking.stackexchange.com/ nextMEDIA D2.3_FF_20110325 Page 33 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
A very interesting service is the Aardvark34 search engine, which finds the most relevant person from the user‘s contact list and the entire community of the users to answer a question. Aardvark accepts questions in natural language (not just keywords) and uses a novel algorithm35 in order to map the question to the most relevant recipient. Mobility and geolocation. Location-based social networks allow members to share their location through GPS, Bluetooth, email or text messaging. The member of the network may also add comments about restaurants, allow friends to know where you are going, share information, or find friends that are few blocks away or even in the Café across the road. Several mobile-only social networks have emerged, all with unique features that would potentially attract users. However, there is one feature that every mobile social network should have and this is physical presence detection and information exchange36. This is exactly the reason for porting a community in a mobile device and detach it from a desktop PC. Some of the well known location based mobile social networks are: BrightKite 37 , Aka-Aki 38 , Mobiluck 39 . The mobile social network sites may be clustered in 6 main categories 40 based on their dominant features. These are: 1. The group texter: This service focuses on sending short, text-based messages in a group of people at once. 2. The radar: The radar knows where the user and his/her friends are. These sites support location-based services by keeping track of where the user‘s contacts are. Most of these sites allow the user to check if there is anyone close to a particular venue or location, while some of them actively alert him/her if any of his/her contacts are within a certain distance. 3. The Geotagger: These sites allow users to tag locations with images and information that appear on a world map. The user may tag favourite places for shopping, dining or any other activity and share these tags with their friends and the network. 4. The dating service: These sites are identical to their online counterparts. Users create a profile and they are matched with other users. Some also use radar features to alert the user if an interesting (according to the profile) person is nearby. 5. The social networker: These sites aim to be as similar as possible to online social networking platforms. Some of the well known SNS such as Facebook and twitter have also a mobile version. 6. The Media share: These sites share media files with groups of people. There are also location-based social networks that are not targeting mobile devices only, such as picasa and flickr image geotagging services.
34 http://vark.com/ 35 http://vark.com/aardvarkFinalWWW2010.pdf 36 Michael Arrington, Techcrunch.com, http://techcrunch.com/2007/09/11/the-holy-grail-for-mobile- social-networks/ (accessed Nov 2010) 37 http://brightkite.com/ 38 http://www.aka-aki.com/ 39 http://www.mobiluck.com/en/ 40 http://www.gomonews.com/moso/ nextMEDIA D2.3_FF_20110325 Page 34 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Social rankings and vertical social networks. Although these may not be considered as pure social networks, special attention should be given to the social sites which are dedicated to vertical markets such as the travellers community (for example tripadvisor41, travbuddy42 or dopplr43), films or music. These sites offer services opinion sharing by the users and they compute rankings. For example, dopplr allows its members to register their personal and business travel plans and get alerts for friends in same places, travel overlaps, or get travel advice from other travellers. One of the open issues is to establish methodologies and tools to distinguish between real and fake opinions on those social sites, which is applicable to social networks in general.
2.4.2 Research Challenges All the projects of the User Centric media cluster have in common that they aim to have the user in the middle of the media chain. Therefore they cooperate in the User Centric Media Cluster within the Networked Media unit, which coordinates strategic research directions44.
ontent
rative User rative User
Profile
content
Content
3DVirtual
Experience Experience
New Media New
broadcasting
Reconstructed
Interactivity in Interactivity in
Virtual Worlds
User Generated Generated User Generated User
Generated C Generated
3Dgamingserious
Collabo User Preferences & & Preferences User MyMedia VirtualLife TA2 PlayMancer Games@Large CITIZEN MEDIA FascinatE OMWeb My eDirector 2012 Table 1. Comparison of User Centric Media Cluster projects Taking into account that people have less and less difficulty to access to the multimedia information, it becomes clear that the content is expected to play a significant role in the Future Internet. In this sense, we can identify at least three main topics where the User Centric Media will have to address clear challenges [109]. We may identify the following research challenges in the area of User Centric Media: Personalization: user centric systems should allow users to have an efficient way of locating the desired information based on their preferences and their profiles. Adaptive Multimedia Content: create new approaches to media containers that allow a semantic-based, personalized creation and delivery concept. Content Mobility and Accessibility: media should adapt to a form that suits the user independently of locations and device.
41 http://www.tripadvisor.com 42 http://www.travbuddy.com/ 43 http://www.dopplr.com/ 44 Please note that project FascinatE objectives fit in more than one clusters. nextMEDIA D2.3_FF_20110325 Page 35 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Availability of 3D Immersive Applications Everywhere and at Any Time: research for more efficient coding, streaming, broadcasting and visualization in different kind of terminals are required Real Time 3D Navigation and Physical and Emotional Involvement of the User: the user should be emotionally and physically involved, and his senses and emotions should be stimulated at a very high level by 3D immersive media. Towards this aim there is a need for applications able to support sensorial interaction and applications for virtual and augmented reality. From 2D to 3D Content: the transaction from 2D to 3D media should be as smooth and easy as possible. Thus, it's of great importance to identify how to transform already available 2D contents into their correspondent 3D version in an easy and possibly automated way. Innovative 3D Networks of People: 3D immersive media should be able to support a new form of social networking, as all the social relationships between people will not only pass through the use of ―nicknames‖ and ―pictures‖, but also through an avatar, a graphic representation. Innovative Organization and Distribution of the 3D Contents and Services: the challenge related to the passage from informational to experimental web will be that of organizing contents and information in a new, innovative and intuitive way, in order to revolutionize the query, retrieval, fruition and creation of information/data/AV products. Customized 3D Contents: the user should have the possibility to personalize and customize the contents and the 3D immersive experience by his/her own. This automatically leads to another important issue to be taken into account: the protection of intellectual property. Dynamic Services Provision: users as individuals or as members of communities will expect, and service providers must deliver, access to a range of different concurrent services, provided regardless of the user‘s location and adapting to a knowledge of the user‘s needs in terms of their context and of the capabilities of their accessing device. Trust and Security: new media Internet communities will thrive only if they are properly secured from the damage that could be caused by malicious attack or negligence and if the users perceive the new communities to safe. Dynamic Service Creation: in a network that links and coordinates things, people and media will need significant new research to help service providers build better services. Ideally it should be possible to dynamically construct services based on the ―on the fly‖ construction of networks of things people and media to a user in any location at any time.
nextMEDIA D2.3_FF_20110325 Page 36 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” 3 Market Assessment and Business Targets The growth of Internet has resulted in a significant impact, as it has become the day by day core communication environment of millions of people worldwide. The Future Internet shall be capable to meet new expectations and needs, by providing the means to: i) facilitate everyday life of people, communities and organizations, ii) allow the creation of any type of business regardless of their size, domain and technology, and iii) break the barriers/boundaries between information producer and information consumer. The latter will foster the emergence of prosumers: people/communities will be part of the creative flow of content and process, and not just consumers. Distributed knowledge can thus be shared easily and opinions can be made public in almost real-time. Current Internet is not able to exploit the possibilities that today‘s increasing bandwidth offers. Moreover, new technologies are constantly emerging in the 3D world, but they concern mainly animation material (scenes created by a computer) together with the well established modelling tools and languages. In this context, the nextMEDIA project aims to coordinate the Networked Media Unit projects in order to impact the Future Internet, and benefit from the new market opportunities. In order to offer a targeted approach, we assess in the following the sections the different market segments that most project target.
3.1 Delivery of multimedia content As already analysed the delivery of multimedia content is the main business target of many projects of the unit (mainly the Future Media Networks cluster). In this market direction we may highlight the following market potentials:
3.1.1 Networked A/V Media market potential There are already many factors that clearly show the great potential of A/V media/content market. First of all, the (user generated) content stored or streamed is growing exponentially. As it is shown in Figure 5 users are changing their content consumption patterns. Moreover, as shown in Figure 6, the percentage of users/subscribers generating personal content has reached 16% by the end of 2009.
Figure 5. Users Content consumption patterns[120] Figure 6. Grown of User Generated Content [124]
nextMEDIA D2.3_FF_20110325 Page 37 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Another important factor is the significant percentages of Internet users, who are now paying for online content experiences (Figure 7). Figure 8 depicts that users are paying for 25% of the videos that they download from the Internet. As more mainstream consumers begin to utilize Internet content services as complements to existing subscriptions (broadband, television, mobile phone, etc.), it is expected to see a growing number of users paying for these services as value-added services rather than stand- alone subscriptions. Also, the growing links between content and the platforms from which consumers will access and enjoy the content are also becoming stronger. Revenues for such online entertainment activities as gaming, music, and video – all of which can be enjoyed with a simple broadband connection and a home computer – will exceed $17 billion world-wide annually by the end of 2012 [125].
Figure 7. Internet Users Paying for Online Content [125] Figure 8. Free vs Purchased Internet Download [126]
Another added valued service is 3D video streaming, even over overlay networks. IPTV is growing rapidly (Figure 9). Only in Europe, the IPTV subscribers will grow from 0.9 millions in 2005 to 24 millions in 2011. Of the three regions, Europe and North America are generating a significant amount of this revenue; in Asia, though the number of subscribers is high, it is generating a lower amount of revenue due to lower average prices.
Figure 9. Growth of IPTV Subscribers (a) world wide (b) in Europe [127]
3.1.2 Home networking market potential Europe a very important value and is continuously increasing. The values of ADSL penetration in Europe range from 78.3 % in Malta down to a 27.8% in Portugal. The market is still developing and what is important to notice is that users that are connected to Internet are switching from narrowband to broadband in most of the cases. The next step is to provide home access for speeds of up to 50 Mbps, while many large telecom operators like France Telecom and Telefonica are already launching commercially the FTTH solution targeting the Home of 2010.
nextMEDIA D2.3_FF_20110325 Page 38 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Yet, in order not only to return the huge investment, but also turn FTTH solution profitable, new broadband services are required. One of these services is IPTV. As already shown in Figure 9 the IPTV subscribers are increasing heavily. Other market studies [128] show that among the telecom operators that offer IPTV service France Telecom ranks first (having 0,426 and 0,1017 millions of European subscribers in 2006 and 2007 respectively), followed by Telefónica (having 0,267 and 0,469 millions of European subscribers in 2006 and 2007 respectively). New really broadband services are required to make FTTH a real need. As it is shown in Figure 10, value-added service providers and telecom operator should move towards more innovative, bandwidth hungry services e.g. HD video and HD video Calls.
Figure 10. Value-added Services vs. bandwidth consumption and revenue generation [129] Apart from the access network and the value-added services, the in-home network represents another very promising market. As shown in Figure 12, the number and penetration of home networks in Europe and World-wide are increasing year by year.
Figure 11. Home Networking Growth wordwide [130] Nowadays, digital content is almost on every terminal or consumer equipment that can be found in home. In Spain, for example, the digital consumption of media measures in traffic has increased from 60,000 Terabytes in 2000 up to 1,000,000 Terabytes in the last few years. Therefore, users have now an increasing need of home networking nextMEDIA D2.3_FF_20110325 Page 39 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” solutions in order to share or stream their digital content among terminals and are willing to share this content with others in many formats.
Figure 12. Home Networking Growth in Europe [130] The Digital Home transformation (from security and domotic services up to advanced multimedia services with very high quality video images (HD)) is shown in Figure 13. The A/V consumer electronic devices (LCD TVs, Home Audio systems, Digital Set-top boxes, Digital Cameras) are among the top 8 revenue areas in the Digital Home, while the DTV chipsets shipment world-wide is another large market area (Figure 14). However, the economic crisis is expected to put barriers on the consumption of digital goods and telecommunications. An important pressure will be put into the commercial offerings for connectivity.
Figure 13. Top Revenue product areas by Application (2007-2012) [131]
nextMEDIA D2.3_FF_20110325 Page 40 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Figure 14. DTV chipsets shipments by region [132]
3.1.3 Content-Aware Distribution, Storage and DPI market potential ABI research foreseen (Figure 15) that media network device shipments will increase from less than 40 millions into 2005 to more than 350 million units in 2011, while the Compound Annual Growth Rate (CAGR) is expected between 44% to 48% during the 2003-2011 period.
Figure 15. Media Network Device Shipments [133] Content aware storage (CAS) is also one of the fastest growing areas of the storage market [120]. Companies are implementing CAS solutions to meet the needs of compliance regulations, litigation protection and to enable better IT efficiencies. CAS storage systems set up retention policies on objects (file + metadata) preventing anyone from editing or deleting them for a period of time. In some cases the period of time can be three years and in other cases it is forever. The method for creating retention periods is proprietary from system to system. This creates a major issue for companies that are implementing CAS systems - vendor lock-in. An easy process is needed to move data from one CAS system to another that guarantees and provides audit trails proving that objects have not been altered or changed and that the retention period is still intact. Today there are no tools that provide this capability and no one seems all that interested in providing one. CAS is rapidly emerging with over $1 billion dollars in revenue, which means a lot of companies are buying these products. nextMEDIA D2.3_FF_20110325 Page 41 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
In parallel, Deep Packet Inspection seams to be the only currently available technology capable to provide security of IP traffic at ever growing rates that has inherent traffic management capabilities. Recently massive growth in data processing power and new cyber threats have spurred the deployment of DPI technologies. As it is reported in [120] just the U.S. Government DPI market is projected to grow at a CAGR of 36% from 2010 to around US$ 1.8 Billion by 2015. The market will see a few good years of explosive growth in all segments before going flat.
Figure 16: Deep Packet Inspection: U.S. Government Market forecast 2010-2015 [120]
3.2 3D Media Internet Technologies The 3D Media Internet will depend on the availability of methods and systems that allow the ordinary and professional users to create, transmit, share and enjoy high- quality 3D media experiences. Moreover, the advancements on research and development of technologies related to the production, transmission and consumption of 3D Media is going through a major revolution. In fact there is a clear trend for equipment manufacturers to increase the refinement and expansion of their 3D business capacities. An example of this is clearly taking place within the movie industry where manufacturers of 3D cameras, dedicated 3D postproduction systems and other business actors in the production of 3D movies are working on a full line of 3D-based products aiming to provide technology and cost viability to make the new next big business revolution out of 3D Media. This example that is evident even to the great public after the breakthrough given by recent film ―Avatar‖ is developing as the new trend for all Media production at different levels such as broadcasters, new home and portable devices manufacturers and content producers among others. The assessment of 3D Media market will be only a matter of time after equipment manufacturers will make available the specific products they are currently working on for 3D acquisition, efficient encoding and decoding of 3D images, dedicated storage, high speed data handling, projection, rendering, content and special effects production. In particular, in the visual domain, navigable 3D environments may either be synthesised by CGI or created from ‗real-world‘ scenes by multi-camera capture, 3D scanning, or omnidirectional video. Synthetic CG is predominant for 3D on the Internet, via means including VRML, x3d, java3d, MPEG4, COLLADA, croquet, and canvas3dr. nextMEDIA D2.3_FF_20110325 Page 42 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Models created and displayed in such environments typically suffer from a highly cartoon-like look. Techniques for creating, animating, editing and interacting with synthetic 3D shapes and appearance are well understood: 3DStudio Max and Maya (and a host of plugins) are very widely used for developing computer games, creating animated movies, film and television visual effects, and visualising information. Recently, Microsoft released Photosynth to the public, allowing users to upload sets of still photographs for the synthesis of browsable 3D panoramas. Photosynth uses computation intensive point detection, feature matching and multi-view geometry algorithms: in effect, the service provides general public access (for Windows Vista and XP clients) to professional model extraction and surface mapping technologies. Apple‘s QuickTime VR and similar technologies based on panoramic photography offer a convincing sense of 3D albeit restricted to static scenes, and allowing very limited navigation possibilities. In the professional domain, dynamic 3D representations of people and objects can be captured by filming multiple view points, using synchronized video cameras. Canesta‘s 3D sensor technology is moving toward applications in games and gesture-controlled TV sets, and 3dvsystems has announced a prototype ‗Zcam‘ or 3D webcam for PC gaming or video-conference background replacement. For many Internet applications, it is not necessary for 3D content to be modified by the viewer, who only needs the freedom to ―look around‖ at will and ―move around‖ a predefined network of paths. In these cases, omnidirectional (immersive, panoramic) video offers a much simpler alternative to multi-camera capture or 3D CGI. The RenderWare system, developed by David Lau Kee, was an early and extremely successful games engine, built around a series of middleware components that could be selectively combined to create a custom engine with graphics, rendering and 3D capabilities. Most 3D engines and rendering systems are built on a graphics API such as Direct3D or OpenGL, which provides a software abstraction of the GPU or video card. Software renderers have been largely replaced by hardware-accelerated 3D graphics, but provide greater visual accuracy where real-time performance (frames-per-second) unimportant, and are essential when the hardware does not support shaders or Direct3D. As game engine technology matures and becomes more user-friendly, independent developers have created applications for devices ranging from the Xbox to the iPhone and there has been a growth in serious games for visualization, training and medical simulation. Millions of Internet users now create content, participate in and manage different forms of communities and activities where this content plays an important role. ―Web 2.0,‖ is characterised by participation and democracy in a world where users own and control the data97 and share their content through popular services such as YouTube and MySpace and interactive entertainment sites such as SecondLife. In 3D virtual community applications, such as Second Life, the focus is on social interaction rather than gaming and users can create their own spaces. Text chat is the predominant way for users communicate and most applications also allow 3D avatar representations to employ a range of predefined gestures. Sites such IMVU offer real-time 3d chat using doll-like but customisable avatars, which look crude but offer an engaging vicarious experience.
3.2.1 3D Media Delivery market potential With no doubt, video/audio market and broadband market are two key business factors to foster competitiveness and create new scales of economy. As shown in Figure 15, these technologies are expected to represent a huge potential market of up 649Trillion Yen (4900 Billion Euros) world-wide by 2020. nextMEDIA D2.3_FF_20110325 Page 43 of 90 Deliberadle D2.3
“Report on current research and business targets V3.0” Yen (in Trillions) (in Yen
Super-High 3D Video Supper-3D Multi-sensory Percept/Transmit Vision Technology surround Communications of the human Boardcasting sound affection & Industry technology sensibility Figure 17. Media delivery technologies market roadmap [100] The roadmap clearly demonstrates the complexity and devotion that these objectives require, while the huge market potential strongly suggests that Europe‘s ICT has to face them seriously in order to be able to compete in the shorter and longer term with the world-wide competition.
3.2.2 Virtual worlds market potential A great deal of emerging technologies, such as simulation rides, video conferencing, virtual reality, home theatre or high definition television are designed to provide end users with the experience of remote presence. Although traditional media such as telephone, radio or television offer also a certain little degree of remote presence, new technologies aim to develop this concept and make it more sophisticated, in a way that it makes effect in physiological and psychological aspects [145]. It is expected that the end user is going to take the main role, not only just act as an observer. This will bring more opportunities for the creation of new applications such as the remote presence in virtual worlds. A Virtual World is normally defined as being a computer-based persistent virtual environment, where users can interact, via their avatars, with each other and the environments. Virtual Worlds can be categorized in different ways: regarding technology used they can be 2D, 2.5D, 3D, regarding target age they can be for kids, tweens, teens or adults, regarding contents they can be realistic or cartoony, representing a real world or pure fantastic locations, regarding fruition they can be business driven, game based, social network like. The market of Virtual Worlds is growing bigger and bigger especially in the kids/tweens/teens market with 2D themed applications like Habbo Hotel (http://www.habbo.com/), Neopets (www.neopets.com), Club Penguin (www.clubpenguin.com), Barbie Girls (http://www.barbiegirls.com/home.html) and Gaia Online (www.gaiaonline.com) counting something like 200 Millions of registered users all together. Market data shows that there are now over 200 youth-oriented virtual worlds live, planned, or in active development [135]. In particular to the youth-oriented virtual, the target age the market can be broken down into virtual worlds targeting kids (7 and under), preteens (8-12), and teens (13+). The kids market is the leader. 107 worlds are aiming for at least part of their main audience to fall in the under-7 range. 90 offerings target tweens, while teens are still a relative open space with only 78 worlds targeting them.
nextMEDIA D2.3_FF_20110325 Page 44 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Figure 18. Virtual Worlds Compared to Other Online Communications Tools Compared to other online communications and social media platforms, virtual world has several technical advantages, including an immersive environment, presence, synchronous communications, high engagement and rich interactive features (Figure 18).
Figure 19. Total and monthly active users in 3D virtual worlds [136] As it is shown in Figure 19, the users of the 3D virtual worlds are expected to be drastically increased. However, both the adoption and active usage of virtual worlds lag behind other online applications. For instance, as shown in (Figure 20) only 13% of the adult broadband users in the U.K. have ever tried a virtual world, compared to 54% for social networks. In addition, only 23% of virtual‐world visitors visit on a weekly basis, compared to 59% for social networks
nextMEDIA D2.3_FF_20110325 Page 45 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Figure 20. Virtual Worlds vs. Social Networks (UK: Q3/08) [136] Moreover, Figure 21 clearly shows that the top 3 reasons for visiting a virtual world are related with social networking. People lack communication and meeting with friends (if any) is difficult. These metrics show that virtual world has yet to find the best intersection between entertainment, socialization, and community features. A lack of content that appeals to mass‐market consumers, suboptimal user interface and technologies, and the unbalanced gender ratio compared to social networks have all contributed to the low adoption rate and low activity ratio. One of the reasons that 2D/2.5D worlds have become very popular, is because many tweens rely on Internet tools to keep up with their friends after school.
Figure 21. Reasons for Visiting Virtual Worlds [137] Virtual worlds provide a good environment for them to socialize with existing friends and make new friends while concurrently engaging in entertaining activities like gaming and listening to music. Several other key factors contribute to the success of youth‐oriented virtual worlds. For these reasons, large media companies, toy companies, and other brands are investing serious money and efforts into the space. Media
nextMEDIA D2.3_FF_20110325 Page 46 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” companies are especially active; Viacom purchased Neopets in 2005 for $160 million, and Disney acquired Club Penguin for $700 million last year. Other popular worlds, including Gaia Online, Habbo Hotel, and Stardoll, are the subject of frequent acquisition rumours. The interest is justified. Not only are virtual worlds attracting eyeball hours and dollars from many kids and teens, they also increase the level of engagement with brands and content. For instance, in May 2008, Nickelodeon announced that, thanks to Neopets, its cluster of family and childoriented sites achieve an astonishing 80 minutes of time spent per visitor per month. This announcement received much attention from many other media companies.
3.2.3 Social Immersive Environments Social networking is one of the most relevant new business models which appeared on the Internet in the past years. Social media aims the grouping of individuals into specific groups inside a domain. Individuals are the most valuable part of these social networks. In this section we cover the social networks focused on new multimedia user-centric content services such as immersive media in social networks (e.g. user generated immersive environments). Participation of the users in new multimedia immersive services in social networks it is expected to grow in a near future. Users can participate in the immersive media services such as immersion in user virtual worlds by creating their user-generated Virtual Worlds and participate in new experiences in social virtual immersive networks (according to the European Commission studies the use of social networks has grown over the past year by 35% in Europe and is expected to more than double to 107.4 million users by 2012).
The following figure indicates the market reach in 16 European countries of social networks compared to the total Internet users per country in 2008.
Figure 22. Market reach of social networks in Europe 2008 (Source: comScore)
nextMEDIA D2.3_FF_20110325 Page 47 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
The social networking sites become a global phenomenon since the number of users who are members to them are spread through out the globe (¡Error! No se encuentra el origen de la referencia.). Moreover, the following facts prove the power of social networking on the Internet45: 1 Billion subscribers in Social Networking Websites World Wide 154 Million people accessing a Social Networking Website every day 1 Billion minutes spent on Social Networking every day 8 Billion pages accessed on Social Networking Websites every day
Figure 23. The world map of social networking
The current forms of online communication in the social networks are text-based, multimedia exchange-based and most advanced are avatar-based. The new research directions for immersive environments are the tele-immersive user- centric services. New collaborative services such as the creation 3D Immersive Augmented Worlds using the tools described in the section 2, enable users to join, interact, communicate and share time with each other participating in the social virtual augmented world network, and expanding their 3D immersive augmented world based on the user-group‘s wishes. The most relevant research directions identified are:
3D social networks equipped with novel immersive experiences, the collaboratively user generated environment; instead of joining a predefined world, the users themselves will recreate real places and will be able to continuously expand their 3D Immersive Augmented World according to their desires, thus allowing them to express themselves in a different way, realism; it will be achieved by replacing avatars with real reconstructions, and through more natural interaction, since users‘ emotions and expressions will actually get depicted.
3.2.4 Value added services and market potential
45 Joao Da Silva, CHORUS Event, Geneva, 10 October 2007 nextMEDIA D2.3_FF_20110325 Page 48 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Value-added services represent a huge market potential. Only in 2007, the U.S. broadband value-added services industry generated approximately $409 million in total revenue (Figure 24). Premium technical support and broadband entertainment services represented the largest categories, with $327 million and $78 million, respectively. The total service provider value-added services market is expected to grow to almost $1.3 billion in 2009.
Figure 24. US Broadband Value-added services revenues [138] 3.3 Networked Search The Internet was initially designed and primarily used by scientists for networking research and for exchanging information between each other. However, due to the explosion of the World Wide Web (which started as a document repository) and its successful descendants (Web 2.0), along with the dramatic increase of net-based audiovisual material (networked media) that has been produced by professional and most recently by amateur users, the Internet is rapidly transforming into a fully fledged virtual environment that facilitates services, interaction and communication.
1800
1600
1400
1200
1000
800
600
Digital Production (exabytes) Production Digital 400
200
0 2006 2007 2010 2011
Figure 25: Audiovisual Content Production (professional or amateur) Figure 25 depicts how digital content production has evolved. In 2006, digital content, produced by either professional or amateur users, reached the capacity of 160 exabytes, while it is expected that by 2010 it will reach the capacity of 990 exabytes, inducing a 6
nextMEDIA D2.3_FF_20110325 Page 49 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” fold increase [140]. In 2011, the amount of digital information produced in the year will be almost 10 times the content produced in 2006. However, the increasing availability of digital content poses, at the same time, an obstacle to its acquisition by users and dictates the need for a search engine to assist easy access and retrieval of relevant content. The use of search engines is adopted by the majority of users worldwide. It is worth to mention that 85% of Internet users use search engines to access desired content. Typing ―Google‖ is easier than remembering a specific website spelling, while its toolbar is by far the most dominant interface to today‘s WEB information. Today, there are thousands of search engines and directories worldwide, but only a handful of those produce the vast majority of all search engine traffic. As it is shown in Figure 26, Google has already captured the 71,49% of the market share in the years 2005-2008, rising even to the 77,34% in 2008.
Figure 26: Search Engine Market Share for Year 2005 to Year 2008 In order to fulfil the emerging need for multimedia content, the existing search engines have been enriched with media-search capabilities (e.g. ―Google Image‖, ―Google Sketchup‖, etc.). Search and retrieval of multimedia content, using the abovementioned search engines, is still limited to text-based queries. The user enters keywords or tight, concentrated phrases and retrieves media objects whose annotation matches the given keywords. Although this mechanism of text-based media retrieval is the most widely adopted, it is not the most efficient way to perform a media search. An ideal search engine should allow users perform search by using as query any type of input modality that best fits to their preferences. Towards the development of content-based audiovisual search engines, current European and regional initiatives have endeavoured to provide solutions for multimedia content retrieval using as input relevant media content. A detailed description of these initiatives is given in CHORUS Coordination Action (www.corus-ist.eu). These search engines, though, suffer from several limitations, which prevent them for being widely accepted by the majority of users. An example is that each engine is designed to deal with only one or few types of content. This forces users to switch from one engine to another when looking for different types of content simultaneously. Another drawback of the existing search engines is that they provide limited search capabilities for mobile devices. This problem is becoming even more critical, taking into account the growth of mobile Internet. The increase of mobile Internet access is depicted in the Table below. The number of mobile Internet users worldwide is expected to reach the 982 million users by 2011, which is three times bigger than it was in 2006 [139] Mobile Internet access is supported by the availability of innovative
nextMEDIA D2.3_FF_20110325 Page 50 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” devices that have recently come out on the market (like the iPhone in 2007), which are making mobile web browsing addictive. The search engines are also betting on mobile, with Yahoo! going so far as to say that there will be more mobile Internet users than Web users by 2017.
2006 2007 2008 2009 2010 2011 Mobile Internet Users 337.3 405.5 489.6 596.4 757.1 982.4 Mobile Search Users 266.0 327.2 410.7 516.8 672.3 901.1 Mobile Search Ad Revenues* $6.8 $63.1 $221.3 $580.3 $1148.9 $2361.5
Table 6. Worldwide Mobile Internet Users, Mobile Search Users and Mobile Search Advertising Revenues, 2006-2011 (millions)
Yet people are conducting mobile searches right now, with 266.0 million mobile search users in 2006. This is predicted to grow quickly to 901.1 million by 2011, with mobile search ad revenues soaring to $2.4 billion in 2011 from $6.8 million in 2006 [139]. Taking these facts into account, mobile search and retrieval should by no means be underestimated, since it is expected to have significant market impact the upcoming years. Towards the creation of a future audiovisual search engine, effort should be put into the design and implementation of specialised mechanisms and interfaces to address the limited presentation and interaction capabilities of the mobile devices. A competitive multimedia searching framework should have a really breakthrough philosophy, rather than simply an innovative technology. Given the enormous demand for personal and commercial content rich services provision, a unified audiovisual search framework is needed, not only as an information space where users can query data, but as a collaborative environment where a large number of users can at any time share, annotate, retrieve and visualize a large amount of data of any kind including text, image, audio, video and 3D media documents.
3.3.1 Social and Economic Impact There are already a number of factors that clearly show the great potential in marketing of searching frameworks of large audiovisual content. First of all, the IP content stored in home systems is growing exponentially. As it is shown in Figure 27, the percentage of users/subscribers having IP-content (personal or commercial available) is expected to reach 16% until 2009 [140].
16
14
12
10
8
6
4
2 Percentage of users/subscribers having IP-content Percentageusers/subscribers of having 0 2005 2006 2007 2008 2009 Figure 27: Grown of IP-based Content
nextMEDIA D2.3_FF_20110325 Page 51 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
The type of the stored content is also very important to define the future services and applications. As it is shown in Figure 28, digital pictures are available in the vast majority of home/personal storage devices, followed by music [142]. Yet, personal videos and movies even today result in a significant percentage (26% and 17% respectively).
90 84
80
70 59 60
50
36 40
26 30 17
20 Percentage of Internet Users (%) Users of Internet Percentage 10
0 Movies and TV Personal videos Video clips Music Digital pictures shows Figure 28: Digital Content on Home Computers Moreover, the user habits start to change. As shown in Figure 28, among the A/V applications, video games and music files play is going to be the majority applications [143]. However, play DVDs and watch TV event on PCs, capture a large percentage of PC usage. On the hand, significant percentages of Internet users are now paying for these online content experiences [144]. As more mainstream consumers begin to utilize Internet content services as complements to existing subscriptions (broadband, television, mobile phone, etc.), it is expect to see a growing number of users paying for these services as value-added services rather than stand-alone subscriptions. Also, the growing links between content and the platforms from which consumers will access and enjoy the content are also growing stronger. Revenues in the U.S. for such online entertainment activities as gaming, music, and video – all of which can be enjoyed with a simple broadband connection and a home computer – will exceed $11 billion annually by the end of 2010.
Play Video Games 57
Play Music Files (on PC) 51
Burn Create CDs or DVDs 46
Edit Digital Photos 45
Play DVDs (on PC) 29
Create Digital Photo Slide Shows 21
Watch TV 10
Edit Videos 8
Record TV shows 7
0 10 20 30 40 50 60 Percentage of Respondents using PC at least once per month for various activities
Figure 29: Computer Usage Habits nextMEDIA D2.3_FF_20110325 Page 52 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
It is also worth to note that the mobile communication users‘ habits are changing. Since 2005, nearly 90% of mobile phones have been camera capable models (Figure 29). Service providers have been offering attractive prices to entice their customers to purchase a camera phone, as camera phones provides them an opportunity to sell services associated with this advanced feature. Subscribers have also shown an increasing interest in these phones. With the introduction of mega-pixel camera phones, with video recording and playback capabilities, the demand has further increased and provides additional opportunities for operators to offer complimenting services.
3.4 Social Network Internet Services Social Networking Internet services are changing the way we communicate with others, entertain and actually live. Social Networking is one of the primary reasons that many people have become avid Internet users; people who until the emergence of social networks could not find interests in the web. This is a very robust indicator of what is really happening online. The Web 2.0 era passed leaving behind great strength to the end-users. Nowadays, users (also known as prosumers46), both produce and consume significant quantities of multimedia content. Moreover, theis behaviour when combined with Social Networking (i.e. communication between users through online communities) has formed a new Internet era where multimedia content sharing through Social Networking Sites (SNSs) is an everyday practice. More than 200 SNSs of worldwide impact are known today and this number is growing quickly. Many of the existing top web sites are either pure SNSs or offer some social networking capabilities47. Except for the well known ―first tier‖ social networks with hundreds of millions of users that span in the entire world, there are also many smaller social networking sites that are equally as popular within the more limited geographical scope of their membership, within a city, country or continent, for example. There are also many vertically oriented communities that gather users around a specific topic and thus, they have many dedicated members48. Facebook49 is ranked as one of the most visited sites in the world, with over than 500 million subscribed users to date. Moreover, Friendster50 is popular in Asia, Orkut51 in Brazil and Vkontakte.ru52 in Russia. On top of that, there are dozens of other purely social networks with vibrant communities, such as Vznet53 (~17 Mio users), Xing54 (8 Mio users), Badoo55 (>70 Mio users), Netlog56 (> 70 Mio users), Tuenti57 (8 Mio users), Barrabes58, Hyves59 (> 10 Mio users), Nasza Klasa60 (> 11 Mio users), LunarStorm61 (>
46 http://en.wikipedia.org/wiki/Prosumer 47 http://www.alexa.com (accessed Nov 2010) 48 http://www.research-write.com/2010/02/social-networking-by-the-numbers.html 49 http://www.facebook.com/ 50 http://www.friendster.com/ 51 http://www.orkut.com/ 52 http://www.vkontakte.ru/ 53 http://www.vznet.net/ 54 https://www.xing.com/ 55 http://badoo.com/ 56 http://netlog.com/ 57 http://www.tuenti.com/ 58 http://www.barrabes.com/home.asp 59 http://www.hyves.nl/ 60 http://nk.pl/ 61 http://www.lunarstorm.se/ nextMEDIA D2.3_FF_20110325 Page 53 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
1.2 Mio users), Zoo62 (~1 Mio users), Sapo63, DailyMotion64, VBOX765, iwiw66 and so on. There are also many vertically oriented communities that gather users around a specific topic, such as Last.fm67 for music or Goodreads68 for books. Finally, many mobile social networks appear to fill the gap and detach social networks from desktops. Some of them are aka-aki69, itsmy70, brightkite71 and mobiluck72, to name a few. The following figure presents a map73 of the social networks worldwide, thematically clustered and sized proportionally to the number of registered users.
Figure 30. Overview map of online social networks As this map shows, not all social networks are oriented to non-professional users. LinkedIn 74 with over 80 Mio users or Viadeo 75 with 30 Mio and Xing are mostly oriented in establishing professional connections between their users and initiate potential business collaborations. The rapid growth in popularity of social networks has enabled large numbers of users to communicate, create and share content, give and receive recommendations, and, at the same time, it opened new challenging problems. The unbounded growth of content and users pushes the Internet technologies to its limits and demands for new solutions. SNSs have the audience to claim their place in the primetime if they solve the challenges they face. Twitter‘s76 ―fail whale‖ (twitter‘s downtime icon) became a social media brand thanks to the frequent twitter‘s outages when (and not only) worldwide events take
62 http://www.zoo.gr/ 63 http://www.sapo.pt/ 64 http://www.dailymotion.com/ 65 http://vbox7.com/ 66 http://iwiw.hu/ 67 http://www.last.fm/ 68 http://www.goodreads.com/ 69 http://www.aka-aki.com/ 70 http://www.itsmy.com/ 71 http://www.brightkite.com/ 72 http://www.mobiluck.com/ 73 Credits: Chiqui Esteban, Miguel Jorge. ( Downloaded from http://www.iredes.es/mapa)/ 74 http://www.linkedin.com/ 75 http://www.viadeo.com/ 76 http://twitter.com/ nextMEDIA D2.3_FF_20110325 Page 54 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” place. Facebook gets severe criticism due to privacy concerns. Such challenges are present in all other SNSs to a greater or lesser extent.
The business perspectives of social networks is growing rapidly. There are now many different solutions in the market for ―Social Business Solutions‖ and large IT companies and consulting forms are starting to create new departments in this area. Businesses are slowly coming to the realisation that traditional communication methods are often less effective than social networking.
3.4.1 Social Businesses Intra company relations: The enterprise can be considered as a dedicated social network and all of the employees as members. In a social network the relationship between members is flat rather than hierarchal. This flat relationship between employees can foster collaboration, brainstorming and creativity. ―Social CRM‖: Just as with intra company relations, a more informal hierarchy can allow companies to work together in a more collaborative way. Blogging: Blogging can be a very direct, and interactive, way for companies to relate to their customers, clients, shareholders, providers, etc. It is much more immediate and personal way to transmit news and information and to get feedback directly from the recipient. Viral marketing (basically driven through social networks). In the last few years viral advertising has taken on a fundamental role in the marketing campaigns of many companies. An ad, new product or news item is directed to key persons who the company knows or suspects have an important social position on the net and especially within social networks. These persons pass on the ad or information to their contacts rapidly, if the key person has been chosen adequately it can soon reach thousands or even millions of people.
3.4.2 Social Analytic tools The technique of data analysis of the communications in social communications (often referred to as the ―social graph‖) is an extremely powerful tool for business. For example, analysis of relations and communications between members of a group or a community can show which are the most important ones (socially). This information can be vital when a viral marketing plan is being formulated or when a loyalty scheme is going to be implanted amongst a company‘s customers. A great market has been opened to both the clients of the data analysis and to the researchers and developers of new solutions for the data analysis. Argyle Social 77, Trackur78 and Radian679 are some of the examples of commercial tools which are used to analyse the data coming from social networks, profile the users and personalise the marketing strategies.
3.4.3 Social Television Social Television stands for technology that provides social interaction in the context of watching TV-programs or related to television content. It is a very active area of
77 www.argylesocial.com 78 www.trakur.com 79 www.radian6.com nextMEDIA D2.3_FF_20110325 Page 55 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” research and was named one of the 10 most important technologies by the MIT. Social Television is a fast growing market and multiple startups have recently appeared. Social TV has been around for more than 10 years already but not till the rise of social networks it has become feasible, since it already encourages constant connection between members of the network and the creation of likely minded groups. Today‘s 18- 28 years old access the internet more often than they watch television and a growing number among them is interested in having more social features integrated into their TV-experience. Furthermore 23 % of U.S. broadband users want to view online content from sites like YouTube or Flickr on their TVs. 80 Social television is connecting viewers with their friends, families and people with same interests giving them a space to discuss and exchange recommendations. But how should this interaction be provoked? Demanding from the user to sit in front of his/her TV-Set using mouse and keyboard for interaction does not sound like the best solution. With Sonys release of its Playstation Move Controller all three major gaming platforms (Sony Playstation, Microsoft XBOX and Nintendo Wii) are now using alternative haptical user interfaces Additional drivers to use the Nintendo‘s Wii controller on Mac, PC or even on a Smartphone already exist for years81, Microsoft‘s Kinect has been reported to be hacked just recently82 and an open source solution for using Sony‘s Move controller on PCs83 is on its way opening up an interesting research field of how to use these controllers as interaction devices for other purposes like controlling Social TV content.
3.4.4 Social Gaming Social Gaming is a term for games that are based on social interaction. By augmenting the game logic with social aspects players have to deal with each other in various ways to advance throughout the game. While social aspects have been part of massively multiplayer online role playing games (MMORPG) like World of Warcraft for quite a while already, the rise of Facebook and Co created the ground for a new field of games with social interaction as the main focus. 52% of the US adult population plays online games whereas 21% of this group is playing on a regular basis. 84 Just recently EA-Games announced the signing of a contract for five year partnership with facebook, producing online games for the world´s biggest social online platform using the new Facebook Credits system85. Playfish86 and Zynga87 are also sound examples of game development agencies that develop social games and use online social networks (Facebook, myspace, etc.) as their main distribution channels. While games in 2009 were often rough-hewn affairs, with a focus on often-spammy viral techniques, 2010 has seen rapidly improving production values with the market being taken over by international companies like Disney88 and games like Farmville are played by millions of people.
80 http://www.parksassociates.com/press/press_releases/2009/mar10-socialnetworking.html 81 http://www.ehow.com/how_2102972_control-computer-wii-remote.html 82 http://www.engadget.com/2010/11/10/open-source-kinect-camera-driver-now-available-for-download/ 83 http://code.google.com/p/moveonpc/ 84 http://pewinternet.org/~/media//Files/Reports/2010/PIP_Nielsen%20Apps%20Report.pdf 85 http://www.facebook.com/help/?page=1038 86 http://www.playfish.com/ 87 http://www.zynga.com/ 88 http://techcrunch.com/2010/07/27/playdom-acquired-by-disney-for-up-to-763-2-million/ nextMEDIA D2.3_FF_20110325 Page 56 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
For 2010 the sales volume of Social Gaming is expected to reach one billion US dollar, still a small number compared to the 19 billion dollar being awaited for the whole global online gaming market. But keeping in mind that last year´s sales volume only reached 500 million and in 2008 it only were a mere 100 million the tremendous speed of the growth of the social online gaming market becomes visible.89 But these numbers also make an underlying problem of the social gaming market visible. Big companies, with the expectations of gaining high profit, are pumping a lot of capital into this sector. Some analysts already fear that a bubble is being created that sooner or later is going to pop90. Research projects aiming at the Social gaming market should keep an eye on dissemination possibilities beside of the usual distribution channels (Facebook and Co.) as well.
89 http://www.gamesbrief.com/2010/06/the-online-games-market-was-worth-15-billion-in-2009-and-will- grow-to-20-billion-in-2010/ 90 http://techcrunch.com/2010/09/17/social-gaming-hi5/ nextMEDIA D2.3_FF_20110325 Page 57 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” 4 Results from the Concertation Meetings In order to collect, analyse and categorise the different scientific areas that are covered by current and evolving Network Media projects and clusters a number of questionnaires have been distributed during the Concertation Meetings. This chapter summarizes the major results.
4.1 Spring 2010 Concertation Questionnaire for Research Challenges In order to collect information about the running and new projects, a questionnaire was prepared after a number of PhC and a physical meeting of the nextMEDIA beneficiaries and distributed on the 3rd of February 2010, during the project presentation at the concertation plenary session. Overall, 18 projects returned the questionnaire completed. The responses have been very important for the success of this delivering. By collecting and analysing the responses we were able to understand better the current situation in Networked Media research area and try to coordinate the projects of the unit to overcome the various obstacles. In order to keeps each cluster informed about FIA activities that lay in the area of their interest, the nextMEDIA consortium has collected, analysed and categorised the different scientific and research areas that are covered by current and evolving Network Media clusters. From the questionnaires results, ―content-aware networking‖ is a very important area, which is partially tackled by the Future Media Networks cluster, but is also considered by the FCN and the MANA FIA groups, and the ―Search Engines‖ is another key area of the Future Internet, which is also considered by the FISO FIA group. Therefore apart from the plenary session presentation, a least one nextMEDIA delegate participated at each cluster in order to follow the discussions and collect the projects‘ research agendas and concerns. From the analysis of the results (Figure 31), we see that a large number of projects are in one way or another associated with the topic ―Architectures and technologies for converged and scalable networking‖ (10 projects), followed by the ―Architectures for massive content distribution‖ and the ―Content Streaming‖ topics (8 projects). ―Content Aware Networks‖, ―Optimised Quality of experience‖ and ―New Media‖ follow with 7 representing projects. As ―Expected project impact‖ from the projects (Figure 32), the most popular reply was ―Reinforced of industry in networking and delivery of multimedia content,‖ with 14 votes, followed by ―Wider market opportunities, including for content-related SMEs‖ with 10 votes and ―Reinforced of industry in multimedia content services‖ with 9 votes. All the results may be found in the following figures and tables.
nextMEDIA D2.3_FF_20110325 Page 58 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Research Challenges Votes
Architectures and technologies for converged and scalable … Streaming Content Architectures for massive content distribution New Media Optimised quality of experience Content aware networks Social Networking Enchancement of real time rendering of complex scenes Content aware routing Architectures for 3D prossesing Enchancement of 3D content representation Enhancement of 3D coding Enhancement of multi view point coding Optimised storage, caching, repositories Architectures for 3D augmented worlds Enhancement of 2D scalable video coding Applications beyond Games Enchancement of beyond HDTV coding and electronic cinema Content Recommendation Systems Optimised content searching, finding and retrieval Optimised searching based on contextual information Optimised searching based on user feedback Optimised filtering, aggregation
0 2 4 6 8 10 12
Figure 31: Research Challenges Votes on the Questionnaire
Expected Impact Votes
Reinforced of industry in networking and delivery of multimedia content
Wider market opportunities, including for content-related SMEs
Reinforced of industry in multimedia content services
Global standards and European IPRs reflecting federated and coherent roadmaps
Innovative business and societal applications (e.g. games, entertainment,…
Reinforced of industry in 3D media Internet technologies
Wider uptake of networked and collaborative platforms based on a '3D media …
Strengthened European industry in multimedia experiences beyond HDTV
Reinforced positioning of industry in Europe in networked search
Enhanced Quality of life for Citizens
0 2 4 6 8 10 12 14
Figure 32: Expected Impact Votes on the Questionnaire
The questionnaire and detailed results of Spring 2010 concertation meeting are available as Annex 1.
nextMEDIA D2.3_FF_20110325 Page 59 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
4.2 Fall 2010 Concertation Questionnaire for Standardisation In order to collect information about the will of the various projects to contribute to standards a new questionnaire was prepared and distributed on the 29th of November 2010, during the project presentation at the concertation plenary session. Overall, 24 questionnaires were returned completed. The responses show that there is a great fragmentation not only at the areas that various projects aim to contribute, but also to the standardization bodies that various projects target.
Multi view point coding Optimised searching Social Networking 2% 2% 2% Quality of experience Streaming Content 4% 14% Beyond HDTV /electronic cinema 4% 3D content representation 4% Content aware networks 3D coding 13% 4% Rendering of complex scenes 4%
Contextual based Searching 4% Architectures for massive Architectures for 3D content distribution augmented worlds 11% 4% Storage, caching, repositories 5% Content searching, finding & Content Filtering, aggregation Content aware routing retrieval 5% 5% 7% Architectures for converged networking 9% Figure 33: Research area of interest in Standardisation As it is shown in Figure 33, there are many research areas which projects are aiming to contribute to standards. Among them, Streaming Content has received the highest score (14%) followed by Content Aware Networks (13%), Architectures for Massive content distribution (11%), Architectures for Converged Networks (9%) and Content Searching and Retrieval (7%). More specific areas received lower marks (e.g. Content aware routing received just 5%). Moreover the projects interest towards various standardization bodies is also fragmented. As shown in Figure 34, IETF is the standardisation body that receives the interest of most projects (32%) followed by ETSI (21%) and MPEG (19%). This may be due to the fact that IETF is an international standardization body (while for instance ETSI is European) and there is no fee for been a member or submitting an Internet draft. Another reason may be that IETF is targeted to Internet protocols, while the other standardization bodies have a broader scope. nextMedia has started to analyse the European participation and contribution to IETF. As it may be seen in Annex II (Section 8.2), Cisco is by far dominating the RFCs (~290 authors). At the second position would be the combination of Alcatel, Lucent and Bell- labs (all together ~120), followed by IBM (~100), Microsoft (~80), BBN (~79) and Ericsson (~77). Moreover overall, US owns the majority of the accepted Request For
nextMEDIA D2.3_FF_20110325 Page 60 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Comments (RFC) documents (with 900), while Europe follows (with 720). However, if we consider that Europe started contributing after 1987, then we can see that Europe‘s contribution the last couple of years is really significant.
IEEE DVB ISO\IEC 3% 4% IETF ITU 3% 32% 9%
W3C 9%
MPEG 19% ETSI 21% Figure 34: Targeted Standardisation Bodies On the other hand, in order to further coordinate contributions to standardization bodies, we have grouped the areas of research which could lead to standardization activties and we have concluded to Figure 35. There it is shown that various architectural issues receive the highest interest (25%) followed by the Streaming Content & Content Aware Routing (21%) and Content Aware Networks & Caching (19%).
New Media Experience 9% Architectural Issues 3D coding, representation and 25% rendering 13%
Searching & Retrieval 13%
Streaming Content, Content Aware Routing 21% Content Aware Networks & Caching 19%
Figure 35: Grouping of research area of interest in Standardisation In order to coordinate the projects towards standardization, the nextMedia project has started specific pre-standardization activities. The FIArch Group, which is coordinated and leaded by nextMedia targets the Future Internet Architecture. So far, the group has created a document describing the Current Internet Fundamental Limitations and the Design Objectives of Future Internet [146]. The next steps will be the Design Principles of the Future Internet. With this activity, nextMedia mainly targets the Architectural Issues pre-standardization activities.
nextMEDIA D2.3_FF_20110325 Page 61 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
The Future Media Internet Architecture Think Tank (FMIA-TT), which is created, coordinated and leaded by nextMedia targets the Future media Internet Architecture and various Streaming Content & Content Aware Routing issues. So far, the group has created a white paper describing the Future Media Internet Reference Model [146]. The next steps will be to collect feedback from industrial and academic experts and proceed towards an open workshop, presenting the reference model to all interested projects of the Unit. In order to face the Content Aware Networks & Caching issues, nextMedia participated in the next IETF meeting in Prague in order to follow the discussion on the Content Delivery Networks Interconnection (CDNI) BOF meeting which may lead to a new IETF workgroup, where interested projects of the Unit may contribute. NextMEDIA became a part of the FISA standardisation support community, which groups seven coordination projects under FP7. The objective of this group is to define a shared action plan to support standardisation activities within the FIA and help FIA projects in their standardisation actions. Through the FISA support group, nextMEDIA will increase the efficiency of R&D projects standardisation activities by o Providing direction to existing pre-standardisation initiatives (draft version can be found in Annex III) o Creating transversal liaison between FIA communities and projects o Identifying and advertising existing and foreseen standardisation activities. o Encouraging cooperation and building synergies in FIA activities o Facilitating the dissemination of project results to wider community, particularly industry
nextMEDIA D2.3_FF_20110325 Page 62 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” 5 Conclusions This deliverable is the 3rd outcome of the WP2 ―Coordination of the activities of the cluster towards the FIA‖. The WP2 aims to collect, analyse and categorise the different scientific areas that are covered by current and evolving Network Media clusters in order to achieve their objectives in view of the Future Internet era. The deliverable provides a study of the research covered by the running and newly started projects of the unit, the relevant state-of the art and the business sectors they cover, in relation with the FIA activities. Taking into account also the market trends and potential business targets, we may define the roadmap shown in Figure 36. Of course there are interrelations between the different research challenges and advances in one area may pose advances in other. Moreover, there will always be the innovations and the early adopters, but in general we conclude that Future Media Internet in the way that we have defined here will be a reality by 2020-2022. 2015 2020 Content Centric Networking Content Awareness Location Awareness Network Awareness Content-Aware Routing Content-Aware Caching Content Delivery Content Encoding Content Adaptation Content Enrichment Content Personalisation 3D Content Creation 3D Capturing 3D Authoring 3D Coding 3D Rendering Content Searching Multimedia Indexing Distributed searching 3D Objects searching Annotation 3D User Interfaces Immersive Environments Virtual Worlds 3D Navigation
Figure 36: Networked Media Research Challenges related to FMI
nextMEDIA D2.3_FF_20110325 Page 63 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
As conclusions we may highlight the current research challenges related to the FMI from all the clusters in the following: Content Centric Networking, including content location awareness, content- aware caching and routing (expected in market by 2017-2022) Content Delivery, including encoding, in the network content adaptation, enrichment & personalisation to achieve better Quality of Experience (expected in market by 2016-2021) 3D Content Creation including capturing, authoring, compression, coding & Rendering (expected in market by 2019-2024) Content Searching & Retrieval including content-based large scale multimedia indexing, annotation and content-aware and (3D) object searching (expected in market by 2015-2020) 3D User Interfaces including immersive environments/applications, virtual world and 3D Navigation (expected in market by 2020-2025)
nextMEDIA D2.3_FF_20110325 Page 64 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” 6 References [1] Future Internet, ―The Cross-ETP Vision Document,‖ Version 1.0, 8/01/2009, http://www.future-internet.eu/fileadmin/documents/reports/Cross ETPs_FI_Vision_Document_v1_0.pdf [2] Future Internet X-ETP Group, ―Future Internet Strategic Research Agenda,‖ Version 1.1, January 2010, http://www.future-internet.eu/fileadmin/documents/ reports/FI-SRA-V1_1__final_clean_050210.pdf [3] ―Research on Future Media Internet‖, Future Media Internet Task Force, ftp://ftp.cordis.europa.eu/pub/fp7/ict/docs/netmedia/research-on-future-media- internet-2009-4072_en.pdf [4] Siena (Content-based Network) http://wwwserl.cs.colorado.edu/ carzanig/siena/ [5] Palo Alto Research Center content-centric network, http://www.parc.com/ research/ projects/networking/contentcentric/ [6] Open CDN project http://labtel.ing.uniroma1.it/opencdn/ [7] Oscar-Data-oriented Structured Overlay http://www.ntu.edu.sg/home/ anwitaman/ ResearchOscar.html [8] Sethuram Kodeswaran et al., "Content and Context Aware Networking (COCONET) Using Semantic Tagging", International Workshop on Semantics enabled Networks and Services, April 2006 [9] http://www.akamai.com/html/technology/edgeplatform.html [10] H. Schwarz, D. Marpe, T. Wiegand, ―Overview of the scalable video coding extension of the H.264 / AVC standard,‖ IEEE Transactions on Circuits and Systems for Video Technology, Vol. 17, Iss. 9, pp. 1103-1120, September 2007 [11] M. Mrak, N. Sprljan, T. Zgaljic, N. Ramzan, S. Wan and E. Izquierdo, Performance evidence of software proposal for Wavelet Video Coding Exploration group, ISO/IEC JTC1/SC29/WG11/ MPEG2006/M13146, 76th MPEG Meeting, Montreux, Switzerland, April (2006). [12] M. Wien, H. Schwarz and T. Oelbaum, ―Performance analysis of SVC,‖ IEEE Transactions on Circuits and Systems for Video Technology, Vol. 17, Iss. 9, pp. 1194 – 1203, September (2007). [13] N. Thomos, N. V. Boulgouris and M.G Strintzis, ―Wireless image transmission using turbo codes and optimal unequal error protection‖, IEEE Trans. Image Process. Vol. 14, no 11, pp. 1890 – 1901, Nov. (2005). [14] Hatem Boujema, Mimouna Ben Said and Mohamed Siala, ―Throughput Performance Of Arq And Harq I Schemes Over A Two-States Markov Channel Model‖, in Proc. of 12 international conference on Electronics, Circuits, and Systems, Dec.(2005).. [15] S. Wan, M. Mrak, N. Ramzan, and E. Izquierdo, "Perceptually adaptive joint deringing-deblocking filtering for scalable video transmission over wireless networks," ELSEVIER Journal of Signal Processing: Image Communication, vol. 22, iss. 3, (2007). [16] T. Zgaljic, N. Sprljan, and E. Izquierdo, "Bit-stream allocation methods for scalable video coding supporting wireless communications," Signal Processing: Image Communication, vol. 22, iss. 3, pp. 298–316, (2007).
nextMEDIA D2.3_FF_20110325 Page 65 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
[17] M. Mrak, T. Zgaljic, and E. Izquierdo, "Influence of downsampling filters characteristics on compression performance in scalable video coding," IET Image Processing, vol. 2, iss. 3, pp. 116–129, (2008). [18] Smolic, A., Kimata, H.: Report on 3dav exploration. ISO/IEC JTC1/SC29/WG11 Doc N5878 (2003) [19] Survey of algorithms used for multi-view video coding (mvc). ISO/IECJTC1/SC29/WG11 Doc N6909 (2005) [20] A. Vetro, P. Pandit, Kimata, H., Smolic, A.: Joint draft 8.0 on multiview video coding. Joint Video Team (JVT) of ISO/IEC MPEG ITU-T VCEG ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6 (2007) [21] C. Bilen, A. Aksay, Bozdagi Akar, G.: A multi-view video codec based on H.264. In: Proc. IEEE Conf. Image Proc. (ICIP), Oct. 8-11, Atlanta, USA (2006) [22] ISO/IEC JTC1/SC29/WG11, ISO/IEC 23002-3 Representation of auxiliary video and supplemental information (2007) [23] P. Merkle, Y. Wang, K. Mueller, A. Smolic, T. Wiegand: Video plus depth compression for mobile 3D services. In: Proc. IEEE 3DTV Conference, Potsdam, Germany. (2009) [24] L. Zhang, W. Tam: Stereoscopic image generation based on depth images for 3 D TV. IEEE transactions on Broadcasting 51 (2005) [25] E. Kurutepe, A. Aksay, C. Bilen, Gurler, C., Sikora, T., Akar, G., Tekalp, A.: A standards-based, flexible, end-to-end multi-view video streaming architecture. In: Packet Video 2007. (2007) [26] F. Pereira, J.R. Smith, A. Vetro, (eds.), "Special Section on MPEG-21", IEEE Transactions on Multimedia, vol. 7, no. 3, Jun. 2005 [27] Th. Zahariadis, C. Lamy-Bergot, Th. Schierl, K. Grüneberg, L. Celetto, C. Timmerer, ―Content Adaptation Issues in the Future Internet,‖ IOS press ―Towards the Future Internet - A European Research Perspective,‖ 2009, ISBN 978-1-60750-007-0 pp. 283-292. [28] Th. Zahariadis, O. Negru, F. Rovati, F. Álvarez, ―Seamless Content Delivery over the Future Internet‖ TEMU 2008, 16-18 July 2008 [29] S. –Y. Hu, T. – H. Huang, S. – Chen, Chang, W. – L. Sung, J. – R. Jiang, B. – Y. Chen, ―FLoD: A Framework for Peer to Peer 3D Streamin‖, 2008. [30] N. Magharei, R. Rejaie, ―Prime: Peer-to-Peer receiver – driven mesh- based streaming‖, in Proc. INFOCOM, 2007, pp. 1415-1423. [31] S. – Y. Hu, ―A case for 3D streaming on peer-to-peer networks‖, in Proc. Web 3D, 2006, p. 57 – 63. [32] J. Sahm, I. Soetebier, H. Birthelmer, ―Efficient representation and streaming of 3D Scenes‖, C & G, vol. 28, No. 1, pp. 15-24, 2004. [33] J. Royan, P. Gioia, R. Cavagna, C. Bouville, ―Network – based Visualisation of 3D landscapes and city models‖, IEEE CG & A, Vol. 27, no. 6, p. 70-79, 2007. [34] G. Hesina, D. Schmalstieg, ―A network architecture for remote rendering‖, in Proc. DIS-RT, p. 88, 1998. [35] Alexander Bogomjakov, Craig Gotsman, ―Universal Rendering Sequences for Transparent Vertex Caching of Progressive Meshes‖, 2006. [36] X. Xiang, M. Held, J. Mitchell, ―Fast and effective stripification of polygonal surface models‖, In Symposium on Interactive 3D Graphics Proceedings, ACM, 1999, pp. 71-78 nextMEDIA D2.3_FF_20110325 Page 66 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
[37] J. Stewart, ―Triangle strips for continous level of detail meshes‖, Graphics Interface Proceedings 2001. [38] M. Deering, Geometry compression, Computer Graphics (SIGGRAPH‘95) pp. 13 – 20 [39] R. Bar-Yehuda, C. Gotsman, ―Time/space tradeoffs for polygon mesh rendering‖, ACM Transaction on Graphics, Vol. 15, No. 2, pp. 141-152. 2007 [40] G. Lin, T. P. Yu, ―A Non-recursive algorithm for minimum-time rendering of meshes with arbitrary genus‖, Preprint RPI 2001. [41] J. Chim, R. W. H. Lau, H. V. Leong, A. Si, ―CyberWalk: A Web based Distributed Virtual Walkthrough Environment‖, IEEE Trans. On Multimedia, Vol. 5, No. 4, pp. 503 – 515, December 2003. [42] A Si, H. V. Leong, ―Adaptive caching and refreshing in mobile databases‖, Personal Technol., vol. 1, no. 3. Pp. 156 – 170, Sept. 1997. [43] B. Y. L. Chan, H. V. Leong, A. Si, K. F. Wong, ―MODEC: A multigranularity mobile object – oriented database caching mechanism prototype and performance‖, J. Distrib, Parallel Databases, vol. 7, no. 3, pp. 343-372, July 1999. [44] D. A. Bowman, E. Kruijff, J. J. LaViola, and I. Poupyrev. 3D User Interfaces: Theory and Practice. Addison Wesley Longman Publishing Co., Inc., Redwood City, CA, USA, 2004. [45] P. Figueroa, R. Dachselt, and I. Lindt. A conceptual model and specification language for mixed reality interface components. In VR ‘06: Proceedings of the Workshop‖Specification of Mixed Reality User Interfaces: Approaches, Languages, Standardization‖, pages 4–11, Washington, DC, USA, 2006. IEEE Computer Society. [46] R. M. Taylor, T. C. Hudson, A. Seeger, H.Weber, J. Juliano, and A. T. Helser. VRPN: a device-independent, network-transparent VR peripheral system. In VRST ‘01: Proceedings of the ACM symposium on Virtual reality software and technology, pages 55–61, New York, NY, USA, 2001. ACM Press. [47] G. Reitmayr and D. Schmalstieg. An open software architecture for virtual reality interaction. In VRST ‘01: Proceedings of the ACM symposium on Virtual reality software and technology, pages 47–54, New York, NY, USA, 2001. ACM Press. [48] Agarawala, A. and Balakrishnan, R. Keepin‘ it real: Pushing the desktop metaphor with hysics, piles and the pen. In Proc. CHI, pp. 1283–1292. ACM Press, 2006 [49] Dragicevic, P. Combining crossing-based and paper-based interaction paradigms for dragging and dropping between overlapping windows. In Proc. UIST, pp. 193–196. ACM Press, 2004. [50] Stahl, O., Wallberg, A., Soderberg, J., Humble, J., Fahl´en, L. E., Bullock, A., and Lundberg, J. Information exploration using the pond. In Proc. CVE, pp. 72– 79. ACM Press, 2002. [51] Matsushita, M., Iida, M., Ohguro, T., Shirai, Y., Kakehi, Y., and Naemura, T. Lumisight table: a face-to-face collaboration support system that optimizes direction of projected information to each stakeholder. In Proc. CSCW, pp. 274– 283. ACM Press, 2004.
nextMEDIA D2.3_FF_20110325 Page 67 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
[52] Nakashima, K., Machida, T., Kiyokawa, K., and Takemura, H. A 2D-3D integrated environment for cooperative work. In Proc. VRST, pp. 16–22. ACM Press, 2005. [53] Hancock, M., Carpendale, S., and Cockburn, A. 2007. Shallow-depth 3d interaction: design and evaluation of one-, two- and three-touch techniques. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA, April 28 - May 03, 2007). [54] http://www.exitreality.com/index.html [55] http://secondlife.com/ [56] Stuerzlinger, W., Dadgari, D., Oh, J-Y. (2006, April). Reality-Based Object Movement Techniques for 3D. CHI 2006 Workshop: "What is the Next Generation of Human-Computer Interaction?". Workshop presentation. [57] E. Lalor et al., ―Steady-State VEP-Based Brain Computer Interface Control in an Immersive 3-D Gaming Environment,‖ EURASIP J. Applied Signal Processing, vol. 19, 2005, pp. 3156-3164. [58] R. Leeb et al., ―Self-Paced (Asynchronous) BCI Control of a Wheelchair in Virtual Environments: A Case Study with a Tetraplegic,‖ Computational Intelligence and Neuroscience, special issue: ―Brain-Computer Interfaces: Towards Practical Implementations and Potential Applications,‖ 2007, pp. 1-8. [59] Niklas Elmqvist, M. Eduard Tudoreanu and Philippas Tsigas. Evaluating Motion Constraints for 3D Wayfinding in Immersive and Desktop Virtual Environments. Proceedings of CHI 2008. April 2008. [60] Moffat Mathews, Madan Challa, Cheng-Tse Chu, Gu Jian, Hartmut Seichter, Raphael Grasset. Evaluation of Spatial Abilities through Tabletop AR. Proceedings of CHINZ '07: June 2007 [61] Nguyen-Thong Dang, Monica Tavanti, Ivan Rankin, Matthew Cooper. A Comparison of Different Input Devices for a 3D Environment. Proceedings of the ECCE 2007 Conference, 28-31 August 2007, London, UK [62] David Wilfinger, Martin Murer, Michael Lankes, Manfred Tscheligi. Using a 3D game to study the perceived quality of lifting device controls. Proceedings of ACE 2009. October 2009. [63] Wen Qi, JeanBernard Martens. Tangible User Interfaces for 3D Clipping Plane Interaction with Volumetric Data : A Case Study. ICMI‘05, October 4–6, 2005, Trento, Italy. [64] Anders Henrysson, Joe Marshally, Mark Billinghurst. Experiments in 3D Interaction for Mobile Phone AR. GRAPHITE 2007, Perth, Western Australia, December 1–4, 2007. [65] Eelke Folmer, Bei Yuan, Dave Carr, Manjari Sapre. TextSL: A Command- Based Virtual World Interface for the Visually Impaired. ASSETS‘09, October 25-28, 2009, Pittsburgh, Pennsylvania, USA. [66] ITU-T Rec. H.264—ISO/IEC 14496-10 AVC, ‖Advanced Video Coding for Generic Audiovisual Services‖ (2009) [67] Wiegand, T., Sullivan, G., Bjontegaard, G., Luthra, A.: Overview of the H. 264/AVC video coding standard. IEEE Transactions on circuits and systems for video technology 13 (2003) [68] Wray Buntine, Open source search A data mining platform SIGIR Forum, 2005,39,306–315 nextMEDIA D2.3_FF_20110325 Page 68 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
[69] Sonia Bergamaschi, Christoph Quix, Matthias Jarke: The SEWASIE EU IST Project, SIG SEMIS Bulletin, Vol. 2, No. 1, Feb. 2005. [70] Sonia Bergamaschi, Pablo Fillottrani, Gionata Gelati: The SEWASIE multi- agent system, Third International Workshop on Agents and Peer-to-Peer Computing (AP2PC 2004), AAMAS´04, New York City, USA, July 2004. [71] D. Friedman et al., ―Navigating Virtual Reality by Thought: What Is It like?‖ Presence, vol. 16, no. 1, 2007, pp. 100-110. [72] http://www.quaero.org/modules/movie/scenes/home/index.php?FUSEBOX_LA NG=2 [73] M. Worring, and T. Gevers, 2001, Interactive retrieval of color images. Int. J. Image Graph. 1, 3, 387–414 [74] Emad Attalla and Pepe Siy, Robust shape similarity retrieval based on contour segmentation polygonal multiresolution and elastic matching, Pattern RecognitionVolume 38, Issue 12, December 2005, Pages 2229-2241. [75] J. Sivic, F. Schaffalitzky and A. Zisserman, Efficient object retrieval from videos, PhD Thesis, University of Oxford, 2006. [76] D. G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, in Int. Journal of Computer Vision, 60(2), 2004 [77] A. Joly, O. Buisson and C. Frélicot, Content-Based Copy Retrieval Using Distorsion-based Probabilistic Similarity Search, in IEEE Transactions on Multimedia, 2007 [78] I. Laptev and T. Lindeberg, Space-time Interest Points, in IEEE Int. Conf. on Computer Vision, 2003 [79] J. Law-To, L. Chen, A. Joly, I. Laptev, O. Buisson, V. Gouet-Brunet, N. Boujemaa and F. Stentiford, Video Copy Detection: a Comparative Study, in Proc. of the ACM International Conference on Image and Video Retrieval, 2007 [80] J. Law-To, O. Buisson, V. Gouet-Brunet and N. Boujemaa, Robust Voting Algorithm Based on Labels of Behavior for Video Copy Detection, in Proc. of the ACM Multimedia, 2006 [81] H.-P. Kriegel, P. Kroeger, Z. Mashael, M. Pfeifle, M. Poetke, and T. Seidl, ―Effective similarity search on voxelized cad objects‖, In Proc. of IEEE Eighth International Conference on Database Systems for Advanced Applications, Washington, DC, USA, 2003 [82] D. Vranic, ―3D model retrieval‖, Ph.D. dissertation, University of Leipzig, 2004. [83] P. Papadakis, I. Pratikakis, S. Perantonis, T. Theoharis, ―Efficient 3D shape matching and retrieval using a concrete radialized spherical projection representation‖, Pattern Recognition 40 (9), pp. 24372452, 2007. [84] P. Daras, D. Zarpalas, D. Tzovaras and M. G. Strintzis, ―Efficient 3D Model Search and Retrieval Using Generalized 3D Radon Transforms‖, IEEE Trans. on Multimedia, 2007 [85] D. Zarpalas, P. Daras, A. Axenopoulos, D. Tzovaras, M. G. Strintzis, ―3D Model Search and Retrieval Using the Spherical Trace Transform‖, EURASIP Journal on Advances in Signal Processing Volume 2007 (2007), Article ID 239110. [86] D. Zarpalas, P. Daras, D. Tzovaras and M. G. Strintzis, ―3D Content-based Search based on Fourier Transforms‖, 6th International Workshop on Image
nextMEDIA D2.3_FF_20110325 Page 69 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Analysis for Multimedia Interactive Services (WIAMIS 2005), Montreux, Switzerland, April 13-15, 2005 [87] A.Mademlis, P.Daras, A.Axenopoulos, D.Tzovaras, and M.G.Strintzis, ―Combining Topological and Geometrical Features for Global and Partial 3D Shape Retrieval‖, IEEE Transactions on Multimedia, Volume 10, Issue 5, pp. 819-831, 2008 [88] R. Ohbuchi, K. Osada, T. Furuya and T. Banno, ‖Salient local visual features for shape-based 3D model retrieval‖, In Proc. of the IEEE International Conference on Shape Modeling and Applications, (SMI 2008), pp. 93-102, 2008 [89] http://www.ist-chorus.org/deliverables.asp, Deliverable D2.1 [90] C. Henson, A. Sheth, P. Jain and J. Pschorr, ―Video on the Semantic Sensor Web‖, W3C Video on the Web Workshop, San Jose, California and Brussels, Belgium, 2007. [91] Advanced Fire Information System , http://divenos.meraka.csir.za/afis/afis.html [92] S. N. Patel and G. D. Abowd, The ContextCam: Automated Point of Capture Video Annotation. 6th International Conference on Ubiquitous Computing. In: Lecture Notes in Computer Science, Volume 3205, pp. 301-318. Springer, 2004 [93] M. Boots and A. Robin, ―Bringing the sensor web together‖, Geosciences n. 6, p. 46-53, Oct. 2007 [94] Erik Selberg and Oren Etzioni, Multi-service search and comparison using the MetaCrawler, In Proceedings of the 4th International World Wide Web Conference, 1995, 195 – 208 [95] Erik Selberg and Oren Etzioni, The metacrawler architecture for resource aggregation on the web, IEEE Expert, 1997, 12, 8 – 14 [96] C. Mic Bowman and Peter B. Danzig, The Harvest information discovery and access system, Computer Networks and ISDN Systems, 1994, 763 – 771 [97] Oren Etzioni and Daniel Weld, A softbot-based interface to the internet, Communications of the ACM, 1994, 37, 72 – 76 [98] Adele E. Howe and Daniel Dreilinger, SavvySearch: A meta-search engine that learns which search engines to query, AI Magazine, 1997 [99] H. Chen and Haiyan Fan and Michael Chau and Daniel Zeng, MetaSpider: meta- searching and categorization on the Web, Journal of the American Society for Information Science and Technology, 2001, 52, 1134 – 1147 [100] User Centric Media: Future and Challenges in European Research. Created by ―User Centric Media Cluster of FP6 projects‖. Coordinated by Networked Media Unit of the DG Information Society and Media of the European Commission. [101] K. Sugiyama, K.. Hatano, and M. Yoshikawa (2004). Adaptive Web search based on user profile constructed without any effort from user. In Proceedings of WWW ‗04, 675-684. [102] Middleton, S.E. De Roure, D.C. Shadbolt, N.R. 2001. Capturing Knowledge of User Preferences: ontologies on recommender systems. In Proceedings of the First International Conference on Knowledge Capture K-CAP 2001, Victoria, B.C. Canada [103] M. Pazzani, D. Billsus, Learning and Revising User Profiles: The Identification of Interesting Web Sites, Machine Learning, vol. 27, pp. 313-331, 1997.
nextMEDIA D2.3_FF_20110325 Page 70 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
[104] Nico Clever, Alexander Kirchner, Daniel Schray and Matthias Schulte. User- generated content, p.1-3. wi.uni-muenster.de [105] S. Casey, B. Kirman, and D. Rowland. The gopher game: a social, mobile, locative game with user generated content and peer review. In ACEt‘07, pp. 9– 16. ACM, 2007. [106] M. Billinghurst, H. Kato, and I. Poupyrev. The magicbook: Moving seamlessly between reality and virtuality. IEEE Computer Graphics and Applications, 21(3):6–8, May 2001. [107] Das, T. K., Singh, G., Mitchell, A., Kumar, P. S., McGhee, K. NetEffect: A Network Architecture for Large-Scale Multi-User Virtual Worlds. In Proc. of the ACM Symposium on Virtual Reality Software and Technology (VRST‘97), 157-164, 1997. [108] Calic, J. Daras, P. Achilleopoulos, N. Panebarco, M. Mayora, O. Stollenmayer, P. Williams, D. Pennick, T. Magnenat-Thalmann, N. Guerrero, C. Pelt, M. McGrath, T. Fuenmayor, E. Papaoulakis, N. Alvarez, F. Kalapanidas, E. Shani, A. Le Moine, J.-Y. I-Lab., Univ. of Surrey, Guildford, UK. User Centric Media of the Future Internet. The Second International Conference on Next Generation Mobile Applications, Services, and Technologies. Sept. 2008, pp. 433 – 438. [109] Oscar Mayora, Petros Daras, Marianna Panebarco, Nick Achilleopoulos, Peter Stollenmayer, Doug Williams, Federico Alvarez, Elias Kalapanidas, Nadia Magnenat-Thalmann, Carmen Guerrero, Michiel Pelt, Tim McGrath, Eugenia Fuenmayor, David Salama, Alex Shani, Jean-Yves Le Moine, User Centric Media in the Future Internet: Trends and Challenges ACM DIMEA conference. Athens, September 2008 [110] Luigi Lancieri and Nicolas Durand: ―Internet User Behavior: Compared Study of the Access Traces and Application to the Discovery of Communities‖. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems And Humans, Vol. 36, No. 1, January 2006, pp. 208-219. [111] Murata, T. ―Discovery of User Communities from Web Audience Measurement Data‖; Proceedings. IEEE/WIC/ACM International Conference on Web Intelligence, 2004. WI 2004. 20-24 Sept. 2004, pp. 673 – 676. [112] Murata, T.; Saito, K.; ―Extracting Keywords of Web Users' Interests and Visualizing their Routine Visits‖. 9th International Conference on Control, Automation, Robotics and Vision, 2006. ICARCV '06, 5-8 Dec. 2006. pp. 1 – 6. [113] Murata, T.; Saito, K ―Extracting Users' Interests from Web Log Data‖; IEEE/WIC/ACM International Conference on Web Intelligence, 2006. WI 2006. 18-22 Dec. 2006 pp. 343 – 346. [114] Carey, M.J.; Tattersall, G.D.; Lloyd-Thomas, H.; Russell, M.J ―Inferring identity from user behaviour‖ Carey, M.J.; Tattersall, G.D.; Lloyd-Thomas, H.; Russell, M.J.; Vision, Image and Signal Processing, IEE Proceedings - Volume 150, Issue 6, 15 Dec. 2003, pp. 383 – 388. [115] D. Boutemedjet, S.; Ziou ―A Graphical Model for Context-Aware Visual Content Recommendation‖, IEEE Transactions on Multimedia. Volume 10, Issue 1, Jan. 2008, pp. 52 – 62 [116] K. Yu, A. Schwaighofer, V. Tresp, X. Xu, and H.-P. Kriegel, ―Probabilistic memory-based collaborative filtering,‖ IEEE Transactions on Knowledge and Data Engineering, vol. 16, no. 1, pp. 56–69, Jan. 2004. nextMEDIA D2.3_FF_20110325 Page 71 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
[117] R. Jin, L. Si, C. Zhai, ―A study of mixture models for collaborative filtering,‖ The Journal of Information Retrieval, vol. 9, no. 3, 2006, pp. 357–382. [118] P. Melville, R. J. Mooney, and R. Nagarajan, ―Content-boosted collaborative filtering for improved recommendations‖. Proceedings of the 18th National Conference on Artificial Intelligence, 2002, pp. 187–192 [119] K. Enami, ―Study Group report on the economic impact of Universal Communications,‖ NEM Summit, St. Malo, Oct. 2008 [120] T. Asaro, ―Content Aware Storage -- Vendor Lock-in is Re-born,‖ http://blogs.computerworld.com/node/2173 [121] Market Research Media ―Deep Packet Inspection (DPI): U.S. Government Market Forecast 2010-2015″ http://www.marketresearchmedia.com/ 2010/02/17/ deep-packet-inspection-market/ [122] Medienstudie 2007, IBM Business Consulting Service [123] nextMedia, D2.2 ―Report on current research and business targets V2.0,‖ Dec 2010 [124] PARKS ASSOCIATES, ―Internet Based Video Services‖, 2008 [125] PARKS ASSOCIATES, ―Trends in Consumer Technology: Defining & Sizing the Market‖, 2007 [126] PARKS ASSOCIATES, ―Entertainment 2.0 in Europe‖, 2009 [127] Parks Associates, ―Company Financials‖, 2007 [128] Parks Associates, ―IPTV in Europe: TV in a Hyper-competitive Market,‖ 2007 [129] Parks Associates, ―Know Thy Customer: Customer Analytics to Drive Value- added Services,― June 2008 [130] Parks Associates, ―Europe: Home Network Update‖ 2008 [131] iSuppli, ―Next Gen. DVD Players,‖ Consumer Platforms, April 2008 [132] iSuppli, ―Next Gen. DTV Semiconductors,‖ Consumer Platforms, May 2008 [133] ABI Research, ―Home Media Devices‖, 2007 [134] K. Enami, ―Ultra-Realistic Communications Forum incremental R&D roadmap‖, Nem Summit, St. Malo, 2009 [135] http://www.virtualworldsmanagement.com/2009/youth-01-26-2009.html [136] Parks Associates, ―Virtual Worlds: The Internet of Avatars,‖ 2008 [137] Parks Associates, ―Electronics Gaming in the Digital Home: A survey of 1985 Internet gamers,‖ 2008 [138] Parks Associates, ―Customer Analysis to drive value-added services,‖ June 2008 [139] Steven Soranno: The 21st Century Office Is Green, Costing Less, 2009 eMarketer, July 2007 [140] IDC consultancy report,‖The expanding Digital Universe‖ March 2007 [141] PARKS ASSOCIATES, ―Internet Based Video Services‖, 2005 [142] PARKS ASSOCIATES, ―Profile of PC Usage, a survey of 2682 PC Users with Internet‖, 2007 [143] PARKS ASSOCIATES, ―Digital Entertainment in the Home‖, 2005 [144] PARKS ASSOCIATES, ―Trends in Consumer Technology: Defining & Sizing the Market‖, 2006 [145] Lombard, M., & Ditton, T. At the heart of it all: The concept of presence. Journal of commter-mediated communication. 1997. nextMEDIA D2.3_FF_20110325 Page 72 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
[146] FIArch Group, ―Fundamental Limitations of Current Internet and the path to Future Internet,‖ December 2010
nextMEDIA D2.3_FF_20110325 Page 73 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” 7 ANNEX I: Spring 2010 Concertation Meeting 7.1 Questionnaire
nextMEDIA D2.3_FF_20110325 Page 74 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
nextMEDIA D2.3_FF_20110325 Page 75 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
nextMEDIA D2.3_FF_20110325 Page 76 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
nextMEDIA D2.3_FF_20110325 Page 77 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
7.2 Detailed Results
Analytically the results are shown in the following table.
Fascinate Media 3D 2020 CNG OCEAN iNEM4U OMWeb DIOMEDES I-SEARCH SARACEN MEDIAPRO ENVISION TA2 3DPhone BEAMING COMET NAPA-WINE SEA COAST Summary Project Budget (please tick only one): Less than 3M€ 1 1 1 3 Between 3M and 6M€ 1 1 1 1 1 1 1 1 1 1 10 Between 6M and 9M€ 0 More than 9M€ 1 1 1 1 4
Research Areas (please tick all applicable) Architectures and technologies for converged and scalable networking 1 1 1 1 1 1 1 1 1 1 1 11 Architectures for massive content distribution 1 1 1 1 1 1 1 1 8 Architectures for 3D prossesing 1 1 1 1 1 5 Architectures for 3D augmented worlds 1 1 1 1 4 Content aware networks 1 1 1 1 1 1 1 7 Content aware routing 1 1 1 1 1 5 Streaming Content 1 1 1 1 1 1 1 1 8 Optimised storage, caching, repositories 1 1 1 1 4 Optimised filtering, aggregation 1 1 Optimised content searching, finding and retrieval 1 1 2 Content Recommendation Systems 1 1 2 Optimised searching based on user feedback 1 1 Optimised searching based on contextual information 1 1 Enhancement of 2D scalable video coding 1 1 1 3 Enhancement of multi view point coding 1 1 1 1 4 Enhancement of 3D coding 1 1 1 1 4 Enchancement of 3D content representation 1 1 1 1 4 Enchancement of real time rendering of complex scenes 1 1 1 1 1 5 Enchancement of beyond HDTV coding and electronic cinema 1 1 2 Optimised quality of experience 1 1 1 1 1 1 1 7 New Media 1 1 1 1 1 1 1 7 Social Networking 1 1 1 1 1 5 Applications beyond Games, Please name……….. 1 1 2 Enhance communication among groups of people separated in spaced time
Expected Impact (please tick all applicable): Reinforced of industry in networking and delivery of multimedia content 1 1 1 1 1 1 1 1 1 1 1 1 1 1 14 Reinforced of industry in multimedia content services 1 1 1 1 1 1 1 1 1 9 Reinforced of industry in 3D media Internet technologies 1 1 1 1 1 5 Reinforced positioning of industry in Europe in networked search 1 1 2 Strengthened European industry in multimedia experiences beyond HDTV 1 1 1 3 Wider uptake of networked and collaborative platforms based on a '3D media Internet' 1 1 1 3 Global standards and European IPRs reflecting federated and coherent roadmaps 1 1 1 1 1 1 1 7 Wider market opportunities, including for content-related SMEs 1 1 1 1 1 1 1 1 1 1 10 Innovative business and societal applications (e.g. games, entertainment, education,1 culture) 1 1 1 1 5 Other. Please specify…………….. 1 1 Enhanced Quality of life for Citizens
nextMEDIA D2.3_FF_20110325 Page 78 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” 8 ANNEX II: Fall 2010 Concertation Meeting 8.1 Questionnaire
nextMEDIA D2.3_FF_20110325 Page 79 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
nextMEDIA D2.3_FF_20110325 Page 80 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
8.1 Detailed Results Analytically the results are shown in the following tables.
# Research Area of Interest in Standardisation hits 1 Streaming Content 7 2 Content aware networks 6 3 Architectures for massive content distribution 5 4 Content searching, finding & retrieval 4 5 Architectures for converged networking 4 6 Content aware routing 3 7 Content Filtering, aggregation 3 8 Storage, caching, repositories 3 9 Architectures for 3D augmented worlds 2 10 Contextual based Searching 2 11 Rendering of complex scenes 2 12 3D coding 2 13 3D content representation 2 14 Beyond HDTV /electronic cinema 2 15 Quality of experience 2 16 Optimised searching 1 17 Social Networking 1 18 Multi view point coding 1
Table 7. Analysis of research interests for standardisation in Future Internet
Standardisation Bodies Number of actions IETF 24 MPEG 12 ETSI 11 W3C 5 ITU 5 ISO\IEC 2 IEEE 2 DVB 2
Table 8. Standardisation actions carried out by projects
nextMEDIA D2.3_FF_20110325 Page 81 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
8.2 IETF statistics Source www.ietf.org
Figure 37: Number of Internet draft authors per Company Cisco is by far dominating the RFCs (~290 authors). At the second position would be the combination of Alcatel, Lucent and Bell-labs (all together ~120), followed by IBM (~100), Microsoft (~80), BBN (~79) and Ericsson (~77).
Figure 38: Number of RFCs per continent
Figure 39: Number of RFCs (Considering EU as a single country)
nextMEDIA D2.3_FF_20110325 Page 82 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Figure 40: Number of recent RFCs (Considering EU as a single country)
Figure 41: Number of IETF drafts (Considering EU as a single country)
Figure 42: Comparison of continents over the Years
nextMEDIA D2.3_FF_20110325 Page 83 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0” 9 ANNEX II: FISA Standardisation Support Group 9.1 Existing pre-standardisation initiatives related to future internet
ETSI, European Telecommunications Standards Institute http://www.etsi.org Description: ETSI produces globally-applicable standards for Information and Communications Technologies (ICT), including fixed, mobile, radio, converged, broadcast and internet technologies. We are officially recognized by the European Union as a European Standards Organization. The high quality of our work and our open approach to standardization has helped us evolve into a European roots - global branches operation with a solid reputation for technical excellence. ETSI is a not-for- profit organization with more than 700 ETSI member organizations drawn from 62 countries across 5 continents world-wide. High quality and low time-to-market are our constant aims and we continually strive to collaborate with research bodies. We are active in vital complementary areas such as interoperability and we offer event services related to standardisation including forum hosting. Our international reputation is built on openness, discussion, consensus and direct input from our members Existing pre-standardisation support scheme Main working groups/areas ISGs (Industry Specification Groups) offer a very Related to FI/IoT: TC M2M, quick and easy alternative to the creation of industry TISPAN, ERM, ITS, MCD, fora, and are focused on a very specific activity. Yet MTS, CLOUD, EHEALTH, they do have their own membership, which may SCP, ISG AFI and 3GPP. consist of both ETSI Members and Non-members http://portal.etsi.org (under certain conditions), they have their own Aeronautical, Broadband voting rules, they decide their own work Wireless Access, Broadcast, programme, and approve their own deliverables. DECT, Digital Mobile Radio, EMC, Emergency, Fixed-line Existing ISGs Access, Environmental Aspects, Open Radio equipment Interface (ORI) Grid and cloud computing, Autonomic network engineering for the self- Human Factors, Intelligent managing Future Internet (AFI) Transport, M2M, Maritime, Mobile Thin Client Computing (MTC) Media Content Distribution, Identity management for Network Services Medical, Mobile, Next (INS) Generation Networks, OSA, Measurement Ontology for IP traffic (MOI) OSS, Powerline, Protocol Quantum Key Distribution (QKD) Specification, Quality of Service, Quantum Key Distribution, Radio, Regulation & Legislation, Safety, Satellite, Security, Smart Cards, Testing, TETRA
Open Grid Forum
nextMEDIA D2.3_FF_20110325 Page 84 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Description The Open Grid Forum (OGF) is a community of users, developers, and vendors leading the global standardization effort for grid computing. The work of OGF is carried out through community-initiated working groups, which develop standards and specifications in cooperation with other leading standards organizations, software vendors, and users. More information on http://www.ogf.org Existing pre-standardisation support Main working groups/areas scheme eResearch Function Standards are developed within Working Grid Interoperation Now Community Groups. There are normally two or three co- Group, chairs of any given working group; working Remote Instrumentation Services in groups are categorized into Areas. Areas allow Grid Environment, Grid Information OGF to group together activities that are Retrieval, related to each other. Currently the OGF has Preservation Environments the following areas: Applications, Enterprise Function Architecture, Compute, Data, Infrastructure, Enterprise Grids Requirements, Management and Security. In the OGF Storage Networking Community document process the standards council plays Group, Telecomm Community Group the role of final reviewer of all documents, Standards Function making sure that the documents produced by The Standards function of OGF is working groups fulfill the requirements of the responsible for development of organization to become part of the OGF architectures, specifications, roadmaps document series. and glossaries for distributed After one year of being a Proposed computing software through OGF Recommendation, a working group can working groups and through the publish a document describing the various application of the OGF document implementations of a specification and, more process. The Standards function importantly, their interoperability or includes the management of technical conformance with the specification. Once this liaisons with other Standards has happened, a Proposed Recommendation is Development Organizations (SDOs) promoted to a full Recommendation and to promote the adoption of document. standardized and interoperable distributed computing software.
SNIA
Description Incorporated in December 1997, the SNIA is a registered non-profit trade association. The members are dedicated to developing and promoting standards, technologies, and educational services to empower organizations in the management of information. The nextMEDIA D2.3_FF_20110325 Page 85 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
SNIA works toward this goal by forming and sponsoring Technical Work Groups (TWGs), producing (with our strategic partner Computerworld) the Storage Networking World Conference series, building and maintaining a vendor neutral Technology Center in Colorado Springs, and promoting activities that expand the breadth and quality of the storage and information management market
Existing pre-standardisation support scheme Main working groups/areas SNIA standards are primarily related to data, SNIA Forums and Initiatives storage, and information management and primarily focus on technology address such challenges as interoperability, promotion and technical marketing usability, and complexity. They start out in activities. Forums do not develop SNIA-sponsored Technical Working Groups technologies or specifications, (member collaboration efforts centering on best though Initiatives can. SNIA's practices) or as contributions from independent Forums and Initiatives include the vendor collaborations, and are subject to a Cloud Storage Initiative, the Data rigorous processes involving checks and Protection and Capacity technology reviews by SNIA‘s Members, Optimization Committee, the Technical Council, and Board of Directors, Ethernet Storage Forum, the Green conducted under provisions that respect and Storage Initiative, Storage protect intellectual property and address Management Initiative, Solid State licensing of essential claims. Storage Initiative, the Storage A SNIA standard that has completed these Security Industry Forum and the processes is labeled a SNIA Technical Position, XAM Initiative and which signifies that the SNIA endorses and recommends the ideas, methodologies, and technologies described. The SNIA may take SNIA Technical Positions to ANSI, ISO, and other national and international standards bodies for additional endorsement.
WholeSale Application Community Description
WAC’s objective is to commercialize products for its member companies. Open Web standards are utilized in support of this commercialization effort as long as such adoption does not impact required time to market. The WAC widget specification is therefore based on W3C and OMTP standards to the greatest extent possible, given that these standards were developed in parallel to the WAC’s specifications. Existing pre-standardisation support Main working groups/areas scheme List here the relevant working groups or Add here a description of pre- working areas standardisation
nextMEDIA D2.3_FF_20110325 Page 86 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
World Wide Web Consortium
Description
The W3C mission is to lead the World Wide Web to its full potential by developing protocols and guidelines that ensure the long-term growth of the Web. Below we discuss important aspects of this mission, all of which further W3C's vision of One Web. Existing pre-standardisation support Main working groups/areas scheme List here the relevant working groups or Add here a description of pre-standardisation working areas
The 3rd Generation Partnership Project (3GPP
Description
The original scope of 3GPP was to produce Technical Specifications and Technical Reports for a 3G Mobile System based on evolved GSM core networks and the radio access technologies that they support (i.e., Universal Terrestrial Radio Access (UTRA) both Frequency Division Duplex (FDD) and Time Division Duplex (TDD) modes).
Existing pre-standardisation support Main working groups/areas scheme List here the relevant working groups or Add here a description of pre- working areas standardisation
Open Mobile Alliance
Description The mission of the Open Mobile Alliance is to facilitate global user adoption of mobile data services by specifying market driven mobile service enablers that ensure service interoperability across devices, geographies, service providers, operators, and networks while allowing businesses to compete through innovation and differentiation. Existing pre-standardisation support Main working groups/areas scheme List here the relevant working groups or Add here a description of pre- working areas standardisation
nextMEDIA D2.3_FF_20110325 Page 87 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
Linux Foundation
Description LiMo Foundation is an industry consortium dedicated to creating the first truly open, hardware-independent, Linux-based operating system for mobile devices. Backing from major industry leaders puts LiMo at the Heart of the Mobile Industry and makes LiMo the unifying force in Mobile Linux. The mission of the LiMo Foundation is to create an open, Linux-based software platform for use by the whole global industry to produce mobile devices through a balanced and transparent contribution process enabling a rich ecosystem of differentiated products, applications, and services from device manufacturers, operators, ISVs and integrators.
Existing pre-standardisation support scheme Main working groups/areas Android List here the relevant working groups or working areas
Symbian foundation
Description
The Symbian Foundation provides, manages and unifies the platform for download and development. With the Symbian platform source code published under the Eclipse Public License the entire platform is available to all for free, bringing additional innovation and more frequent and widely-sourced code and feature contributions; and engaging an even broader community in future development.
Existing pre-standardisation support Main working groups/areas scheme List here the relevant working groups or Add here a description of pre- working areas standardisation
IETF, The Internet Engineering Task Force http://www.ietf.org
Description: The Internet Engineering Task Force (IETF) is a large open international community of network designers, operators, vendors, and researchers concerned with the evolution of the Internet architecture and the smooth operation of the Internet. It is open to any interested individual. The actual technical work of the IETF is done in its working groups, which are organized by topic into several areas (e.g., routing, transport, security, etc.). Much of the work is handled via mailing lists. The IETF holds meetings three times per year.
The IETF working groups are grouped into areas, and managed by Area Directors, or nextMEDIA D2.3_FF_20110325 Page 88 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
ADs. The ADs are members of the Internet Engineering Steering Group (IESG). Providing architectural oversight is the Internet Architecture Board, (IAB). The IAB also adjudicates appeals when someone complains that the IESG has failed. The IAB and IESG are chartered by the Internet Society (ISOC) for these purposes. The General Area Director also serves as the chair of the IESG and of the IETF, and is an ex-officio member of the IAB. Existing pre-standardisation support scheme Main working groups/areas IETF Working Groups (WGs) are the primary 6lowpan - IPv6 over Low mechanism for development of IETF specifications power WPAN and guidelines, many of which are intended to be 6man - IPv6 Maintenance standards or recommendations. Working Groups are Ancp - Access Node Control typically created to address a specific problem or to Protocol produce one or more specific deliverables (a Autoconf - Ad-Hoc Network guideline, standards specification, etc.). Working Autoconfiguration Groups are generally expected to be short-lived in Csi - Cga & Send nature. Upon completion of its goals and maIntenance achievement of its objectives, the Working Group is Dhc - Dynamic Host terminated. Each Working Group has a charter. Configuration WG charters state the scope of work for group, and Dnsext - DNS Extensions lay out goals and milestones that show how this Hip - Host Identity Protocol work will be completed. More information about formation of IETF working groups can be found at Intarea - Internet Area http://www.ietf.org/wg/ Working Group Ipdvb - IP over DVB l2tpext - Layer Two Tunneling Protocol Extensions lisp - Locator/ID Separation Protocol mext - Mobility EXTensions for IPv6 mif - Multiple Interfaces mip4 - Mobility for IPv4 mipshop - Mobility for IP: Performance, Signaling and Handoff Optimization multimob - Multicast Mobility netext - Network-Based Mobility Extensions netlmm - Network-based Localized Mobility Management ntp - Network Time Protocol pppext - Point-to-Point Protocol Extensions savi - Source Address Validation Improvements shim6 - Site Multihoming by IPv6 Intermediation nextMEDIA D2.3_FF_20110325 Page 89 of 90 Deliberadle D2.3 “Report on current research and business targets V3.0”
softwire - Softwires tictoc - Timing over IP Connection and Transfer of Clock trill - Transparent Interconnection of Lots of Links
IRTF, The Internet Research Task Force http://www.irtf.org/
Description: The Internet Research Task Force (IRTF) is a sister group to the Internet Engineering Task Force (IETF). Its stated mission is ―To promote research of importance to the evolution of the future Internet by creating focused, long-term and small Research Groups working on topics related to Internet protocols, applications, architecture and technology‖. Existing pre-standardisation support scheme Main working groups/areas The IRTF Research Groups work on topics related Anti-Spam Research Group to Internet protocols, applications, architecture and (ASRG) technology. Research Groups are expected to have Crypto Forum Research the stable long term (with respect to the lifetime of Group the Research Group) membership needed to Delay-Tolerant Networking promote the development of research collaboration Research Group (DTNRG) and teamwork in exploring research issues. Host Identity Protocol (HIP) Participation is by individual contributors, rather Research Group than by representatives of organizations. Internet Congestion Control Research Group The IRTF is managed by the IRTF Chair in IP Mobility Optimizations consultation with the Internet Research Steering (Mob Opts) Research Group Group (IRSG). A Research Group may be Network Management established at the initiative of an individual or group Research Group Charter of individuals. Anyone interested in creating an (NMRG) IRTF Research Group must submit a charter for the Peer-to-Peer Research Group proposed group to the IRTF Chair along with a list Public Key Next-Generation of proposed founding members. The charter will be Research Group reviewed by the IRSG and then forwarded to the Routing Research Group IAB for approval. Scalable Adaptive Multicast Research Group More information about formation of IRTF working Transport Modeling Research groups can be found at http://www.irtf.org/ Group Virtual Networks Research Group (VNRG)
nextMEDIA D2.3_FF_20110325 Page 90 of 90