UniversitÄat Ulm Institut furÄ Verteilte Systeme Multimedia Stream Adaptation Services Dissertation zur Erlangung des Doktorgrades Dr. rer. nat. der FakultÄat furÄ Ingenieurwissenschaften und Informatik der UniversitÄat Ulm vorgelegt von Andreas Schorr aus Ulm an der Donau Ulm, Oktober 2006 Multimedia Stream Adaptation Services BY Andreas Schorr A DISSERTATION SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DR. RER. NAT. AT UNIVERSITY OF ULM JAMES-FRANCK-RING, 89069 ULM, GERMANY OCTOBER, 2006 °c Copyright by Andreas Schorr, 2006 Amtierender Dekan: Prof. Dr. Helmuth Partsch 1. Gutachter: Prof. Dr. Franz J. Hauck 2. Gutachter: Prof. Dr. Michael Weber 3. Gutachter: Dr. Andreas J. Kassler (Assoc. Prof.) Tag der Kolloquiumsprufung:Ä noch nicht bekannt Abstract During the last decade, a clear trend towards all-over-IP multimedia communication has evolved. There exists a great variety of stationary and mobile devices which allow streaming of digital audio and video content over IP networks. But this variety of multimedia enabled devices comes along with a great variety of di®erent hardware and software capabilities and di®erent access network technologies (having heterogeneous properties like bandwidth or packet loss rate). Even if two applications support a common set of media formats, it is often impossible to use such a common format because of resource constraints imposed by the di®erent hardware capabilities and network types. Adaptation of media formats is thus required to allow communication between heterogeneous peers. A second problem is that although quality of service has been a very popular research topic for several years, we still have no end-to-end QoS support for users in the Internet. When QoS cannot be guaranteed, media adaptation is a means to cope with fluctuating resource availability. Performing adaptation at the source or at the sink of a media stream is in many cases the most e±cient solution, but in some cases it is impossible to adapt a media stream on one of the communicating end-terminals because of limited hardware and software capabilities of these devices. In such a case, a stream adaptation node in the network can perform the required adaptation operations. In the context of voice-over-IP, media gateways are used to connect the switched telephony network with the Internet. Such media gateways handle only audio streams and they o®er only static adaptation services, i.e., they do not adapt to changing network conditions, which may occur in networks without QoS guarantees. Furthermore, existing adaptation services support only a limited set of media formats and adaptation operations, and they are usually tightly coupled with a certain communication infrastruture. So it is not possible to ¯nd and request such services using arbitrary legacy applications. This thesis enhances the current state-of-the-art by developing a generic stream-adaptation service that supports a great variety of di®erent adaptation operations, automatically adapts to changing resource availability, and can be easily enhanced with new functionality. Since speci¯c service announcement mechanisms for adaptation services currently do not exist, a mechanism for describing stream-adaptation services is developed which enables the dis- covery of adaptation nodes suitable for speci¯c types of adaptation operations. Using this service description, adaptation services may either be discovered and used directly by end users who are explicitly requesting certain adaptation operations, or they may be used by network operators and service providers to serve the customers' needs transparently while optimising the total network load. In the last part of this thesis, mechanisms for providing novel heterogeneous broadcast streaming services are elaborated. Parts of this work were motivated by the research projects MASA-N (in cooperation with Siemens AG and NEC Europe), POBNET (in cooperation with Siemens AG), AKOM (funded by Deutsche Forschungsgemeinschaft) and the EU-funded DAIDALOS project. TABLE OF CONTENTS List of Figures iv List of Tables vi 1 Introduction 1 1.1 Background . 1 1.2 Problem Statement . 3 1.3 Proposed Solution . 3 1.4 Main Contributions of this Thesis . 5 1.5 Overview of this Thesis . 7 2 Adaptation of Multimedia Streams 9 2.1 De¯nitions . 9 2.2 Media Adaptation . 11 2.2.1 Sender-Driven and Receiver-Driven Adaptation . 11 2.2.2 Transcoding . 13 2.2.3 Media Filtering . 18 2.2.4 Digital Item Adaptation . 21 2.3 Network-Flow Adaptation . 26 2.3.1 Changing Protocols . 26 2.3.2 Changing the RTP Pro¯le . 27 2.3.3 Changing Error Correction Mechanisms . 27 2.3.4 Changing Rate and Congestion Control Mechanisms . 30 2.4 Summary . 31 3 Survey of the Current State of the Art 33 3.1 Adaptation of Multimedia Streams . 33 3.2 Media Gateways . 36 3.3 Service Discovery . 37 3.4 Heterogeneous Broadcast . 38 3.5 Summary . 40 4 A Multimedia-Stream Adaptation Service 41 4.1 Use Cases and Design Decisions . 42 4.2 Architecture and Functional Description . 44 4.2.1 Interworkig Between Subsystems . 45 4.2.2 The Manager Subsystems . 47 4.2.3 Architecture and Functional Description of the Media Manager . 49 4.3 Adaptation-Service Invocation . 57 4.3.1 Session Description . 58 4.3.2 Broker-Initiated Adaptation . 60 i 4.3.3 Terminal-Initiated Adaptation . 65 4.4 Implementation . 67 4.4.1 Usage of Operating-System Modules . 68 4.4.2 E±cient Processing . 68 4.4.3 Error Correction Modules . 70 4.4.4 RTP Payload Format for the Transmission of gBSD Metadata . 75 4.4.5 Dynamic Adaptation Features . 76 4.5 Business Models . 78 4.6 Security Aspects . 80 4.6.1 Privacy . 81 4.6.2 Property Rights . 85 4.7 Summary . 86 5 Performance Evaluation 89 5.1 Processing Time and Scalability . 89 5.1.1 Test Environment . 89 5.1.2 Measurement Results . 90 5.1.3 Comparison with Commercial Products . 99 5.2 Hardware-Supported Video Transcoding . 100 5.3 Application-Layer Error Correction Mechanisms . 102 5.3.1 Test Environment . 102 5.3.2 Measurement Results . 104 5.4 Summary . 107 6 Description of Adaptation Services 109 6.1 Description, registration and discovery of services . 109 6.2 Describing Stream-Adaptation Services . 112 6.2.1 Service Models . 112 6.2.2 Service Properties . 114 6.2.3 A Note on Media-Format Identi¯ers . 117 6.2.4 A Note on Delay, Jitter, Cost, and Quality Reduction . 119 6.2.5 Considering Status Information for Adaptation-Node Selection . 123 6.3 Describing Adaptation Services with SLP Service Templates . 124 6.3.1 SLP Service URL . 125 6.3.2 Limitations of SLP Templates . 125 6.3.3 An SLP-based Description of an Adaptation Service . 126 6.4 Describing Adaptation Services with RDF . 128 6.4.1 RDF Overview . 128 6.4.2 An RDF-based Description of Adaptation Services . 132 6.4.3 Extending the MSAS Schema . 139 6.4.4 Creation of RDF Service Descriptions and Registration via SLP . 139 6.5 Querying for Adaptation Services . 142 6.5.1 Formulating RDF Queries . 142 6.5.2 Transportation of RDF Queries . 149 6.6 Summary . 149 ii 7 Heterogeneous Broadcast Services 151 7.1 Broadcast Services Today . 151 7.2 Announcement of Broadcast Sessions . 153 7.3 Setup of heterogeneous broadcast sessions . 155 7.3.1 Session-Setup Signalling with SDS-based Service Announcement . 156 7.3.2 Session Signalling for SAP-based Service Announcement . 160 7.3.3 Encrypted Broadcast Streams . 161 7.4 Summary . 161 8 Conclusion 163 8.1 Achievements . 163 8.2 Outlook and Future Work . 165 8.3 Final Remarks . 167 Bibliography 169 A SDPng++ Example 181 B XSLT Stylesheet for gBSD Transformation 185 C SLP Service Templates 187 D RDF Schema for Adaptation Services 193 E Media-Format Identi¯ers 217 F Deutschsprachige Zusammenfassung 227 G Deutschsprachiger Lebenslauf 229 iii LIST OF FIGURES Figure Number Page 1.1 Adaptation on a Proxy Node . 4 1.2 Heterogeneous Multicast . 5 2.1 Video Encoder and Decoder . 14 2.2 Sequence of Video Frames . 15 2.3 Spatial Domain Cascaded Video Transcoder . 15 2.4 Fast Video Transcoder . 16 2.5 Optimized Spatial Resolution Video Transcoder . 17 2.6 Scalable Encoding of a Video Frame . 19 2.7 SNR Scalability Video Encoder . 19 2.8 Adaptation of Error Correction Mechanism . 28 2.9 Unequal Erasure Protection . 29 4.1 MSAN Architecture . 44 4.2 Facade Design Pattern Implemented by Service Manager . 45 4.3 MSANMessage Class . 46 4.4 Media Manager Architecture . ..