Remote Collaborative Real-Time Multimedia Experience Over The
Total Page:16
File Type:pdf, Size:1020Kb
Remote C ollaborative Real-Time Multimedia Experience over the Future Internet ROMEO Grant Agreement Number: 287896 D4.2 Report on streaming/broadcast techniques for 3D multi-view video and spatial audio ROMEO WP4 Page 1/50 Document description Name of document Report on streaming/broadcast techniques for 3D multi-view video and spatial audio Abstract This document provides a detailed description of the packetization schemes in ROMEO and specifies high level syntax elements of the media formats in order to perform efficient transport and synchronization of the 3D audio and multiview video streams. Adaptation mechanisms and error concealment methods are also proposed in the context of degraded network conditions. Document identifier D4.2 Document class Deliverable Version 1.0 Author(s) N.Tizon, D. Nicholson (VITEC) H. Weigold, H. Ibl, J. Lauterjung (R&S) K. Birkos, A. Kordelas, A. Lykourgiotis, I. Politis (UPAT) Xiyu Shi (MulSys) M.Laabs (IRT) E. Ekmekcioglu (UNIS) A. Akman, S. O. Pelvan, S. Çiftçi, E. Çimen Öztürk (TTA) QAT team D. Doyen (TEC) F. Pascual Blanco (TID) H. Marques (IT) Date of creation 24-Jul-2012 Date of last modification 21-Dec-2012 Status Final Destination European Commission WP number WP4 Dissemination Level Public Deliverable Nature Report ROMEO WP4 Page 2/50 TABLE OF CONTENTS TABLE OF CONTENTS ............................................................................................................. 3 LIST OF FIGURES...................................................................................................................... 5 LIST OF TABLES ....................................................................................................................... 7 1 INTRODUCTION ................................................................................................................. 8 1.1 Purpose of the Document ........................................................................................... 8 1.2 Objectives and Achievements ..................................................................................... 8 1.3 Structure of the Document .......................................................................................... 8 2 Low layer requirements and specification for packetization ....................................... 9 2.1 MPEG-TS requirements .............................................................................................. 9 2.1.1 General principle ................................................................................................. 9 2.1.2 Multiplex of MPEG-4 SVC Elementary Streams ............................................... 13 2.1.3 Audio Elementary Streams................................................................................ 17 2.2 P2P network/protocol requirements .......................................................................... 18 2.3 DVB network/protocol................................................................................................ 19 3 Proposed packetization schemes ................................................................................. 24 3.1 Protocol stack ............................................................................................................ 24 3.1.1 State of the art ................................................................................................... 24 3.1.2 Proposed protocol stack for ROMEO ................................................................ 24 3.2 Packetization schemes.............................................................................................. 25 3.3 Media partitioning and muxing .................................................................................. 26 3.3.1 Multiview video streams .................................................................................... 26 3.3.2 Metadata streams .............................................................................................. 28 3.3.3 Audio streams ................................................................................................... 29 3.3.4 Synchronization and buffer management ......................................................... 32 4 Adaptation and error resilience ..................................................................................... 34 4.1 Media aware mechanisms ........................................................................................ 34 4.1.1 Packet discarding mechanisms for mobility component ................................... 35 4.1.2 Experimental Setup ........................................................................................... 38 4.1.3 Experimental Results ........................................................................................ 39 4.2 Application based resilience mechanisms ................................................................ 41 4.2.1 Multiple description mechanisms ...................................................................... 41 4.2.2 Experimental results of rate adaptation in parallel with MDC ........................... 45 4.3 Fail-over mechanism at P2P level ............................................................................. 47 5 CONCLUSIONS ................................................................................................................ 49 ROMEO WP4 Page 3/50 6 REFERENCES .................................................................................................................. 50 ROMEO WP4 Page 4/50 LIST OF FIGURES Figure 1 - General principle of a Transport Stream multiplexer ................................................................ 10 Figure 2 - MPEG2-TS packet header ....................................................................................................... 11 Figure 3 - PAT and PMT relationship in MPEG2-TS ................................................................................ 11 Figure 4 - PES packet .............................................................................................................................. 12 Figure 5 - MPEG-4 AVC bitstream structure ............................................................................................ 14 Figure 6 - SVC NALU structure ................................................................................................................ 14 Figure 7 - SVC NALU header extension .................................................................................................. 15 Figure 8 - MPEG-4 AVC/H.264 Byte Stream format ................................................................................ 16 Figure 9 - MPEG-4 ADTS frame structure ............................................................................................... 17 Figure 10 - High-level view of the DVB-T2 system ................................................................................... 19 Figure 11 - Principle of Baseband Frame building in DVB-T2 .................................................................. 20 Figure 12 - FEC for Baseband frames in DVB-T2 .................................................................................... 20 Figure 13 - Packet structure for transmission over IP .............................................................................. 25 Figure 14 - SVC bit-stream structure (single slice) ................................................................................... 26 Figure 15 - SVC bit-stream structure (multiple slices) .............................................................................. 27 Figure 16 - Packetized Elementary Stream (PES) formation ................................................................... 27 Figure 17 - PES to TS to chunk ............................................................................................................... 28 Figure 18 - Synchronous Metadata PES Stream ..................................................................................... 29 Figure 19 - PES packet format for audio encapsulation in Romeo ........................................................... 30 Figure 20 - Timing between DVB and P2P if the delay is longer in the P2P network ............................... 33 Figure 21 - Traffic through Media Aware Proxy ........................................................................................ 34 Figure 22 - Abstract MAP Flowchart ........................................................................................................ 35 Figure 23 - First adte_decision method .................................................................................................... 36 Figure 24 - Second adte_decision method ............................................................................................... 37 Figure 25 - adte_check_packet method ................................................................................................... 38 Figure 26 - Testbed Components and Topology ...................................................................................... 39 Figure 27 - Comparison of PSNR per frame of SVC side-by-side sequence obtained with and without enabling MAP (6Mbps B/W) ............................................................................................................. 39 Figure 28 - Comparison of PSNR per frame of SVC side-by-side sequence obtained with and without enabling MAP (6.25Mbps B/W) ........................................................................................................ 39 ROMEO WP4 Page 5/50 Figure 29 - Comparison