Quality-Aware 3D Video Delivery

Quality-aware 3D Video Delivery by Ahmed Hamza M.Sc., Mansoura University, Egypt, 2008 B.Sc., Mansoura University, Egypt, 2003 Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in the School of Computing Science Faculty of Applied Science c Ahmed Hamza 2017 SIMON FRASER UNIVERSITY Spring 2017 All rights reserved. However, in accordance with the Copyright Act of Canada, this work may be reproduced without authorization under the conditions for “Fair Dealing.” Therefore, limited reproduction of this work for the purposes of private study, research, criticism, review and news reporting is likely to be in accordance with the law, particularly if cited appropriately. Approval Name: Ahmed Hamza Degree: Doctor of Philosophy Title: Quality-aware 3D Video Delivery Examining Committee: Chair: Arrvindh Shriraman Associate Professor Mohamed Hefeeda Senior Supervisor Professor Joseph Peters Supervisor Professor Jiangchuan Liu Internal Examiner Professor School of Computing Science Abdulmotaleb El Saddik External Examiner Professor School of Electrical Engineering and Computer Science University of Ottawa Date Defended: April 18, 2017 ii Abstract Three dimensional (3D) videos are the next natural step in the evolution of digital media technologies. In order to provide viewers with depth perception and immersive experience, 3D video streams contain one or more views and additional information describing the scene’s geometry. This greatly increases the bandwidth requirements for 3D video transport. In this thesis, we address the challenges associated with delivering high quality 3D video content to heterogeneous devices over both wired and wireless networks. We focus on three problems: energy-efficient multicast of 3D videos over 4/5G networks, quality-aware HTTP adaptive streaming of free-viewpoint videos, and achieving quality-of-experience (QoE) fairness in free-viewpoint video streaming in mobile networks. In the first problem, multiple 3D videos represented in the two-view-plus-depth format and scalably coded into several substreams are multicast over a broadband wireless network. We show that optimally select- ing the substreams to transmit for the multicast sessions is an NP-complete problem and present a polynomial time approximation algorithm to solve it. To maximize the power sav- ings of mobile receivers, we extend the algorithm to efficiently schedule the transmission of the chosen substreams from each video. In the second problem, we present a free-viewpoint video streaming architecture based on state-of-the-art HTTP adaptive streaming protocols. We propose a rate adaptation method for streaming clients based on virtual view quality models, which relate the quality of synthesized views to the qualities of the reference views, to optimize the user’s quality-of-experience. We implement the proposed adaptation method in a streaming client and assess its performance. Finally, in the third problem, we propose an efficient radio resource allocation algorithm in mobile wireless networks where multiple free-viewpoint video streaming clients compete for the limited resources. The re- sulting allocation achieves QoE fairness across the streaming sessions and it reduces quality fluctuations. Keywords: Free-viewpoint video; adaptive video streaming; rate adaptation; DASH; 3D video; energy efficiency; mobile multimedia; multi-view video; wireless networks iii To mom, dad, and Amgad. iv “Read!” — First verse of The Noble Qur’an (96:1) “And of knowledge, you (mankind) have been given only a little.” — The Noble Qur’an (17:85) “Details matter, it’s worth waiting to get it right.” — Steve Jobs “There is no greatness where there is not simplicity, goodness, and truth.” — Leo Tolstoy, War and Peace “He who can have patience can have what he will.” — Benjamin Franklin v Acknowledgements First and foremost, I owe my deepest gratitude to Dr. Mohamed Hefeeda. It has been a great honour to work with Dr. Hefeeda and have him as my senior supervisor. I like to thank him for his endless encouragement, patience, support, and guidance throughout this journey. His critical reviews and intellectual input has enabled me to develop a deeper understanding of the exciting fields of multimedia networking and distributed systems. I would like to extend my sincerest gratitude to Dr. Joseph Peters, my supervisor, for his valuable advice and comments during my graduate studies. I am heartily thankful to him for sharing his time and his insights whenever I needed them and for the valuable brainstorming and discussion sessions from which I have learned a lot. I would also like to express my gratitude to Dr. Jiangchuan Liu, my thesis examiner, and Dr. Abdulmotaleb El Saddik, my external thesis examiner, for being on my committee and reviewing this thesis. Many thanks to Dr. Arrvindh Shriraman for taking the time to chair my thesis defence. I want to thank all my colleagues at the Network Systems Lab throughout the years of my graduate career. I am especially grateful to Cheng-Hsin Hsu whom I greatly respect and greatly learned from. Thank you for all the valuable advice and support. Your dedication and hard work has been an inspiration for me to keep on going. Special thanks also goes to Cong Ly, Shabnam Mirshokraie, Somsubhra Sharangi, Saleh Almowuena, Ahmed Abdel- sadek, Kiana Calagari, Tarek El-Ganainy, and Khaled Diab. I also want to thank Hamed Ahmadi for all his help and for a great collaboration. I am really fortunate to have worked with such talented and amazing people and I cannot imagine this journey without them. More importantly, this thesis would not have been possible without the endless love and support of my parents and my brother, Amgad. Words cannot express my eternal gratitude to my parents who have made great sacrifices so that I can pursue my dreams. Thank you for your constant encouragement in all my pursuits and for always pushing me to succeed and to be a better person. Amgad, thank you for being an amazing brother. And thank you for all your support and care, and for cheering me up whenever I felt down. I am truly blessed to have you. vi Table of Contents Approval ii Abstract iii Dedication iv Quotations v Acknowledgements vi Table of Contents vii List of Tables x List of Figures xi List of Acronyms xiv 1 Introduction 1 1.1 Introduction.................................... 1 1.2 ThesisContributions . 5 1.2.1 Energy-efficient Multicast of 3D Videos over Wireless Networks . 5 1.2.2 Quality-aware HAS-based Free-viewpoint Video Streaming...... 7 1.2.3 QoE-fair Adaptive Streaming of FVV over LTE Networks . 8 1.3 ThesisOrganization .............................. 9 2 Background 10 2.1 Introduction.................................... 10 2.2 3D Content Capturing and Post-processing . 11 2.2.1 Camera Parameters and Geometric Calibration . 13 2.2.2 ImageRectification............................ 15 2.2.3 ColorCorrection ............................. 15 2.3 HumanVisualSystem .............................. 16 2.4 3DDisplayTechnologies . 17 vii 2.5 3DVideoRepresentationandCoding. 20 2.5.1 3DVideoRepresentations . 20 2.5.2 3DVideoCoding............................. 25 2.6 HTTPAdaptiveStreaming . 32 2.7 WirelessCellularNetworks . 35 2.7.1 IEEE802.16WiMAXNetworks. 35 2.7.2 LongTermEvolutionNetworks . 35 2.7.3 MultimediaMulticastServices . 38 3 Energy-Efficient Multicasting of Multiview 3D Videos over Wireless Networks 40 3.1 Introduction.................................... 40 3.2 RelatedWork................................... 42 3.2.1 3D Video Transmission Over Wireless Networks . 42 3.2.2 ModelingSynthesizedViewsQuality . 43 3.2.3 Optimal Texture-Depth Bit Allocation . 43 3.3 SystemOverview................................. 44 3.4 Problem Statement and Formulation . 45 3.5 ProposedSolution ................................ 49 3.5.1 Analysis.................................. 53 3.6 EnergyEfficientRadioFrameScheduling . 54 3.6.1 ProposedAllocationAlgorithm . 54 3.7 ValidationofVirtualViewQualityModel . 58 3.8 PerformanceEvaluation . 59 3.8.1 Setup ................................... 59 3.8.2 SimulationResults ............................ 63 3.9 Summary ..................................... 66 4 Virtual View Quality-aware Rate Adaptation for HTTP-based Free- viewpoint Video Streaming 70 4.1 Introduction.................................... 70 4.2 RelatedWork................................... 72 4.2.1 Server-basedApproaches. 72 4.2.2 Client-basedApproaches. 73 4.3 ProblemDefinition ................................ 73 4.4 ReferenceViewScheduling. 74 4.5 Virtual View Quality-aware Rate Adaptation . 77 4.5.1 Rate Adaptation Based on Empirical Virtual View Quality Measure- ments ................................... 77 4.5.2 Rate Adaptation Based on Analytical Virtual View QualityModels. 78 viii 4.6 System Architecture and Client Implementation . ......... 82 4.6.1 ContentServer .............................. 82 4.6.2 FVVDASHClient ............................ 86 4.7 Evaluation..................................... 90 4.7.1 ContentPreparation . 91 4.7.2 ExperimentalSetup ............................ 91 4.7.3 EmpiricalQualityModelsResults . 93 4.7.4 AnalyticalQualityModelsResults . 94 4.7.5 SubjectiveEvaluation . 97 4.8 Summary ..................................... 98 5 QoE-fair HTTP Adaptive Streaming of Free-viewpoint Videos in LTE Networks 105 5.1 Introduction.................................... 105 5.2 RelatedWork................................... 107 5.2.1 FairnessinWiredNetworks . 107 5.2.2 FairnessinWirelessNetworks . 108 5.3 SystemModelandOperation

Load more