Low Delay Video Coding
Total Page:16
File Type:pdf, Size:1020Kb
LOW DELAY VIDEO CODING THÈSE NO 3453 (2006) PRÉSENTÉE À LA FACULTÉ SCIENCES ET TECHNIQUES DE L'INGÉNIEUR Institut de traitement des signaux SECTION DE GÉNIE électrique ET électronique ÉCOLE POLYTECHNIQUE FÉDÉRALE DE LAUSANNE POUR L'OBTENTION DU GRADE DE DOCTEUR ÈS SCIENCES PAR Graziano VAROTTO Laurea in ingegneria elettronica, Università di Pisa, Italie et de nationalité italienne acceptée sur proposition du jury: Prof. D. Mlynek, directeur de thèse Dr B. Hochet, rapporteur Prof. G. Iannaccone, rapporteur Prof. M. Kunt, rapporteur Lausanne, EPFL 2006 iii Aerodynamically, the bumble bee should not be able to fly, but the bumble bee does not know it so it goes on flying anyway. Mary Kay Ash Acknowledgments I would like to acknowledge that persons that either directly either indirectly have supported me in these years of work. First of all I would like to thank Prof. Daniel Mlynek who gives me the opportunity to join the LTS3 laboratory, where I have spent these years learning the importance to work in a group and to be strong motivated to obtain the best results. I also thank Dr. Marco Mattavelli for his guidance and encouragement during this long period. I am grate to Professors Murat Kunt, Bertrand Hochet and Giuseppe Iannaccone who accepted to be members of the jury, for their suggestions and useful comments to improve the quality of this work. I would like to thank my colleagues Mauro Lattuada and Jean-Fran»coiseCsomo for the close collab- oration and for their help and suggestion during the di±cult development of this work. A great thank to Renzo Posega for his friendship during these years of sharing a considerable part of problems and solutions, and not least his help in reading and reviewing this work. A double thank to Massimo Ravasi for his frequent suggestions on C++ and further programming issues and to his wife Ester, who has accepted to correct my written English. A special thank to the old and present LTS3 collegues, friends and all Livetools sta® for the professional experience and pleasant moments we have shared. Finally, I would like to express my love and gratitude to Saviana for her everlasting support and encouragement. Lausanne 19th Jannuary 2006. vi ACKNOWLEDGMENTS Contents Acknowledgments v Contents vii List of Figures xi List of Tables xv Glossary xvii Abstract xxi Version Abr¶eg¶ee xxiii 1 Introduction 3 1.1 DIGITAL COMMUNICATIONS . 4 1.2 WIRELESS DIGITAL TELEVISION PRODUCTION . 5 1.2.1 Studio Production . 7 1.3 GOAL OF THIS WORK . 8 1.4 MAIN CONTRIBUTIONS . 9 1.5 STRUCTURE OF THIS THESIS . 9 2 Video Coding Concepts 11 2.1 INTRODUCTION . 12 2.2 NATURAL VIDEO SEQUENCES . 13 2.3 VIDEO DATA SAMPLING . 13 2.3.1 Spatial Sampling . 14 2.3.2 Temporal Sampling . 14 2.3.3 Frames and Fields . 14 2.4 COLOR SPACES . 15 2.4.1 RGB . 16 2.4.2 YCbCr . 17 2.4.3 YCbCr Sampling Formats . 18 2.5 VIDEO FORMATS . 19 2.6 QUALITY ASSESSMENT . 21 2.6.1 Subjective Quality Measurement . 21 vii viii CONTENTS 2.6.2 Objective Quality Measurement . 23 2.7 VIDEO COMPRESSION . 24 2.8 SPATIAL REDUNDANCY . 25 2.8.1 Predictive Coding . 26 2.8.2 Transform Coding . 27 2.8.3 Quantization . 30 2.8.4 Reordering . 32 2.9 TEMPORAL REDUNDANCY . 33 2.9.1 Di®erence Coding . 34 2.9.2 Block Based Motion Estimation and Compensation . 35 2.9.3 Bidirectional Motion Compensation . 36 2.9.4 Variable Block Size Motion Compensation . 37 2.9.5 Sub-pixel Motion Compensation . 38 2.10 ENTROPY CODER . 38 2.11 DPCM/DCT VIDEO CODEC MODEL . 40 2.12 TIME STAMPS . 42 2.13 RATE CONTROLLED ENCODER . 43 2.14 CONCLUSIONS . 45 3 State of The Art: MPEG-4 and H.264 47 3.1 INTRODUCTION . 48 3.2 ORGANIZATIONS AND STANDARDS . 48 3.2.1 ITU-T VCEG . 48 3.2.2 ISO MPEG . 49 3.2.3 JVT . 51 3.3 MPEG-4 OVERVIEW . 52 3.4 MPEG-4 VISUAL . 53 3.4.1 Main Features . 54 3.4.2 Tools, Pro¯les and Levels . 55 3.5 H.264/AVC . 55 3.5.1 Main Features . 57 3.5.2 Tools, Pro¯les and Levels . 60 3.6 CONCLUSIONS . 61 4 Coding/Decoding Delay in MPEG 63 4.1 INTRODUCTION . 64 4.2 MAIN CAUSES OF DELAY IN MPEG . 64 4.2.1 Data Format . 65 4.2.2 Frame and Field Encoding . 65 4.2.3 GOP Size and Structure . 66 4.2.4 Video Bu®er Veri¯er . 68 4.3 MPEG LOW DELAY SOLUTIONS . 69 4.3.1 VBV Bu®er Size . 70 4.3.2 I and P Only . 71 4.3.3 Slice Encoding . 72 4.3.4 VBV Bu®er Rate Control . 73 CONTENTS ix 4.3.5 NEL MPEG-2 Encoder . 74 4.3.6 IBM MPEG-2 Decoder . 78 4.4 CONCLUSIONS . 80 5 HERMES Codec 83 5.1 INTRODUCTION . 84 5.2 CODEC CORE . 84 5.2.1 Activity Function . 86 5.2.2 Median Adaptive Predictor . 86 5.2.3 Nonuniform Adaptive Quantizers . 88 5.2.4 Conditional Coding . 90 5.2.5 Entropy Coding . 91 5.2.6 Chrominance Scrambling . 93 5.2.7 Slice Coding . 95 5.3 RATE LIMITATIONS . 96 5.4 BUFFER and DELAY . 97 5.4.1 Bu®er Control Strategy . 98 5.4.2 Constant Quality Reference . 103 5.4.3 Vertical Blanking Discontinuity . 105 5.5 SYSTEM LAYER . 107 5.5.1 Synchronization . 107 5.5.2 Packetised Elementary Stream . ..