DOCSLIB.ORG

Efficient Motion Estimation and Mode Decision Algorithms for Advanced Video Coding

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Efficient Motion Estimation and Mode Decision Algorithms for Advanced Video Coding University of Windsor Scholarship at UWindsor Electronic Theses and Dissertations Theses, Dissertations, and Major Papers 2011 Efficient Motion Estimation and Mode Decision Algorithms for Advanced Video Coding Mohammed Golam Sarwer University of Windsor Follow this and additional works at: https://scholar.uwindsor.ca/etd Recommended Citation Sarwer, Mohammed Golam, "Efficient Motion Estimation and Mode Decision Algorithms for Advanced Video Coding" (2011). Electronic Theses and Dissertations. 439. https://scholar.uwindsor.ca/etd/439 This online database contains the full-text of PhD dissertations and Masters’ theses of University of Windsor students from 1954 forward. These documents are made available for personal study and research purposes only, in accordance with the Canadian Copyright Act and the Creative Commons license—CC BY-NC-ND (Attribution, Non-Commercial, No Derivative Works). Under this license, works must always be attributed to the copyright holder (original author), cannot be used for any commercial purposes, and may not be altered. Any other use would require the permission of the copyright holder. Students may inquire about withdrawing their dissertation and/or thesis from this database. For additional inquiries, please contact the repository administrator via email ([email protected]) or by telephone at 519-253-3000ext. 3208. Efficient Motion Estimation and Mode Decision Algorithms for Advanced Video Coding by Mohammed Golam Sarwer A Dissertation Submitted to the Faculty of Graduate Studies through the Department of Electrical and Computer Engineering in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy at the University of Windsor Windsor, Ontario, Canada 2011 © 2011, Mohammed Golam Sarwer All Rights Reserved. No part of this document may be reproduced, stored or otherwise retained in a retrieval system or transmitted in any form, on any medium by any means without prior written permission of the author. Declaration of Co-Authorship / Previous Publication I. Co-Authorship Declaration I hereby declare that this thesis incorporates material that is result of joint research, as follows: This thesis also incorporates the outcome of a joint research undertaken by me under the supervision of Professor Dr. Jonathan Wu. The collaboration is covered in Chapter 3, 4, 5, 6, and 7 of the thesis. In all cases, the key ideas, primary contributions, experimental designs, data analysis and interpretation, were performed by the author, and the contribution of co-author was primarily through the provision of valuable suggestions and helping in comprehensive analysis of the simulation results for publication. I am aware of the University of Windsor Senate Policy on Authorship and I certify that I have properly acknowledged the contribution of other researchers to my thesis, and have obtained written permission from each of the co-author(s) to include the above material(s) in my thesis. I certify that, with the above qualification, this thesis, and the research to which it refers, is the product of my own work. II. Declaration of Previous Publication This thesis includes 16 original papers that have been previously published/submitted for publication in peer reviewed journals and conferences, as follows: Thesis Publication title/full citation Publication Chapter status* Mohammed Golam Sarwer, and Q.M. Jonathan Wu published “Efficient Two Step Edge based Partial Distortion Search for Fast Block Motion Estimation” IEEE Transactions on Consumer Electronics, Vol. 55, No. 4, Nov. 2009, pp. 2154- 2162. Mohammed Golam Sarwer, and Q.M. Jonathan Wu, “Fast published Block Motion Estimation by edge based partial distortion search”, Proceedings of the IEEE International Conference on Image Processing 2009 (ICIP-2009), Cairo, Egypt, Chapter 3 November 7-10, 2009, pp. 1573 - 1576. Mohammed Golam Sarwer, and Q.M. Jonathan Wu, published “Efficient Partial Distortion Search Algorithm for Block based Motion Estimation”, Proceedings of the IEEE CCECE 2009, St John’s, NL, pp. 890-893. Mohammed Golam Sarwer, and Q.M. Jonathan Wu published “Adaptive Variable Block-Size Early motion estimation iv termination algorithm for H.264/AVC Video Coding Standard.” IEEE Transaction on Circuit and System for Video Technology, volume 19, number 8, August 2009, pp 1196-1201. Mohammed Golam Sarwer, Thanh Minh Nguyen and Q.M. published Jonathan Wu, “Fast Motion estimation of H.264/AVC by adaptive early termination”, Proceedings of the 10th IASTED International Conference SIP 2008, HI, USA, pp. Chapter 4 140-145. Mohammed Golam Sarwer, and Q.M. Jonathan Wu, published “Region Based Searching for Early Terminated Motion Estimation Algorithm of H.264/AVC Video Coding Standard”, Proceedings of the IEEE CCECE 2009, St John’s, NL, pp. 468-471. Mohammed Golam Sarwer, and Q.M. Jonathan Wu, " An published Efficient search range Decision Algorithm for Motion Estimation of H.264/AVC, " International Journal of Circuits, Systems and Signal Processing, vol. 3, issue 4, pp. 173-180, 2009. Mohammed Golam Sarwer, and Q.M. Jonathan Wu, published Chapter 5 “Adaptive Search Area Selection of Variable Block-Size Motion Estimation of H.264/AVC Video Coding Standard”, IEEE International Symposium on Multimedia ISM 2009, San Diego, California, USA, December 14-16, 2009, pp. 100-105. Mohammed Golam Sarwer, and Q.M. Jonathan Wu In Press “Improved Intra Prediction of H.264/AVC” accepted in the book "Video Coding", ISBN 978-953-7619-X-X, IN-TECH publisher. Mohammed Golam Sarwer, and Q.M. Jonathan Wu, published “Enhanced Intra Coding of H.264/AVC Advanced Video Coding Standard with Adaptive Number of Modes”, International Conferences on Active Media Technology (AMT 2010), Toronto, Canada, LNCS 6335, pp. 361–372, August 2010. Mohammed Golam Sarwer, and Q.M. Jonathan Wu, “A published Novel Bit Rate Reduction Method of H.264/AVC Intra Chapter 6 Coding”, International Congress on Image and Signal Processing (CISP'10), 16-18 October 2010, Yantai, China, pp. 24-28. Mohammed Golam Sarwer, and Q. M. Jonathan Wu, " Accepted Improved DC Prediction for H.264/AVC intra Coding " 2011 International Conference on Communication and Electronics Information - ICCEI 2011, Haikou, China. Mohammed Golam Sarwer, and Q.M. Jonathan Wu Submitted “Performance Improvement of Intra Coding in H.264/AVC v Advanced Video Coding Standard”. Submitted to Journal of Visual Communication and Image Representation. Chapter 7 Mohammed Golam Sarwer, and Q.M. Jonathan Wu Submitted “Enhanced SATD based Cost Function for Mode Selection of H.264/AVC Intra Coding”. Submitted to Springer Journal of Signal, Image and Video Processing. Mohammed Golam Sarwer, and Q.M. Jonathan Wu Accepted “Enhanced low complex cost function of H.264/AVC intra mode decision”. International Conference on Multimedia and Signal Processing (CMSP'11), China, Mohammed Golam Sarwer, Q.M. Jonathan Wu, X. P Zhang Submitted “Efficient rate-distortion optimization of H.264/AVC intra coder”. Submitted to International Conference on Image Proceeding, 2011 I certify that I have obtained permission from the copyright owner(s) to include the above published material(s) in my thesis. I certify that the above material describes work completed during my registration as graduate student at the University of Windsor. I declare that, to the best of my knowledge, my thesis does not infringe upon anyone’s copyright nor violate any proprietary rights and that any ideas, techniques, quotations, or any other material from the work of other people included in my thesis, published or otherwise, are fully acknowledged in accordance with the standard referencing practices. Furthermore, to the extent that I have included copyrighted material that surpasses the bounds of fair dealing within the meaning of the Canada Copyright Act, I certify that I have obtained a written permission from the copyright owner(s) to include such material(s) in my thesis. I declare that this is a true copy of my thesis, including any final revisions, as approved by my thesis committee and the Graduate Studies office, and that this thesis has not been submitted for a higher degree to any other University or Institution. vi ABSTRACT H.264/AVC video compression standard achieved significant improvements in coding efficiency, but the computational complexity of the H.264/AVC encoder is drastically high. The main complexity of encoder comes from variable block size motion estimation (ME) and rate-distortion optimized (RDO) mode decision methods. This dissertation proposes three different methods to reduce computation of motion estimation. Firstly, the computation of each distortion measure is reduced by proposing a novel two step edge based partial distortion search (TS-EPDS) algorithm. In this algorithm, the entire macroblock is divided into different sub-blocks and the calculation order of partial distortion is determined based on the edge strength of the sub-blocks. Secondly, we have developed an early termination algorithm that features an adaptive threshold based on the statistical characteristics of rate-distortion (RD) cost regarding current block and previously processed blocks and modes. Thirdly, this dissertation presents a novel adaptive search area selection method by utilizing the information of the previously computed motion vector differences (MVDs). In H.264/AVC intra coding, DC mode is used to predict regions with no unified direction and the predicted pixel values are same and thus smooth varying regions are not well de-correlated. This dissertation proposes
Load more

Recommended publications
  • Hi3716m V430 Hi3716m V430 Cable HD Chip Brief Data Sheet
    Hi3716M V430 Hi3716M V430 Cable HD Chip Brief Data Sheet Key Specifications CPU Graphics and Display Processing (Imprex 2.0 High-performance ARM Cortex-A7 processor Processing Engine) Built-in I-cache, D-cache, and L2 cache Hardware TDE Hardware Java acceleration 3-layer OSD Floating-point coprocessor Two video layers Memory Control Interfaces 16-bit and 32-bit color depth DDR3/DDR3L interface Full-hardware anti-aliasing and anti-flicker − Maximum capacity of 512 MB for an external DDR IE, NR, and CCS SDRAM or of 128 MB/256 MB for a built-in DDR DEI SDRAM Audio and Video Interfaces − 16-bit data width PAL, NTSC, and SECAM standard outputs, and forcible A SPI NOR flash/SPI NAND flash, a parallel SLC NAND standard conversion flash, or a SPI NOR flash+a parallel SLC NAND flash Aspect ratio of 4:3 or 16:9, forcible aspect ratio Video Decoding (HiVXE 2.0 Processing Engine) conversion, and free scaling H.265 Main/Main 10@Level 4.1 high-tier 1080p50(60)/1080i/720p/576p/576i/480p/480i outputs H.264 BP/MP/HP@Level 4.2; MVC HD and SD output for the same source MPEG-1 Color gamut compliant with the xvYCC (IEC 61966-2-4) MPEG-2 SP@ML and MP@HL standard MPEG-4 SP@Levels 0–3, ASP@Levels 0–5, GMC, and HDMI 1.4b with HDCP1.4 MPEG-4 short header format (H.263 baseline) Analog video interfaces AVS baseline@Level 6.0 and AVS+ − One CVBS interface VC-1 SP@ML, MP@HL, and AP@Levels 0–3 − One built-in VDAC VP6/VP8 − VBI 1-channel 1080p@60 fps decoding Audio interface − Audio-left and audio-right channels Image Decoding − S/PDIF
    [Show full text]
  • Computational Complexity Optimization on H.264 Scalable/Multiview Video Coding
    Computational Complexity Optimization on H.264 Scalable/Multiview Video Coding By Guangyao Zhang A thesis submitted in partial fulfilment for the requirements for the degree of PhD, at the University of Central Lancashire April 2014 Student Declaration Concurrent registration for two or more academic awards I declare that while registered as a candidate for the research degree, I have not been a registered candidate or enrolled student for another award of the University or other academic or professional institution. Material submitted for another award I declare that no material contained in the thesis has been used in any other submission for an academic award and is solely my own work. Collaboration This work presented in this thesis was carried out at the ADSIP (Applied Digital Signal and Image Processing) Research Centre, University of Central Lancashire. The work described in the thesis is entirely the candidate’s own work. Signature of Candidate ________________________________________ Type of Award Doctor of Philosophy School School of Computing, Engineering and Physical Sciences Abstract Abstract The H.264/MPEG-4 Advanced Video Coding (AVC) standard is a high efficiency and flexible video coding standard compared to previous standards. The high efficiency is achieved by utilizing a comprehensive full search motion estimation method. Although the H.264 standard improves the visual quality at low bitrates, it enormously increases the computational complexity. The research described in this thesis focuses on optimization of the computational complexity on H.264 scalable and multiview video coding. Nowadays, video application areas range from multimedia messaging and mobile to high definition television, and they use different type of transmission systems.
    [Show full text]
  • Hi3796m V200 Brief Data Sheet
    Hi3796M V200 Hi3796M V200 Brief Data Sheet Key Specifications High-Performance CPU AAC-LC and HE-AAC V1/V2 decoding 64-bit quad-core high-performance ARM Cortex A53 APE, FLAC, Ogg, AMR-NB, and AMR-WB decoding Integrated multimedia acceleration engine NEON G.711 (u/a) audio decoding Hardware Java acceleration G.711 (u/a), AMR-NB, AMR-WB, and AAC-LC audio Integrated hardware floating-point coprocessor encoding Integrated L1 and L2 Cache HE-AAC transcoding DD (AC3) 3D GPU Channel Decoding and TS De-multiplexing Integrated high-performance multi-core GPU Mali 450 One embedded QAM module, supporting the ITU OpenGL ES 2.0/1.1 and OpenVG 1.1 J83-A/B/C standard Memory Control Interfaces Multi TS processing simultaneously DDR3/DDR3L/DDR4 interface, supporting maximum TS interface for Full Band Capture device 32-bit data width Various descrambling algorithms of the DVB eMMC 5.0 flash interface Security Processing SPI NOR flash interface Advanced CA SPI NAND flash interface Downloadable Conditional Access System Asynchronous/Synchronous NAND flash interface TVOS security mechanism − SLC/MLC flash memory Secure boot, secure storage, and secure upgrade − Maximum 64-bit ECC DRM and hardware watermark Video Decoding (HiVXE 2.0 Processing Engine) HDCP 2.2/1.4 protection for HDMI outputs AVS2 Main-10bit Profile, Level 8.2.60, supporting Graphics and Display Processing (Imprex 2.0 maximum 4K x 2K@60 fps 10-bit decoding Processing Engine) H.265/HEVC Main/Main 10 Profile@Level 5.1 high-tier, Dolby Vision, HDR10, HLG, and SLF
    [Show full text]
  • Video Encoding with Open Source Tools
    Video Encoding with Open Source Tools Steffen Bauer, 26/1/2010 LUG Linux User Group Frankfurt am Main Overview Basic concepts of video compression Video formats and codecs How to do it with Open Source and Linux 1. Basic concepts of video compression Characteristics of video streams Framerate Number of still pictures per unit of time of video; up to 120 frames/s for professional equipment. PAL video specifies 25 frames/s. Interlacing / Progressive Video Interlaced: Lines of one frame are drawn alternatively in two half-frames Progressive: All lines of one frame are drawn in sequence Resolution Size of a video image (measured in pixels for digital video) 768/720×576 for PAL resolution Up to 1920×1080p for HDTV resolution Aspect Ratio Dimensions of the video screen; ratio between width and height. Pixels used in digital video can have non-square aspect ratios! Usual ratios are 4:3 (traditional TV) and 16:9 (anamorphic widescreen) Why video encoding? Example: 52 seconds of DVD PAL movie (16:9, 720x576p, 25 fps, progressive scan) Compression Video codec Raw Size factor Comment 1300 single frames, MotionTarga, Raw frames 1.1 GB - uncompressed HUFFYUV 459 MB 2.2 / 55% Lossless compression MJPEG 60 MB 20 / 95% Motion JPEG; lossy; intraframe only lavc MPEG-2 24 MB 50 / 98% Standard DVD quality X.264 MPEG-4 5.3 MB 200 / 99.5% High efficient video codec AVC Basic principles of multimedia encoding Video compression Lossy compression Lossless (irreversible; (reversible; using shortcomings statistical encoding) in human perception) Intraframe encoding
    [Show full text]
  • Introduction
    CMPT 820 Multimedia Systems Introduction Ze-Nian Li Spring 2019 1 CMPT820 Multimedia Systems Why this course? Multimedia is cool Media -> Multimedia Everywhere Requires broad knowledge in mathematics, signal processing, communications, networking, software, hardware, … Job opportunities Multimedia is a booming industry • in the metro Vancouver area Tons of opportunities created by next-generation standards and emerging applications: • JPEG/JPEG 2000 • MPEG-1/2/4 H.264/265/HEVC 4K/8K TV 3D/freeview • 3G/4G/5G mobile communications • Multimedia-enabled smartphone, tablets • Social media, Cloud media, Crowd media • Online gaming 2 CMPT820 Multimedia Systems Multimedia is Multidisciplinary Computer hardware, network, operating system, database Image, audio, Multimedia Computer vision, speech computing pattern recognition, processing Machine learning Human computer Computer interaction graphics 3 CMPT820 Multimedia Systems Books and References Recommended Textbook Fundamentals of Multimedia, 2nd Edition, by Z.N. Li, M.S. Drew, and J. Liu, Springer, 2014. Reference books Video Processing and Communications, Y. Wang, J. Ostermann, Y-Q Zhang, Prentice Hall, 2002. Resource Home page • www.cs.sfu.ca/CC/820/li/ Please check it regularly 4 CMPT820 Multimedia Systems Grading Scheme Two programming assignments 2x10% Presentation and class participation 40% Term project 40% It is a Graduate seminar course ! 5 CMPT820 Multimedia Systems Topics Introduction to Image and Video Compression Wavelets and JPEG-2000 H.264/MPEG-4 AVC, H.265, and MPEG-7 Image and Video Quality Assessment Content Based Image and Video Retrieval Visual Content Analysis Digital Audio Compression 6 CMPT820 Multimedia Systems Questions? 7 CMPT820 Multimedia Systems What is Multimedia? Multimedia means that information can be represented through audio, images, graphics and animation, video, in addition to traditional media (i.e., text and graphics drawings).
    [Show full text]
  • 3-D Video Formats and Coding- a Review
    International Journal of Engineering Science Invention ISSN (Online): 2319 – 6734, ISSN (Print): 2319 – 6726 www.ijesi.org ||Volume 6 Issue 2|| February 2017 || PP. 26-36 3-D Video Formats and Coding- A review Khalid Mohamed Alajel1, Khairi Muftah Abusabee1, Ali Tamtum1 1(Electrical and Computer Engineering Department, Faculty of Engineering / Al-Mergib University, Libya) Abstract: The objective of a video communication system is to deliver the maximum of video data from the source to the destination through a communication channel using all of its available bandwidth. To achieve this objective, the source coding should compress the original video sequence as much as possible and the compressed video data should be robust and resilient to channel errors. However, while achieving a high coding efficiency, compression also makes the coded video bitstream vulnerable to transmission errors. Thus, the process of video data compression tends to work against the objectives of robustness and resilience to errors. Therefore, extra information that needs to be transmitted in 3-D video has brought new challenge and consumer applications will not gain more popularity unless the 3-D video coding problems are addressed. Keywords: 2-D video; 3-D video; 3-D Formats; 3-D video coding. I. Introduction A digital video sequence consists of images, which are known as frames. Each frame consists of small picture elements, pixels that describe the color at that point in the frame. To describe fully the video sequence, a huge amount of data is required. Therefore, the video sequence is compressed to reduce the amount of data to make possible transmission over channels with limited bandwidth.
    [Show full text]
  • Multi⁃Layer Extension of the High Efficiency Video Coding (HEVC) Standard
    Special Topic DOI: 10.3969/j. issn. 1673Ƽ5188. 2016. 01. 003 http://www.cnki.net/kcms/detail/34.1294.TN.20160119.0957.002.html, published online January 19, 2016 Multi⁃Layer Extension of the High Efficiency Video Coding (HEVC) Standard Ming Li and Ping Wu (ZTE Corporation, Shenzhen 518057, China) Abstract Multi⁃layer extension is based on single⁃layer design of High Efficiency Video Coding (HEVC) standard and employed as the com⁃ mon structure for scalability and multi⁃view video coding extensions of HEVC. In this paper, an overview of multi⁃layer extension is presented. The concepts and advantages of multi⁃layer extension are briefly described. High level syntax (HLS) for multi⁃layer extension and several new designs are also detailed. Keywords HEVC; multi⁃layer extension wide color gamut (HDR & WCG) [8] are being developed in 1 Introduction VCEG and MPEG and are scheduled for release in the coming igh Efficiency Video Coding (HEVC) standard is one or two years. the newest video coding standard of the ITU ⁃ T In the second version of the HEVC standard, the concept of Q6/16 Video Coding Experts Group (VCEG) and layer refers to a scalable layer (e.g. a spatial scalable layer) in H the ISO/IEC JTC 1 SC 29/WG 11 Moving Pic⁃ SHVC or a view in MV⁃HEVC. Fig. 1 shows an example SH⁃ ture Experts Group (MPEG). The first version of HEVC stan⁃ VC bitstream of spatial scalability with two layers. The base dard was released in 2013 [1] and referred to as“HEVC Ver⁃ layer (BL) is of lower resolution, and the enhancement layer sion 1”standard.
    [Show full text]
  • Multiview Video Compression and Discuss Them in More Detail
    [Markus Flierl and Bernd Girod] © BRAND X PICTURES Multiview Video Compression [Exploiting inter-image similarities] dvances in display and camera technology enable new applications for three-dimen- sional (3-D) scene communication. Among the most important of these applica- tions is 3-D TV; it strives to create realistic 3-D impressions of natural 3-D scenes [1]. Usually, multiple video cameras are used to simultaneously acquire various viewpoints of a scene. The resulting data are often referred to as multiview Avideo. As the potential degree of 3-D realism improves with the camera density around the scene, a vast amount of multiview video data needs to be stored or transmitted for 3-D TV. Multiview video data is also expected to consume a large portion of the bandwidth available in the Internet of the future. This will include point-to-point communication as well as multicast scenarios. Multimedia distribution via sophisticated content delivery networks and flexible peer-to-peer networks enable possible multiview video on demand as well as live broadcasts. Due to the vast raw bit rate of multiview video, efficient compression techniques are essen- tial for 3-D scene communication [2]. As the video data originate from the same scene, the inherent similarities of the multiview imagery are exploited for efficient compression. These similarities can be classified into two types, inter-view similarity between adjacent camera views and temporal similarity between temporally successive images of each video. Digital Object Identifier 10.1109/MSP.2007.905699 IEEE SIGNAL PROCESSING MAGAZINE [66] NOVEMBER 2007 1053-5888/07/$25.00©2007IEEE Temporal similarities can be exploited with motion com- Figure 1 depicts a matrix of pictures for N = 4 image pensation techniques that are well known from single-view sequences, each composed of K = 4 temporally successive pic- video compression.
    [Show full text]
  • Low-Complexity Scalable and Multiview Video Coding
    Low-Complexity Scalable and Multiview Video Coding – Laagcomplexe schaalbare en meervoudig-perspectieve videocompressie – Sebastiaan Van Leuven Promotor: prof. dr. ir. R. Van de Walle, dr. ir. J. De Cock Proefschrift ingediend tot het behalen van de graad van Doctor in de Ingenieurswetenschappen: Computerwetenschappen Vakgroep Elektronica en Informatiesystemen Voorzitter: prof. dr. ir. J. Van Campenhout Faculteit Ingenieurswetenschappen en Architectuur Academiejaar 2012-2013 Laagcomplexe schaalbare en meervoudig-perspectieve videocompressie Low-Complexity Scalable and Multiview Video Coding Sebastiaan Van Leuven Promotoren: prof. dr. ir. R. Van de Walle, dr. ir. J. De Cock Proefschrift ingediend tot het behalen van de graad van Doctor in de Ingenieurswetenschappen: Computerwetenschappen Vakgroep Elektronica en Informatiesystemen Voorzitter: prof. dr. ir. J. Van Campenhout Faculteit Ingenieurswetenschappen en Architectuur Academiejaar 2012 - 2013 ISBN 978-90-8578-612-2 NUR 965 Wettelijk depot: D/2013/10.500/45 Dankwoord “You know, it’s funny what’s happening to us. Our lives have become digital, our friends now virtual, and everything you could ever want to know is just a click away. Experiencing the world through endless second hand information isn’t enough. If we want authenticity, we have to initiate it.” –Travis Rice, The Art of Flight (2011) Het werk dat hier voor u ligt, is niet het werk van e´en´ persoon. Vele men- sen hebben bijgedragen tot wat het uiteindelijk geworden is. Graag had ik de mensen bedankt die mij op e´en´ of andere manier geholpen hebben om dit te verwezenlijken. Vooreerst had ik graag mijn promotor prof. Rik Van de Walle bedankt omdat hij mij de kans heeft gegeven dit doctoraat aan te vangen, omdat hij mij de vrijdheid heeft gegeven verschillende pistes te bewandelen en mij hiervoor te voorzien in voldoende middelen.
    [Show full text]
  • The Effects of Multiview Depth Video Compression on Multiview Rendering
    ARTICLE IN PRESS Signal Processing: Image Communication 24 (2009) 73–88 Contents lists available at ScienceDirect Signal Processing: Image Communication journal homepage: www.elsevier.com/locate/image The effects of multiview depth video compression on multiview rendering P. Merkle a,Ã, Y. Morvan b, A. Smolic a, D. Farin b,1,K.Mu¨ ller a, P.H.N. de With b, T. Wiegand a,c a Fraunhofer Institute for Telecommunications Heinrich-Hertz-Institut, Image Processing Department, Einsteinufer 37, 10587 Berlin, Germany b Eindhoven University of Technology, Signal Processing Systems, PO Box 513, 5600 MB Eindhoven, The Netherlands c Image Communication Chair, Technical University of Berlin, Einsteinufer 17, 10587 Berlin, Germany article info abstract Article history: This article investigates the interaction between different techniques for depth Received 7 October 2008 compression and view synthesis rendering with multiview video plus scene depth Accepted 19 October 2008 data. Two different approaches for depth coding are compared, namely H.264/MVC, using temporal and inter-view reference images for efficient prediction, and the novel Keywords: platelet-based coding algorithm, characterized by being adapted to the special 3D television characteristics of depth-images. Since depth-images are a 2D representation of the 3D Multiview video scene geometry, depth-image errors lead to geometry distortions. Therefore, the Depth compression influence of geometry distortions resulting from coding artifacts is evaluated for both View synthesis rendering coding approaches in two different ways. First, the variation of 3D surface meshes is analyzed using the Hausdorff distance and second, the distortion is evaluated for 2D view synthesis rendering, where color and depth information are used together to render virtual intermediate camera views of the scene.
    [Show full text]
  • Multiview Video Coding Extension of the H.264/AVC Standard
    Multiview Video Coding Extension of the H.264/AVC Standard Ognjen Nemþiü 1, Snježana Rimac-Drlje 2, Mario Vranješ 2 1 Supra Net Projekt d.o.o., Zagreb, Croatia, [email protected] 2 University of Osijek, Faculty of Electrical Engineering, Osijek, Croatia, [email protected] Abstract - An overview of the new Multiview Video Coding approved an amendment of the ITU-T Rec. H.264 & ISO/IEC (MVC) extension of the H.264/AVC standard is given in this 14996-10 Advanced Video Coding (AVC) standard on paper. The benefits of multiview video coding are examined by Multiview Video Coding. encoding two data sets captured with multiple cameras. These sequences were encoded using MVC reference software and the results are compared with references obtained by encoding II. MULTIVIEW VIDEO CODING multiple views independently with H.264/AVC reference MVC was designed as an amendment of the H.264/AVC software. Experimental quality comparison of coding efficiency is and uses improved coding tools which have made H.264/AVC given. The peak signal-to-noise ration (PSNR) and Structural so superior over older standards. It is based on conventional Similarity (SSIM) Index are used as objective quality metrics. block-based motion-compensated video coding, with several new features which significantly improve its rate-distortion Keywords - H.264/AVC, MVC, inter-view prediction, temporal performance. The main new features of H.264/AVC, also used prediction, video quality by MVC, are the following: • I. INTRODUCTION variable block-size motion-compensated prediction with the block size down to 4x4 pixels; The H.264/AVC is the state-of-the-art video coding standard, [1], developed jointly by the ITU-T Video Coding • quarter-pixel motion vector accuracy; Experts Group (VCEG) and the ISO/IEC Moving Pictures • Experts Group (MPEG).
    [Show full text]
  • Interactive Internet TV Architecture Based on Scalable Video Coding
    Interactive Internet TV Architecture Based on Scalable Video Coding Pedro Gomes Moscoso Dissertac¸ao˜ para obtenc¸ao˜ do Grau de Mestre em Engenharia de Redes de Comunicac¸oes˜ J ´uri Presidente: Prof. Doutor Paulo Jorge Pires Ferreira Orientador: Prof. Doutor Mario´ Serafim dos Santos Nunes Co-Orientador: Prof. Rui Antonio´ dos Santos Cruz Vogais: Prof. Doutor Paulo Luis Serras Lobato Correia Maio de 2011 Acknowledgments I would like to thank my parents for their friendship, encouragement and caring over all these years, for always being there for me through thick and thin and without whom this project would not have been possible. I would also like to thank my sisters, Ana, Teresa and Joana, grandparents, aunts, uncles and cousins for their support throughout all this time. I would also like to acknowledge my dissertation super- visors Professor Rui Cruz and Professor Mario´ Nunes for their insight, support and sharing of knowledge that has made this dissertation possible. Last but not least, to all my friends and colleagues that helped me grow as a person and were always there for me during the good and bad times in my life. Thank you. To each and every one of you - thank you. Abstract In a diversified environment as the Internet, with a wide diversity of terminals and connectivity, a proper video distribution is a complex problem. The multimedia distribution through several service providers has grown broadly in last years, constituting a large portion of the Internet traffic. However, many of these services do not support mechanisms to provide quality of user experience, causing stoppages in video playing, degradation of image or even failure.
    [Show full text]