DOCSLIB.ORG

Algorithms for Compression of High Dynamic Range Images and Video

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Algorithms for Compression of High Dynamic Range Images and Video Algorithms for Compression of High Dynamic Range Images and Video by Vladimir Dolzhenko A Doctoral Thesis Submitted in partial fulfilment of the requirements for the award of Doctor of Engineering of Loughborough University 31.05.2015 © by Vladimir Dolzhenko 2015 Abstract The recent advances in sensor and display technologies have brought upon the High Dynamic Range (HDR) imaging capability. The modern multiple exposure HDR sensors can achieve the dynamic range of 100-120 dB and LED and OLED display devices have contrast ratios of 105:1 to 106:1. Despite the above advances in technology the image/video compression algorithms and associated hardware are yet based on Standard Dynamic Range (SDR) technology, i.e. they operate within an effective dynamic range of up to 70 dB for 8 bit gamma corrected images. Further the existing infrastructure for content distribution is also designed for SDR, which creates interoperability problems with true HDR capture and display equipment. The current solutions for the above problem include tone mapping the HDR content to fit SDR. However this approach leads to image quality associated problems, when strong dynamic range compression is applied. Even though some HDR-only solutions have been proposed in literature, they are not interoperable with current SDR infrastructure and are thus typically used in closed systems. Given the above observations a research gap was identified in the need for efficient algorithms for the compression of still images and video, which are capable of storing full dynamic range and colour gamut of HDR images and at the same time backward compatible with existing SDR infrastructure. To improve the usability of SDR content it is vital that any such algorithms should accommodate different tone mapping operators, including those that are spatially non-uniform. In the course of the research presented in this thesis a novel two layer CODEC architecture is introduced for both HDR image and video coding. Further a universal and computationally efficient approximation of the tone mapping operator is developed and presented. It is shown that the use of perceptually uniform colourspaces for internal representation of pixel data enables improved compression efficiency of the algorithms. Further proposed novel approaches to the compression of metadata for the tone mapping operator is shown to improve compression performance for low bitrate video content. Multiple compression algorithms are designed, implemented and compared and quality-complexity trade-offs are identified. Finally practical aspects of implementing the developed algorithms are explored i by automating the design space exploration flow and integrating the high level systems design framework with domain specific tools for synthesis and simulation of multiprocessor systems. The directions for further work are also presented. Vladimir Dolzhenko, 31st May 2015 ii Acknowledgements I would like to express my special appreciation to my advisor Dr Eran Edirisinghe who has been tremendous mentor to me over all these years. Without his help, advice and encouragement completing this work would not be possible. I would like to thank Dr Helmut Bez for his course and guidance during annual reviews. I would like to thank my sponsoring company Apical Ltd, its CEO Dr Michael Tusch and the industrial supervisor Dr Slava Chesnokov for all the help with tuition fees and for providing priceless experience of understanding and designing imaging algorithms and complex systems. Thanks to their understanding at the times when the University assignments clashed with the hard deadlines. I would like to thank all the staff of Loughborough University Systems Engineering Doctorate Centre for brilliant management of the programme and for providing world class training in systems engineering. Specific thanks to Dr Stuart Burdges for excellent introduction to Systems Engineering. I am thankful to all my fellow students, it was fun time together and I have learned a lot from you. A special thanks to my family. I am very grateful to my mother-in-law, father-in-law, my mother and father for all their help, encouragement and understanding. I would like to specifically thank my beloved wife Anastasia Lazareva for all her help, encouragement and belief in me. It is because of you I did not give up on this work. After all the sacrifices you did on my behalf it is your thesis as well. Vladimir Dolzhenko, 31st May 2015 iii Table of contents Abstract ....................................................................................................................................... i Acknowledgements .................................................................................................................. iii Table of contents ....................................................................................................................... iv Abbreviations ........................................................................................................................... vii List of figures ............................................................................................................................ xi List of tables ............................................................................................................................. xii Chapter 1 Introduction .......................................................................................................... 1 1.1 Research problem ........................................................................................................ 1 1.2 Sponsoring company ................................................................................................... 2 1.3 Research aim and objectives ....................................................................................... 2 1.4 Chapter description ..................................................................................................... 3 1.5 Research contribution .................................................................................................. 4 Chapter 2 Background .......................................................................................................... 6 2.1 Introduction ................................................................................................................. 6 2.2 H.264/AVC Video coding standard ............................................................................ 6 2.3 Scalable Video Coding. ............................................................................................... 8 2.4 H.265 High efficiency video coding ......................................................................... 11 2.5 High dynamic range imaging .................................................................................... 14 2.6 Complexity of the compression algorithms .............................................................. 15 Chapter 3 Literature Review............................................................................................... 17 3.1 Introduction ............................................................................................................... 17 3.2 HDR image compression .......................................................................................... 18 3.3 HDR video compression ........................................................................................... 21 3.4 Alternative compression schemes ............................................................................. 24 3.5 System design ............................................................................................................ 25 Chapter 4 High Dynamic Range Still Image Compression ................................................ 29 iv 4.1 Introduction. .............................................................................................................. 29 4.2 Overall codec structure.............................................................................................. 30 4.3 Experimental setup .................................................................................................... 37 4.4 Experimental results .................................................................................................. 40 4.5 Conclusion ................................................................................................................. 43 Chapter 5 High Dynamic Range Video Compression ........................................................ 45 5.1 Introduction ............................................................................................................... 45 5.2 Overview of the proposed method ............................................................................ 45 5.3 The proposed CODEC .............................................................................................. 46 5.4 Experimental setup .................................................................................................... 51 5.5 Experimental results .................................................................................................. 53 5.6 Conclusion ................................................................................................................. 55 Chapter 6 Integration of the SysML Design Framework with a Domain-Specific Modelling System .................................................................................................................... 56 6.1 Introduction ..............................................................................................................
Load more

Recommended publications
  • Digital Video Quality Handbook (May 2013
    Digital Video Quality Handbook May 2013 This page intentionally left blank. Executive Summary Under the direction of the Department of Homeland Security (DHS) Science and Technology Directorate (S&T), First Responders Group (FRG), Office for Interoperability and Compatibility (OIC), the Johns Hopkins University Applied Physics Laboratory (JHU/APL), worked with the Security Industry Association (including Steve Surfaro) and members of the Video Quality in Public Safety (VQiPS) Working Group to develop the May 2013 Video Quality Handbook. This document provides voluntary guidance for providing levels of video quality in public safety applications for network video surveillance. Several video surveillance use cases are presented to help illustrate how to relate video component and system performance to the intended application of video surveillance, while meeting the basic requirements of federal, state, tribal and local government authorities. Characteristics of video surveillance equipment are described in terms of how they may influence the design of video surveillance systems. In order for the video surveillance system to meet the needs of the user, the technology provider must consider the following factors that impact video quality: 1) Device categories; 2) Component and system performance level; 3) Verification of intended use; 4) Component and system performance specification; and 5) Best fit and link to use case(s). An appendix is also provided that presents content related to topics not covered in the original document (especially information related to video standards) and to update the material as needed to reflect innovation and changes in the video environment. The emphasis is on the implications of digital video data being exchanged across networks with large numbers of components or participants.
    [Show full text]
  • (A/V Codecs) REDCODE RAW (.R3D) ARRIRAW
    What is a Codec? Codec is a portmanteau of either "Compressor-Decompressor" or "Coder-Decoder," which describes a device or program capable of performing transformations on a data stream or signal. Codecs encode a stream or signal for transmission, storage or encryption and decode it for viewing or editing. Codecs are often used in videoconferencing and streaming media solutions. A video codec converts analog video signals from a video camera into digital signals for transmission. It then converts the digital signals back to analog for display. An audio codec converts analog audio signals from a microphone into digital signals for transmission. It then converts the digital signals back to analog for playing. The raw encoded form of audio and video data is often called essence, to distinguish it from the metadata information that together make up the information content of the stream and any "wrapper" data that is then added to aid access to or improve the robustness of the stream. Most codecs are lossy, in order to get a reasonably small file size. There are lossless codecs as well, but for most purposes the almost imperceptible increase in quality is not worth the considerable increase in data size. The main exception is if the data will undergo more processing in the future, in which case the repeated lossy encoding would damage the eventual quality too much. Many multimedia data streams need to contain both audio and video data, and often some form of metadata that permits synchronization of the audio and video. Each of these three streams may be handled by different programs, processes, or hardware; but for the multimedia data stream to be useful in stored or transmitted form, they must be encapsulated together in a container format.
    [Show full text]
  • On the Accuracy of Video Quality Measurement Techniques
    On the accuracy of video quality measurement techniques Deepthi Nandakumar Yongjun Wu Hai Wei Avisar Ten-Ami Amazon Video, Bangalore, India Amazon Video, Seattle, USA Amazon Video, Seattle, USA Amazon Video, Seattle, USA, [email protected] [email protected] [email protected] [email protected] Abstract —With the massive growth of Internet video and therefore measuring and optimizing for video quality in streaming, it is critical to accurately measure video quality these high quality ranges is an imperative for streaming service subjectively and objectively, especially HD and UHD video which providers. In this study, we lay specific emphasis on the is bandwidth intensive. We summarize the creation of a database measurement accuracy of subjective and objective video quality of 200 clips, with 20 unique sources tested across a variety of scores in this high quality range. devices. By classifying the test videos into 2 distinct quality regions SD and HD, we show that the high correlation claimed by objective Globally, a healthy mix of devices with different screen sizes video quality metrics is led mostly by videos in the SD quality and form factors is projected to contribute to IP traffic in 2021 region. We perform detailed correlation analysis and statistical [1], ranging from smartphones (44%), to tablets (6%), PCs hypothesis testing of the HD subjective quality scores, and (19%) and TVs (24%). It is therefore necessary for streaming establish that the commonly used ACR methodology of subjective service providers to quantify the viewing experience based on testing is unable to capture significant quality differences, leading the device, and possibly optimize the encoding and delivery to poor measurement accuracy for both subjective and objective process accordingly.
    [Show full text]
  • The H.264 Advanced Video Coding (AVC) Standard
    Whitepaper: The H.264 Advanced Video Coding (AVC) Standard What It Means to Web Camera Performance Introduction A new generation of webcams is hitting the market that makes video conferencing a more lifelike experience for users, thanks to adoption of the breakthrough H.264 standard. This white paper explains some of the key benefits of H.264 encoding and why cameras with this technology should be on the shopping list of every business. The Need for Compression Today, Internet connection rates average in the range of a few megabits per second. While VGA video requires 147 megabits per second (Mbps) of data, full high definition (HD) 1080p video requires almost one gigabit per second of data, as illustrated in Table 1. Table 1. Display Resolution Format Comparison Format Horizontal Pixels Vertical Lines Pixels Megabits per second (Mbps) QVGA 320 240 76,800 37 VGA 640 480 307,200 147 720p 1280 720 921,600 442 1080p 1920 1080 2,073,600 995 Video Compression Techniques Digital video streams, especially at high definition (HD) resolution, represent huge amounts of data. In order to achieve real-time HD resolution over typical Internet connection bandwidths, video compression is required. The amount of compression required to transmit 1080p video over a three megabits per second link is 332:1! Video compression techniques use mathematical algorithms to reduce the amount of data needed to transmit or store video. Lossless Compression Lossless compression changes how data is stored without resulting in any loss of information. Zip files are losslessly compressed so that when they are unzipped, the original files are recovered.
    [Show full text]
  • AVC, HEVC, VP9, AVS2 Or AV1? — a Comparative Study of State-Of-The-Art Video Encoders on 4K Videos
    AVC, HEVC, VP9, AVS2 or AV1? | A Comparative Study of State-of-the-art Video Encoders on 4K Videos Zhuoran Li, Zhengfang Duanmu, Wentao Liu, and Zhou Wang University of Waterloo, Waterloo, ON N2L 3G1, Canada fz777li,zduanmu,w238liu,[email protected] Abstract. 4K, ultra high-definition (UHD), and higher resolution video contents have become increasingly popular recently. The largely increased data rate casts great challenges to video compression and communication technologies. Emerging video coding methods are claimed to achieve su- perior performance for high-resolution video content, but thorough and independent validations are lacking. In this study, we carry out an in- dependent and so far the most comprehensive subjective testing and performance evaluation on videos of diverse resolutions, bit rates and content variations, and compressed by popular and emerging video cod- ing methods including H.264/AVC, H.265/HEVC, VP9, AVS2 and AV1. Our statistical analysis derived from a total of more than 36,000 raw sub- jective ratings on 1,200 test videos suggests that significant improvement in terms of rate-quality performance against the AVC encoder has been achieved by state-of-the-art encoders, and such improvement is increas- ingly manifest with the increase of resolution. Furthermore, we evaluate state-of-the-art objective video quality assessment models, and our re- sults show that the SSIMplus measure performs the best in predicting 4K subjective video quality. The database will be made available online to the public to facilitate future video encoding and video quality research. Keywords: Video compression, quality-of-experience, subjective qual- ity assessment, objective quality assessment, 4K video, ultra-high-definition (UHD), video coding standard 1 Introduction 4K, ultra high-definition (UHD), and higher resolution video contents have en- joyed a remarkable growth in recent years.
    [Show full text]
  • Video Quality Assessment: Subjective Testing of Entertainment Scenes Margaret H
    1 Video Quality Assessment: Subjective testing of entertainment scenes Margaret H. Pinson, Lucjan Janowski, and Zdzisław Papir This article describes how to perform a video quality subjective test. For companies, these tests can greatly facilitate video product development; for universities, removing perceived barriers to conducting such tests allows expanded research opportunities. This tutorial assumes no prior knowledge and focuses on proven techniques. (Certain commercial equipment, materials, and/or programs are identified in this article to adequately specify the experimental procedure. In no case does such identification imply recommendation or endorsement by the National Telecommunications and Information Administration, nor does it imply that the program or equipment identified is necessarily the best available for this application.) Video is a booming industry: content is embedded on many Web sites, delivered over the Internet, and streamed to mobile devices. Cisco statistics indicate that video exceeded 50% of total mobile traffic for the first time in 2012, and predicts that two-thirds of the world’s mobile data traffic will be video by 2017 [1]. Each company must make a strategic decision on the correct balance between delivery cost and user experience. This decision can be made by the engineers designing the service or, for increased accuracy, by consulting users [2]. Video quality assessment requires a combined approach that includes objective metrics, subjective testing, and live video monitoring. Subjective testing is a critical step to ensuring success in operational management and new product development. Carefully conducted video quality subjective tests are extremely reliable and repeatable, as is shown in Section 8 of [3]. This article provides an approachable tutorial on how to conduct a subjective video quality experiment.
    [Show full text]
  • Using H.264 and H.265 Codecs for 4K Transmissions
    ISSN 1843-6188 Scientific Bulletin of the Electrical Engineering Faculty – Year 15 No.1 (29) OBJECTIVE VIDEO QUALITY ASSESMENT: USING H.264 AND H.265 CODECS FOR 4K TRANSMISSIONS NICOLETA ANGELESCU Department of Electronics, Telecommunications and Energetics, Valahia University of Targoviste E-mail: [email protected] Abstract. H.265 codecs has attracted a lot of attention solution can be applied only to MPEG 2 video files to for transmission of digital video content with 4k reduce storage space. In [5], the H.265 and H.264 codecs resolution (3840x2160 pixels), since offers necessary are evaluated on video quality for 4k resolution. The bandwidths saving in comparison with H.264 codec. We authors use three objective quality metrics for evaluation. compare both codecs using three objective video quality The authors conclude that H.265 codec offer the best metrics on five standard 4k video test sequences for improvement comparing to H.264 codec for low bit-rate. digital TV transmissions using both codecs. In [6], a solution to reduce bit rate for MPEG 2 and H.264 transmission is proposed. The base layer at MPEG Keywords: H.264, HECV, VQM, SSIM, objective video 2 standard resolution is multiplexed with the enhanced quality layer (which is the difference between scaled MPEG 2 video content and original high resolution video captured) compressed with H.264 codec and transmitted to user 1. INTRODUCTION destination. The same solution can be applied for H.264 and H.265 content. The users with receivers which Nowadays, the majority of video content is stored in support Full HD resolution view the 4K content scaled lossy compressed form.
    [Show full text]
  • On Evaluating Perceptual Quality of Online User-Generated Videos Soobeom Jang, and Jong-Seok Lee, Senior Member, IEEE
    JOURNAL OF LATEX CLASS FILES, VOL. 13, NO. 9, SEPTEMBER 2014 1 On Evaluating Perceptual Quality of Online User-Generated Videos Soobeom Jang, and Jong-Seok Lee, Senior Member, IEEE Abstract—This paper deals with the issue of the perceptual quality assessment by employing multiple human subjects. quality evaluation of user-generated videos shared online, which However, subjective quality assessment is not feasible for is an important step toward designing video-sharing services online videos because of a tremendous amount of videos that maximize users’ satisfaction in terms of quality. We first analyze viewers’ quality perception patterns by applying graph in online video-sharing services. An alternative is objective analysis techniques to subjective rating data. We then examine quality assessment, which uses a model that mimics the human the performance of existing state-of-the-art objective metrics for perceptual mechanism. the quality estimation of user-generated videos. In addition, we The traditional objective quality metrics to estimate video investigate the feasibility of metadata accompanied with videos quality have two limitations. First, the existence of the ref- in online video-sharing services for quality estimation. Finally, various issues in the quality assessment of online user-generated erence video, i.e., the pristine video of the given video, is videos are discussed, including difficulties and opportunities. important. In general, objective quality assessment frameworks are classified into three groups: full-reference (FR), reduced- Index Terms—User-generated video, paired comparison, qual- ity assessment, metadata. reference (RR), and no-reference (NR). In the cases of FR and RR frameworks, full or partial information about the reference is provided.
    [Show full text]
  • Video Resolution Enhancement Using Deep Neural Networks and Intensity Based Registrations
    International Journal of Innovative Computing, Information and Control ICIC International ⃝c 2018 ISSN 1349-4198 Volume 14, Number 5, October 2018 pp. 1969{1976 VIDEO RESOLUTION ENHANCEMENT USING DEEP NEURAL NETWORKS AND INTENSITY BASED REGISTRATIONS Gholamreza Anbarjafari1;2 1iCV Research Group Institute of Technology University of Tartu Nooruse Street, Tartu 50411, Estonia [email protected] 2Faculty of Engineering Hasan Kalyoncu University Gaziantep, Turkey Received January 2018; revised May 2018 Abstract. Thanks to the recent rapid improvements made to the maximum possible resolution of display devices, higher qualities of experience have been made possible, which necessitates either producing and transmitting considerably higher volumes of data or super-resolving lower-resolution contents at the display side, where the former might not be practically feasible. Therefore, aiming at the latter, this paper proposes a novel super-resolution technique, which takes advantage of convolutional neural networks. Each image is registered into a window consisting of two frames, the second one standing for the reference image, using various intensity-based techniques, which have been tested and compared throughout the paper. According to the experimental results, the proposed method leads to substantial enhancements in the quality of the super-resolved images, compared with the state-of-the-art techniques introduced within the existing literature. On the Akiyo video sequence, on average, the result possesses 5.38dB higher PSNR values than those of the Vandewalle registration technique, with structure adaptive normalised convolution being utilized as the reconstruction approach. Keywords: Super resolution, Deep learning, Convolutional neural network, Intensity based registration, Video resolution enhancement 1. Introduction. The issue of high quality image reconstruction from the low quality version of the same image or the sequence of similar video frames has been extensively studied for several decades, and it is still an ongoing research [1, 2, 3, 4, 5, 6].
    [Show full text]
  • Video Quality Assessment Using a Statistical Model of Human Visual Speed Perception
    Video Quality Assessment Using a Statistical Model of Human Visual Speed Perception Zhou Wang and Qiang Li Dept. of Electrical & Computer Engineering, University of Waterloo, ON N2L3G1, Canada Dept. of Electrical Engineering, The University of Texas at Arlington, TX 76019, USA [email protected]; [email protected] Motion is one of the most important types of information contained in natu- ral video, but direct use of motion information in the design of video quality assessment algorithms has not been deeply investigated. Here we propose to in- corporate a recent model of human visual speed perception [Stocker & Simon- celli, Nature Neuroscience 9, 578-585 (2006)] and model visual perception in an information communication framework. This allows us to estimate both the motion information content and the perceptual uncertainty in video signals. Improved video quality assessment algorithms are obtained by incorporating the model as spatiotemporal weighting factors, where the weight increases with the information content and decreases with the perceptual uncertainty. Consistent improvement over existing video quality assessment algorithms is observed in our validation with the video quality experts group Phase I test data set. °c 2007 Optical Society of America OCIS codes: 110.3000, 330.5510, 330.4060, 100.2960. 1. Introduction The capability of representing motion is probably the most critical characteristic that dis- tinguishes a natural video sequence from a stack of independent still image frames. If we believe that the central goal of vision is to extract useful information from the visual scene, then the perception of motion information would play an important role in the perception of natural video.
    [Show full text]
  • Subjective Analysis of Video Quality on Mobile Devices
    SUBJECTIVE ANALYSIS OF VIDEO QUALITY ON MOBILE DEVICES Anush K. Moorthy, Lark K. Choi, Gustavo de Veciana and Alan C. Bovik Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, Tx - 78712, USA. ABSTRACT Some video quality studies have also been conducted on mobile devices [9–13]. However, almost all of these studies suffer from We summarize a recent mobile video quality assessment (VQA) several limitations: (1) small datasets, (2) insufficient distortions and database that we have created which incorporates a wide variety of distortion severities, (3) unknown sources for the reference videos distortions including uniform compression and wireless packet loss, used with unknown corruptions, (4) small video resolutions and (5) along with dynamically varying distortions that change as a user lack of public availability. views a video on his mobile device. We evaluate a large number The LIVE Mobile VQA database that we describe here aims to of objective image and video quality assessment (IQA/VQA) algo- overcome the above limitations. This adequate and modern database rithms with regards to their efficacy in predicting visual quality on will aid the development of perceptually optimized algorithms for these videos on such small form-factor displays. A detailed corre- wireless video transmission and serve as a testing ground for VQA lation analysis and statistical hypothesis testing is carried out. Our algorithms. The LIVE Mobile VQA database consists of 200 dis- results indicate that current algorithms for IQA and VQA are not torted videos evaluated by over 30 human subjects on a small mo- well-equipped to handle time-varying distortions and need to be aug- bile screen.
    [Show full text]
  • COMPRESSIVE TECHNIQUES APPLICABLE for VIDEO COMPRESSION Jayshree R
    COMPRESSIVE TECHNIQUES APPLICABLE FOR VIDEO COMPRESSION Jayshree R. Pansare1, Ketki R. Jadhav2 1,2 Department of Computer Engineering, M.E.S. College of Engineering, Pune, India Abstract there lot of techniques for compression of Communication over internet is becoming images/ videos, finding better compression important part of our lives. Now days techniques for diversified applications or millions of peoples spend their time on you improving performance of existing techniques is tube, video communication and video still concern of lot of researchers. conferencing etc. Videos require large Images/videos are either get compressed by amount of bandwidth and transmission time. using lossless compression methods consist of Lot of video compression standards, Run length encoding (RLE), LZW ( Lempel Ziv- techniques and algorithms had been Welch) Coding, Huffman coding, Area Coding developed to reduce data quantity and gain or using lossy compression techniques consist of expected quality as possible as can. Video Transformation coding, Fractal segmentation, compressions technologies are now become Vector quantization, Sub band coding and Block the important part of the way we create, truncation coding. Even though most of us want consume and communicate visual image representation of content rather than text information. Paper reviews no of video for better understanding and visual appearance, compression techniques. It divides those videos turned out to be standards now days and techniques based on used techniques and used used by lot of applications such as DVD, digital concepts in survey papers. And shows that for TV, HDTV, Video calls and teleconferencing. which type of videos these techniques are Because of lot of advancement in network useful.
    [Show full text]