The AV1 Constrained Directional Enhancement Filter (CDEF)

Total Page：16

File Type：pdf, Size：1020Kb

The AV1 Constrained Directional Enhancement Filter (CDEF) Steinar Midtskogen Cisco Jean-Marc Valin Mozilla ICASSP 2018 The AV1 Video Codec ● Royalty-free licensing ● Created by the Alliance for Open Media ● Officially released March 28th 2018 ● Based on VP9 (Google) with technology from Thor (Cisco) and Daala (Mozilla) Coding loop (enhancement blocks) Constrained Directional Deblocking Loop Enhancement Filter Filter Restoration (CDEF) CDEF Overview ● Applied after deblocking filter, in coding loop ● Reduces ringing (and other) artifacts ● Low hardware and software complexity ● Main ideas – Non-linear filter – Direction search – Direction-adaptive taps – Applied to both luma and chroma Non-Linear Filter ● Blurs ringing while preserving edges – Behaves like an FIR at low contrast – Ignores large contrasts (edges) ● Fully vectorizable, no division Output Center Weight Non-linear Pixel Filter value value function difference parameters Constraint Function ● Parameterized by strength and damping – Trade-off between ringing removal and blurring – Typically use higher strength at lower bitrate Strength: end of linear region Damping: point of zero output Direction Estimation ● Runs on 8x8 decoded blocks (no signaling) – Small enough for tracking curves – Large enough to reliably estimate direction ● Find direction that minimizes “prediction” error – Fully vectorizable Directional Filter ● Sum of two direction-dependent sets of taps – Primary taps along direction (higher strength) – Secondary taps off direction (lower strength) Primary taps Secondary taps Example Example Example Directions Example Directions 8 x 8 Example Directions 8 x 8 Filter X Primary Secondary Example Example (Before) Example (After) Signaling ● Limited side information ● Two levels of signaling – Frame-level list of 1-8 presets – Preset selection at 64x64 level (0-3 bits) ● No 8x8 signaling, no direction signaling ● No signaling when 64x64 filter block is skipped Results ● PSNR BD-rate improvement – 1.1% for high-latency, 3.7% for low-latency ● Significant subjective improvement (HL) .

Copy Link

Recommended publications

High Efficiency, Moderate Complexity Video Codec Using Only RF IPR
Thor High Efficiency, Moderate Complexity Video Codec using only RF IPR draft-fuldseth-netvc-thor-00 Arild Fuldseth, Gisle Bjontegaard (Cisco) IETF 93 – Prague, CZ – July 2015 1 Design principles • Moderate complexity to allow real-time implementation in SW on common HW, as well as new HW designs • Basic building blocks from well-known hybrid approach (motion compensated prediction and transform coding) • Common design elements in modern codecs – Larger block sizes and transforms, up to 64x64 – Quarter pixel interpolation, motion vector prediction, etc. • Cisco RF IPR (note well: declaration filed on draft) – Deblocking, transforms, etc. (some also essential in H.265/4) • Avoid non-RF IPR – If/when others offer RF IPR, design/performance will improve 2 Encoder Architecture Input Transform Quantizer Entropy Output video Coding bitstream - Inverse Transform Intra Frame Prediction Loop filters Inter Frame Prediction Reconstructed Motion Frame Estimation Memory 3 Decoder Architecture Input Entropy Inverse Bitstream Decoding Transform Intra Frame Prediction Loop filters Inter Frame Prediction Output video Reconstructed Frame Memory 4 Block Structure • Super block (SB) 64x64 • Quad-tree split into coding blocks (CB) >= 8x8 • Multiple prediction blocks (PB) per CB • Intra: 1 PB per CB • Inter: 1, 2 (rectangular) or 4 (square) PBs per CB • 1 or 4 transform blocks (TB) per CB 5 Coding-block modes • Intra • Inter0 MV index, no residual information • Inter1 MV index, residual information • Inter2 Explicit motion vector information, residual information
[Show full text]
Hardware for Speech and Audio Coding
Linköping Studies in Science and Technology Thesis No. 1093 Hardware for Speech and Audio Coding Mikael Olausson LiU-TEK-LIC-2004:22 Department of Electrical Engineering Linköpings universitet, SE-581 83 Linköping, Sweden Linköping 2004 Linköping Studies in Science and Technology Thesis No. 1093 Hardware for Speech and Audio Coding Mikael Olausson LiU-TEK-LIC-2004:22 Department of Electrical Engineering Linköpings universitet, SE-581 83 Linköping, Sweden Linköping 2004 ISBN 91-7373-953-7 ISSN 0280-7971 ii Abstract While the Micro Processors (MPUs) as a general purpose CPU are converging (into Intel Pentium), the DSP processors are diverging. In 1995, approximately 50% of the DSP processors on the market were general purpose processors, but last year only 15% were general purpose DSP processors on the market. The reason general purpose DSP processors fall short to the application specific DSP processors is that most users want to achieve highest performance under mini- mized power consumption and minimized silicon costs. Therefore, a DSP proces- sor must be an Application Specific Instruction set Processor (ASIP) for a group of domain specific applications. An essential feature of the ASIP is its functional acceleration on instruction level, which gives the specific instruction set architecture for a group of appli- cations. Hardware acceleration for digital signal processing in DSP processors is essential to enhance the performance while keeping enough flexibility. In the last 20 years, researchers and DSP semiconductor companies have been working on different kinds of accelerations for digital signal processing. The trade-off be- tween the performance and the flexibility is always an interesting question because all DSP algorithms are "application specific"; the acceleration for audio may not be suitable for the acceleration of baseband signal processing.
[Show full text]
Efficient Multi-Codec Support for OTT Services: HEVC/H.265 And/Or AV1?
Efficient Multi-Codec Support for OTT Services: HEVC/H.265 and/or AV1? Christian Timmerer†,‡, Martin Smole‡, and Christopher Mueller‡ ‡Bitmovin Inc., †Alpen-Adria-Universität San Francisco, CA, USA and Klagenfurt, Austria, EU ‡{firstname.lastname}@bitmovin.com, †{firstname.lastname}@itec.aau.at Abstract – The success of HTTP adaptive streaming is un- multiple versions (e.g., different resolutions and bitrates) and disputed and technical standards begin to converge to com- each version is divided into predefined pieces of a few sec- mon formats reducing market fragmentation. However, other onds (typically 2-10s). A client first receives a manifest de- obstacles appear in form of multiple video codecs to be sup- scribing the available content on a server, and then, the client ported in the future, which calls for an efficient multi-codec requests pieces based on its context (e.g., observed available support for over-the-top services. In this paper, we review the bandwidth, buffer status, decoding capabilities). Thus, it is state of the art of HTTP adaptive streaming formats with re- able to adapt the media presentation in a dynamic, adaptive spect to new services and video codecs from a deployment way. perspective. Our findings reveal that multi-codec support is The existing different formats use slightly different ter- inevitable for a successful deployment of today's and future minology. Adopting DASH terminology, the versions are re- services and applications. ferred to as representations and pieces are called segments, which we will use henceforth. The major differences between INTRODUCTION these formats are shown in Table 1. We note a strong differ- entiation in the manifest format and it is expected that both Today's over-the-top (OTT) services account for more than MPEG's media presentation description (MPD) and HLS's 70 percent of the internet traffic and this number is expected playlist (m3u8) will coexist at least for some time.
[Show full text]
CALIFORNIA STATE UNIVERSITY, NORTHRIDGE Optimized AV1 Inter
CALIFORNIA STATE UNIVERSITY, NORTHRIDGE Optimized AV1 Inter Prediction using Binary classification techniques A graduate project submitted in partial fulfillment of the requirements for the degree of Master of Science in Software Engineering by Alex Kit Romero May 2020 The graduate project of Alex Kit Romero is approved: ____________________________________ ____________ Dr. Katya Mkrtchyan Date ____________________________________ ____________ Dr. Kyle Dewey Date ____________________________________ ____________ Dr. John J. Noga, Chair Date California State University, Northridge ii Dedication This project is dedicated to all of the Computer Science professors that I have come in contact with other the years who have inspired and encouraged me to pursue a career in computer science. The words and wisdom of these professors are what pushed me to try harder and accomplish more than I ever thought possible. I would like to give a big thanks to the open source community and my fellow cohort of computer science co-workers for always being there with answers to my numerous questions and inquiries. Without their guidance and expertise, I could not have been successful. Lastly, I would like to thank my friends and family who have supported and uplifted me throughout the years. Thank you for believing in me and always telling me to never give up. iii Table of Contents Signature Page ................................................................................................................................ ii Dedication .....................................................................................................................................
[Show full text]
Arxiv:2002.01657V1 [Eess.IV] 5 Feb 2020 Port Lossless Model to Compress Images Lossless
LEARNED LOSSLESS IMAGE COMPRESSION WITH A HYPERPRIOR AND DISCRETIZED GAUSSIAN MIXTURE LIKELIHOODS Zhengxue Cheng, Heming Sun, Masaru Takeuchi, Jiro Katto Department of Computer Science and Communications Engineering, Waseda University, Tokyo, Japan. ABSTRACT effectively in [12, 13, 14]. Some methods decorrelate each Lossless image compression is an important task in the ﬁeld channel of latent codes and apply deep residual learning to of multimedia communication. Traditional image codecs improve the performance as [15, 16, 17]. However, deep typically support lossless mode, such as WebP, JPEG2000, learning based lossless compression has rarely discussed. FLIF. Recently, deep learning based approaches have started One related work is L3C [18] to propose a hierarchical archi- to show the potential at this point. HyperPrior is an effective tecture with 3 scales to compress images lossless. technique proposed for lossy image compression. This paper In this paper, we propose a learned lossless image com- generalizes the hyperprior from lossy model to lossless com- pression using a hyperprior and discretized Gaussian mixture pression, and proposes a L2-norm term into the loss function likelihoods. Our contributions mainly consist of two aspects. to speed up training procedure. Besides, this paper also in- First, we generalize the hyperprior from lossy model to loss- vestigated different parameterized models for latent codes, less compression model, and propose a loss function with L2- and propose to use Gaussian mixture likelihoods to achieve norm for lossless compression to speed up training. Second, adaptive and ﬂexible context models. Experimental results we investigate four parameterized distributions and propose validate our method can outperform existing deep learning to use Gaussian mixture likelihoods for the context model.
[Show full text]
Course Outline & Schedule
Course Outline & Schedule Call US 408-759-5074 or UK +44 20 7620 0033 Open Media Encoding Techniques Course Code PWL396 Duration 3 Day Course Price $2,815 Course Description Video, TV and Image technology today dominates Internet Services. Whether it be for live TV, Streamed movies or video clips within social media video images are everywhere. The long-term efficiency of services depends upon the methods and mechanisms used to encode these services. We have relied upon the developments from the Digital Video Broadcast industry, ISO, MPEG and the ITU to provide us with standard ways to achieve this. However the patent royalty cost is now considered to be holding back this efficient development. All commercial users of the normal encoding used such as H.264, H.265, HEVC and other standardised codecs are required to pay royalties for using this technology through a firm of US patent Lawyers known as MPEG-LA. Each new ITU- T standard encoding requires new and increased payments. The Alliance for Open Media is founded by leading Internet companies focused on developing next-generation media formats, codecs and technologies in the public interest. The new Alliance is committing its collective technology and expertise to meet growing Internet demand for top-quality video, audio, imagery and streaming across devices of all kinds and for users worldwide. The aim is to develop royalty free standardized encoding based upon the technology contributed by its members. This course provides a technical study of Video Coding and the technologies which the developing Open Media implementations are based upon.
[Show full text]
Video Compression Optimized for Racing Drones
Video compression optimized for racing drones Henrik Theolin Computer Science and Engineering, master's level 2018 Luleå University of Technology Department of Computer Science, Electrical and Space Engineering Video compression optimized for racing drones November 10, 2018 Preface To my wife and son always! Without you I'd never try to become smarter. Thanks to my supervisor Staffan Johansson at Neava for providing room, tools and the guidance needed to perform this thesis. To my examiner Rickard Nilsson for helping me focus on the task and reminding me of the time-limit to complete the report. i of ii Video compression optimized for racing drones November 10, 2018 Abstract This thesis is a report on the findings of different video coding tech- niques and their suitability for a low powered lightweight system mounted on a racing drone. Low latency, high consistency and a robust video stream is of the utmost importance. The literature consists of multiple comparisons and reports on the efficiency for the most commonly used video compression algorithms. These reports and findings are mainly not used on a low latency system but are testing in a laboratory environment with settings unusable for a real-time system. The literature that deals with low latency video streaming and network instability shows that only a limited set of each compression algorithms are available to ensure low complexity and no added delay to the coding process. The findings re- sulted in that AVC/H.264 was the most suited compression algorithm and more precise the x264 implementation was the most optimized to be able to perform well on the low powered system.
[Show full text]
Premam 2015 Malayalam 720P Bdrip X264 AC3 51 14GB 53Golkes
1 / 4 Premam [2015] Malayalam 720p BDRip X264 AC3 5.1 1.4GB 53 Jun 11, 2021 — skanda sashti kavacham malayalam skanda sashti kavacham malayalam ... Premam [2015] Malayalam 720p BDRip x264 AC3 5.1 1.4GB 53. Apr 15, 2021 — premam malayalam movie download tamilrockers premam malayalam movie ... Free Movies Download-sites: Here is a list of top 53 websites to download ... Premam [2015] Malayalam 720p BDRip x264 AC3 5.1 1.4GB ESubs .... Dec 1, 2017 — Prev · 43 · 44 · 45 · 46 · 47 · 48 · 49 · 50 · 51 · 52 · 53 · Next; Page 48 of 61. Dookudu (2011) - [Telugu+Hindi+Tamil] - 720p - BDRIP - x264 - AC3 + ESubs ... Orange (2010) - Telugu - 720p - BDRIP - x264 - AC3 5.1 + ESubs - 1.4GB ... PREMAM (2015) - Malayalam - 720p - BRRIP - x264 - AAC + ESubs - 1GB .... Apr 17, 2021 — Download premam malayalam movie download tamilrockers premam ... Premam [2015] Malayalam 720p BDRip x264 AC3 5.1 1.4GB ESubs TorrentPk ... Free Movies Download-sites: Here is a list of top 53 websites to .... Gratuit Ambush Karaoke (2019) VOSTFR BDRIP 1080p regarder en ligne. ... (BDRip 1080p Hi10P DTS) 9AE5EFBC mkv: 9.7 GiB: 2019-09-01 04:53: 9: 1: ... AwardSubsPlease Fruits Basket (2019) S3 - 05 (1080p) 18C540A4 1.4 GB: 1 ... Premam 2015 Malayalam 720p BDRip x264 AC3 5 1 1 4GB mp4: 2019-10-13 10:02.. Results 1 - 15 of 15 — Dobara Full Movie With English Subtitles 720p DOWNLOAD (Mirror #1). ... Premam 2015 Malayalam 720p BDRip X264 AC3 5.1 1.4GB. Premam [2015] Malayalam 720p BDRip X264 AC3 5.1 1.4GB 53 22 Feb, ... Adrishya (2018) 720p Hindi HDRip - x264 - DD 5.1 - 1.4GB - ESub ...
[Show full text]
Sparsity in Linear Predictive Coding of Speech
Sparsity in Linear Predictive Coding of Speech Ph.D. Thesis Daniele Giacobello Multimedia Information and Signal Processing Department of Electronic Systems Aalborg University Niels Jernes Vej 12, 9220 Aalborg Ø, Denmark Sparsity in Linear Predictive Coding of Speech Ph.D. Thesis August 2010 Copyright c 2010 Daniele Giacobello, except where otherwise stated. All rights reserved. Abstract This thesis deals with developing improved techniques for speech coding based on the recent developments in sparse signal representation. In particular, this work is motivated by the need to address some of the limitations of the well- known linear prediction (LP) model currently applied in many modern speech coders. In the first part of the thesis, we provide an overview of Sparse Linear Predic- tion, a set of speech processing tools created by introducing sparsity constraints into the LP framework. This approach defines predictors that look for a sparse residual rather than a minimum variance one with direct applications to coding but also consistent with the speech production model of voiced speech, where the excitation of the all-pole filter can be modeled as an impulse train, i.e., a sparse sequence. Introducing sparsity in the LP framework will also bring to de- velop the concept of high-order sparse predictors. These predictors, by modeling efficiently the spectral envelope and the harmonics components with very few coefficients, have direct applications in speech processing, engendering a joint estimation of short-term and long-term predictors. We also give preliminary results of the effectiveness of their application in audio processing. The second part of the thesis deals with introducing sparsity directly in the linear prediction analysis-by-synthesis (LPAS) speech coding paradigm.
[Show full text]
Lossless and Nearly-Lossless Image Compression Based on Combinatorial Transforms
UNIVERSITÉ DE BOURGOGNE ECOLE DOCTORALE ENVIRONNEMENT - SANTÉ / STIC (E2S) THÈSE Pour obtenir le grade de Docteur de l’université de Bourgogne dicipline : Instrumentation et Informatique de l’Image par Elfitrin SYAHRUL le 29 Juin 2011 Lossless and Nearly-Lossless Image Compression Based on Combinatorial Transforms Directeur de thése : Vincent VAJNOVSZKI Co-encadrant de thése : Julien DUBOIS JURY Abderrafiâa KOUKAM Professeur, UTBM - Belfort Rapporteur Sarifudin MADENDA Professeur, Université de Gunadarma, Indonésie Rapporteur Vlady RAVELOMANANA Professeur, LIAFA Examinateur Michel PAINDAVOINE Professeur, Université de Bourgogne Examinateur Vincent VAJNOVSZKI Professeur, Université de Bourgogne Directeur de thèse Julien DUBOIS Maître de Conférences, Université de Bourgogne Co-encadrant ACKNOWLEDGMENTS In preparing this thesis, I am highly indebted to pass my heartfelt thanks many people who helped me in one way or another. Above all, I would like to appreciate Laboratoire Electronique, Informatique et Image (LE2I) - Université de Bourgogne for valuable facility and support during my research. I also gratefully acknowledge financial support from the French Embassy in Jakarta, Universitas Gunadarma and Indonesian Government. My deepest gratitude is to my advisor, Prof. Vincent Vajnovszki and co-advisor, Julien Dubois. Without their insights and guidance, this thesis would not have been possible. During thesis period, they played a crucial role. Their insights, support and patience allowed me to finish this dissertation. I am blessed with wonderful friends that I could not stated one by one here in many ways, my successes are theirs, too. Most importantly, SYAHRUL family, my parents, dr. Syahrul Zainuddin and Netty Syahrul, my beloved sister Imera Syahrul and my only brother Annilka Syahrul, with their unconditional support, none of this would have been possible.
[Show full text]
Anti-Forensics of Digital Image Compression Matthew C
1050 IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL. 6, NO. 3, SEPTEMBER 2011 Anti-Forensics of Digital Image Compression Matthew C. Stamm, Student Member, IEEE,andK.J.RayLiu, Fellow, IEEE Abstract—As society has become increasingly reliant upon all without relying on an extrinsically inserted watermark or ac- digital images to communicate visual information, a number of cess to the original image. Instead, these techniques make use forensic techniques have been developed to verify the authen- of intrinsic fingerprints introduced into an image by editing op- ticity of digital images. Amongst the most successful of these are techniques that make use of an image’s compression history and erations or the image formation process itself [5]. its associated compression fingerprints. Little consideration has Image compression fingerprints are of particular forensic sig- been given, however, to anti-forensic techniques capable of fooling nificance due the fact that most digital images are subjected to forensic algorithms. In this paper, we present a set of anti-forensic compression either by the camera used to capture them, during techniques designed to remove forensically significant indicators image storage, or for the purposes of digital transmission over of compression from an image. We do this by first developing a generalized framework for the design of anti-forensic techniques the Internet. Techniques have been developed to determine if to remove compression fingerprints from an image’s transform an image saved in a lossless format has ever undergone JPEG coefficients. This framework operates by estimating the distribu- compression [6], [7] or other types of image compression in- tion of an image’s transform coefficients before compression, then cluding wavelet-based techniques [7].
[Show full text]
Arxiv:1602.05975V3 [Cs.MM] 28 Oct 2017 CDEF Works by Identifying the Direction [4] of Each Block Ed =  (Xp − Μd,K) 
THE AV1 CONSTRAINED DIRECTIONAL ENHANCEMENT FILTER (CDEF) Steinar Midtskogen Jean-Marc Valin Cisco Systems Inc. Mozilla Corporation Lysaker, Norway Mountain View, CA, USA [email protected] [email protected] 0 1 2 3 4 5 6 7 0 0 1 1 2 2 3 3 0 0 0 0 0 0 0 0 3 3 2 2 1 1 0 0 ABSTRACT 1 2 3 4 5 6 7 8 1 1 2 2 3 3 4 4 1 1 1 1 1 1 1 1 4 4 3 3 2 2 1 1 2 3 4 5 6 7 8 9 2 2 3 3 4 4 5 5 2 2 2 2 2 2 2 2 5 5 4 4 3 3 2 2 This paper presents the constrained directional enhancement filter 3 4 5 6 7 8 9 10 3 3 4 4 5 5 6 6 3 3 3 3 3 3 3 3 6 6 5 5 4 4 3 3 4 5 6 7 8 9 10 11 4 4 5 5 6 6 7 7 4 4 4 4 4 4 4 4 7 7 6 6 5 5 4 4 designed for the AV1 royalty-free video codec. The in-loop filter 5 6 7 8 9 10 11 12 5 5 6 6 7 7 8 8 5 5 5 5 5 5 5 5 8 8 7 7 6 6 5 5 is based on a non-linear low-pass filter and is designed for vector- 6 7 8 9 10 11 12 13 6 6 7 7 8 8 9 9 6 6 6 6 6 6 6 6 9 9 8 8 7 7 6 6 7 8 9 10 11 12 13 14 7 7 8 8 9 9 10 10 7 7 7 7 7 7 7 7 10 10 9 9 8 8 7 7 ization efficiency.
[Show full text]