Universidade Federal Do Rio Grande Do Sul Instituto De Informática Programa De Pós-Graduação Em Computação

Total Page:16

File Type:pdf, Size:1020Kb

Universidade Federal Do Rio Grande Do Sul Instituto De Informática Programa De Pós-Graduação Em Computação UNIVERSIDADE FEDERAL DO RIO GRANDE DO SUL INSTITUTO DE INFORMÁTICA PROGRAMA DE PÓS-GRADUAÇÃO EM COMPUTAÇÃO FÁBIO LUÍS LIVI RAMOS Efficient High-Throughput and Power-Saving Hardware Architectural Design for the HEVC Entropy Encoder Ph.D. Thesis Thesis presented as partial requirement for the Ph.D. degree in Computer Science Advisor: Prof. Dr. Sergio Bampi Co-Advisor: Prof. Dr. Marcelo Schiavon Porto Porto Alegre 2019 CIP – CATALOGAÇÃO NA PUBLICAÇÃO Ramos, Fábio Luís Livi Efficient High-Throughput and Power-Saving Hardware Architectural Design the HEVC Entropy Encoder / Fábio Luís Livi Ramos. -- 2019. 187 f.:il. Orientador: Sergio Bampi; Co-orientador: Marcelo Schiavon Porto. Tese (Doutorado) – Universidade Federal do Rio Grande do Sul, Instituto de Informática, Programa de Pós-Graduação em Computação, Porto Alegre, BR – RS, 2019. 1. HEVC. 2. CABAC 3.Binary Arithmetic Encoder 4. Real-time Video Processing 5. Power-saving Hardware Design. I. Bampi, Sergio, orient. II. Porto, Marcelo Schiavon, coorient. III. Título. UNIVERSIDADE FEDERAL DO RIO GRANDE DO SUL Reitor: Prof. Rui Vicente Oppermann Vice-Reitor: Prof ª. Jane Fraga Tutikian Pró-Reitor de Pós-Graduação: Prof. Celso Gianetti Loureiro Chaves Diretor do Instituto de Informática: Prof ª. Carla Maria Dal Sasso Freitas Coordenador do PPGC: Prof ª. Luciana Salete Buriol Bibliotecária-Chefe do Instituto de Informática: Beatriz Regina Bastos Haro There are places I'll remember All my life, though some have changed Some forever, not for better Some have gone, and some remain All these places had their moments With lovers and friends, I still can recall Some are dead, and some are living In my life, I've loved them all John Lennon Stop you're trying to bruise my mind I can do it on my own Stop you're trying to kill my time It's been my death since I was born I don't remember half the time If I'm hiding or I'm lost But I'm on my way, on my way Chris Cornell Toda vez que falta luz Toda a vez que algo nos falta O invisível nos salta aos olhos Um salto no escuro da piscina Humberto Gessinger I can’t see, the end of me My whole expanse, I cannot see I formulated infinity Stored deep inside me Curt Kirkwood AGRADECIMENTOS Uma tese de doutorado, apesar de ser, a grosso modo, o trabalho de uma pessoa só, requer uma rede de apoio de pessoas amadas e queridas para dar suporte durante todo o processo. De alguma forma, todas essas pessoas são coautoras desse trabalho, seja no apoio emocional e carinho dados, seja efetivamente em contribuições técnicas para essa tese, seja em ambos. Primeiramente, as duas pessoas mais importantes na minha vida, que são a minha esposa Betina e meu filho Pedro, os quais são o centro do meu universo, sem sombras de dúvidas. Minha esposa, que é, sem dúvida, minha melhor amiga e companheira, que estamos há mais de uma década juntos aprendendo juntos sobre a vida, e a quem eu amo muito! E meu filho, que fez com que minha percepção sobre o que é o amor e sobre o que é a vida mudar de uma forma como nunca antes, por quem eu tento ser um ser humano melhor, e a quem eu amo muito! Obrigado por todo o amor, carinho, ajuda e incentivo!! Sem vocês dois, nada disso teria sido possível e sem vocês dois também não teria valido a pena! Meus pais, Gregório e Rosa, e minha irmã, Renata, são parte fundamental de quem eu sou. Os agradeço por terem me dado uma criação digna, respeito, amor, carinho, por sempre terem estado presentes, sempre terem me incentivado no que tange a questão do ensino, e também pelos (merecidos) puxões de orelha. Hoje percebo como sou privilegiado de ter tido uma família como a que tive, o que nem sempre é verdade no mundo que vivemos. Obrigado por tudo, amos vocês! Meus sogros, Carmo e Leoni, e minha cunhada, Fernanda, que foram como uma segunda família para mim, igualmente sempre me apoiando em tudo desde e época do mestrado, sendo igualmente companheiros e fundamentais em quem eu sou hoje. São pessoas dignas, a quem devo meu reconhecimento e respeito. Obrigado por tudo! Meu orientador, professor Sergio Bampi, pela sua orientação, apoio e respeito ao longo de todo o processo de doutoramento, além de me ensinar pontos muitos importantes que vão além da questão técnica da orientação. Cabe destacar o seu apoio incomensurável durante o começo do doutoramento onde, por questão burocráticas, quase tive que interromper o curso. Graças a sua ajuda (literalmente comprando briga com algumas pessoas para me apoiar) pude seguir no curso da forma como planejada. Meu reconhecimento e agradecimento ao prof. Bampi! Meus co-orientadores Marcelo Porto (oficialmente) e Bruno Zatt (não oficialmente) por todo o apoio, aconselhamento e, por último, mas não menos, pela amizade. Os agradeço profundamente por toda ajuda que me deram ao longo do doutoramento, os eventos durante minhas estadias em Pelotas, e pela parceria e amizade desde a época da graduação/mestrado. As suas orientações, sem dúvida, facilitaram as decisões sobre os caminhos a seguir ao longo do doutoramento! Muito obrigado, gurizada!! A gurizada de “Imbé”, cuja a amizade vem desde a nossa época da graduação na Engenharia de Computação, especialmente: irmãos Zatt (Bruno e Gordo), Osvaldo, Kunz, Giancarlo, gêmeos (Artur e Guilherme), Nondillo e Jonas. Agradeço pela parceria e amizade. Uma pena não podermos nos ver mais tanto quanto antes, visto que cada um mora em uma parte do mundo, mas o elo e amizade continuam firmes e fortes!! Não vejo a hora de podermos nos reunir novamente em Imbé ou onde quer que seja! A gurizada do CEITEC, onde trabalhei por quase cinco anos, e onde formamos um círculo forte de amizade e respeito: Wagston, Hervé, João, Fred, Ferrão, Rohde, Rubinei, Marcelo, Zé, Lauro, Garibotti, Cyrille e Jana. Fico no aguardo para nosso próximo churrasco de confraternização! Além disso, cabe um agradecimento ao Muru e ao Alain, que foram meus gerentes durante o período que lá estive, e foram líderes dignos e justos. Também cabe meu agradecimento aos meus alunos da Unipampa, especialmente meus orientados, e mais especialmente a Camila, a Luana, e o Alessandro, cujos TCCs estiveram envolvidos no escopo dessa tese. Um reconhecimento especial ao pessoal da época do colégio militar, da época do curso de Engenharia de Computação na UFRGS, da época da Datacom, e também da época do mestrado no Laboratório 215, assim como a alguns colegas professores da Unipampa. Muitos eu acabei perdendo o contato, mas ainda tenho carinho e reconhecimento a todos. Devo também um agradecimento a algumas pessoas e entidades, seja por elas serem identitárias a meu respeito, seja por fazerem parte da minha vida de forma intrínseca: as bandas Soundgarden, Beatles, Engenheiros do Hawaii, Alice in Chains, Nirvana, Kyuss, Meat Puppets, Helmet, Raimundos, QOTSA, Pixies, Ween, Faith No More, Red Hot Chili Peppers, Led Zeppelin, Black Sabbath, Silverchair, Foo Fighters, Pearl Jam, Pantera, Rammstein, Offspring, Weezer, para citar as mais importantes. Aos artistas Van Gogh, Raul Seixas, Noel Guarany, Cenair Maicá, Júpiter Maçã, Bert Jansch, Jackson C. Frank, os comediantes do Chaves e do Hermes e Renato. Ao filme Clube da Luta e aos dois primeiros filmes do Predador e Alien. Às universidades UFRGS, Unipampa e UFpel. Às cidades de Porto Alegre, Santa Cruz do Sul, Imbé, São Borja, Canela e Bagé. Uma menção honrosa às bandas Temple of the Dog, Mad Season, Rage Against the Machine, Green Day, Legião Urbana, Titãs, Queen, Creedence Clearwater Revival, Stone Temple Pilots, Red Fang, Stoned Jesus, Morphine, Ace of Base, Banzai e Slayer, e ao artista José Mendes e Michael Jackson. Por fim, dentro da minha atual crença de vida, creio que uma coisa tão complexa como a nossa consciência não suma simplesmente ao morrer. Assim sendo, creio que as coisas podem ter um propósito e espero estar cumprindo, dentro do possível, ao meu. Agradeço por estar vivo, ter saúde, poder trabalhar, estudar, e viver com minha família e amigos momentos bons e ruins, como a vida tem que ser! Sou um privilegiado por tudo! ABSTRACT The advances in digital video processing, such as the new generation of videos resolutions, led to new challenges in order to transmit and storage the related data. In this scenario, real- time digital video processing is an important goal, which requires specific video-processing architectures to accomplish the demanded constraints. Moreover, tethered devices that transmit and receive video draw the attention to power-saving design for these architectures, due to the energy constraints of the battery-based context of these solutions. HEVC (High- Efficiency Video Coding) standard emerges as an alternative to cope with the mentioned situations involving digital video processing. In HEVC, only one entropy-encoding algorithm exists, which is the CABAC (Context-Adaptive Binary Arithmetic Coding). A single high- throughput instance of a hardware CABAC block is a desirable goal in order to save power, area, and coding efficiency. Therefore, the global goal of this research is configurable high- throughput power-efficient single-instance CABAC design, where high-throughput scheme along with power saving techniques are integrated, considering a compromise trade-off between both performance and power/energy dissipation, adapting the architecture according to it. This Thesis focused on the BAE (Binary Arithmetic Encoder) block, which is the processing bottleneck of the CABAC. As a first contribution, a low-power hardware BAE design is presented, where fine-grain insertion of power-saving reduction techniques into different proposed BAE designs, leading to power savings ranging from 10-40% for different BAEs architectural designs. Towards ultra-high throughput performance, the novel Multiple- Bypass Bins Scheme (MBBS) proposition happens within the context presented, where multiple values of a particular type of BAE data (i.e., bypass bins) are processed at the same time.
Recommended publications
  • Network Working Group A
    Network Working Group A. Filippov Internet Draft Huawei Technologies Intended status: Informational A. Norkin Netflix J.R. Alvarez Huawei Technologies Expires: May 17, 2017 November 17, 2016 <Video Codec Requirements and Evaluation Methodology> draft-ietf-netvc-requirements-04.txt Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. The list of current Internet-Drafts can be accessed at http://datatracker.ietf.org/drafts/current/ Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/1id-abstracts.html The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on May 17, 2017. Copyright Notice Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust’s Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with <Filippov> Expires May 17, 2017 [Page 1] Internet-Draft Video Codec Requirements and Evaluation November 2016 respect to this document.
    [Show full text]
  • DVP Tutorial
    IP Video Conferencing: A Tutorial Roman Sorokin, Jean-Louis Rougier Abstract Video conferencing is a well-established area of communications, which have been studied for decades. Recently this area has received a new impulse due to significantly increased bandwidth of Local and Wide area networks, appearance of low-priced video equipment and development of web based media technologies. This paper presents the main techniques behind the modern IP-based videoconferencing services, with a particular focus on codecs, network protocols, architectures and standardization efforts. Questions of security and topologies are also tackled. A description of a typical video conference scenario is provided, demonstrating how the technologies, responsible for different conference aspects, are working together. Traditional industrial disposition as well as modern innovative approaches are both addressed. Current industry trends are highlighted in respect to the topics, described in the tutorial. Legacy analog/digital technologies, together with the gateways between the traditional and the IP videoconferencing systems, are not considered. Keywords Video Conferencing, codec, SVC, MCU, SIP, RTP Roman Sorokin ALE International, Colombes, France e-mail: [email protected] Jean-Louis Rougier Télécom ParisTech, Paris, France e-mail: [email protected] 1 1 Introduction Video conferencing is a two-way interactive communication, delivered over networks of different nature, which allows people from several locations to participate in a meeting. Conference participants use video conferencing endpoints of different types. Generally a video conference endpoint has a camera and a microphone. The video stream, generated by the camera, and the audio stream, coming from the microphone, are both compressed and sent to the network interface.
    [Show full text]
  • High Efficiency, Moderate Complexity Video Codec Using Only RF IPR
    Thor High Efficiency, Moderate Complexity Video Codec using only RF IPR draft-fuldseth-netvc-thor-00 Arild Fuldseth, Gisle Bjontegaard (Cisco) IETF 93 – Prague, CZ – July 2015 1 Design principles • Moderate complexity to allow real-time implementation in SW on common HW, as well as new HW designs • Basic building blocks from well-known hybrid approach (motion compensated prediction and transform coding) • Common design elements in modern codecs – Larger block sizes and transforms, up to 64x64 – Quarter pixel interpolation, motion vector prediction, etc. • Cisco RF IPR (note well: declaration filed on draft) – Deblocking, transforms, etc. (some also essential in H.265/4) • Avoid non-RF IPR – If/when others offer RF IPR, design/performance will improve 2 Encoder Architecture Input Transform Quantizer Entropy Output video Coding bitstream - Inverse Transform Intra Frame Prediction Loop filters Inter Frame Prediction Reconstructed Motion Frame Estimation Memory 3 Decoder Architecture Input Entropy Inverse Bitstream Decoding Transform Intra Frame Prediction Loop filters Inter Frame Prediction Output video Reconstructed Frame Memory 4 Block Structure • Super block (SB) 64x64 • Quad-tree split into coding blocks (CB) >= 8x8 • Multiple prediction blocks (PB) per CB • Intra: 1 PB per CB • Inter: 1, 2 (rectangular) or 4 (square) PBs per CB • 1 or 4 transform blocks (TB) per CB 5 Coding-block modes • Intra • Inter0 MV index, no residual information • Inter1 MV index, residual information • Inter2 Explicit motion vector information, residual information
    [Show full text]
  • Video Compression Optimized for Racing Drones
    Video compression optimized for racing drones Henrik Theolin Computer Science and Engineering, master's level 2018 Luleå University of Technology Department of Computer Science, Electrical and Space Engineering Video compression optimized for racing drones November 10, 2018 Preface To my wife and son always! Without you I'd never try to become smarter. Thanks to my supervisor Staffan Johansson at Neava for providing room, tools and the guidance needed to perform this thesis. To my examiner Rickard Nilsson for helping me focus on the task and reminding me of the time-limit to complete the report. i of ii Video compression optimized for racing drones November 10, 2018 Abstract This thesis is a report on the findings of different video coding tech- niques and their suitability for a low powered lightweight system mounted on a racing drone. Low latency, high consistency and a robust video stream is of the utmost importance. The literature consists of multiple comparisons and reports on the efficiency for the most commonly used video compression algorithms. These reports and findings are mainly not used on a low latency system but are testing in a laboratory environment with settings unusable for a real-time system. The literature that deals with low latency video streaming and network instability shows that only a limited set of each compression algorithms are available to ensure low complexity and no added delay to the coding process. The findings re- sulted in that AVC/H.264 was the most suited compression algorithm and more precise the x264 implementation was the most optimized to be able to perform well on the low powered system.
    [Show full text]
  • Daala: a Perceptually-Driven Still Picture Codec
    DAALA: A PERCEPTUALLY-DRIVEN STILL PICTURE CODEC Jean-Marc Valin, Nathan E. Egge, Thomas Daede, Timothy B. Terriberry, Christopher Montgomery Mozilla, Mountain View, CA, USA Xiph.Org Foundation ABSTRACT and vertical directions [1]. Also, DC coefficients are com- bined recursively using a Haar transform, up to the level of Daala is a new royalty-free video codec based on perceptually- 64x64 superblocks. driven coding techniques. We explore using its keyframe format for still picture coding and show how it has improved over the past year. We believe the technology used in Daala Multi-Symbol Entropy Coder could be the basis of an excellent, royalty-free image format. Most recent video codecs encode information using binary arithmetic coding, meaning that each symbol can only take 1. INTRODUCTION two values. The Daala range coder supports up to 16 values per symbol, making it possible to encode fewer symbols [6]. Daala is a royalty-free video codec designed to avoid tra- This is equivalent to coding up to four binary values in parallel ditional patent-encumbered techniques used in most cur- and reduces serial dependencies. rent video codecs. In this paper, we propose to use Daala’s keyframe format for still picture coding. In June 2015, Daala was compared to other still picture codecs at the 2015 Picture Perceptual Vector Quantization Coding Symposium (PCS) [1,2]. Since then many improve- Rather than use scalar quantization like the vast majority of ments were made to the bitstream to improve its quality. picture and video codecs, Daala is based on perceptual vector These include reduced overlap in the lapped transform, finer quantization (PVQ) [7].
    [Show full text]
  • Arxiv:1602.05975V3 [Cs.MM] 28 Oct 2017 CDEF Works by Identifying the Direction [4] of Each Block Ed =  (Xp − Μd,K) 
    THE AV1 CONSTRAINED DIRECTIONAL ENHANCEMENT FILTER (CDEF) Steinar Midtskogen Jean-Marc Valin Cisco Systems Inc. Mozilla Corporation Lysaker, Norway Mountain View, CA, USA [email protected] [email protected] 0 1 2 3 4 5 6 7 0 0 1 1 2 2 3 3 0 0 0 0 0 0 0 0 3 3 2 2 1 1 0 0 ABSTRACT 1 2 3 4 5 6 7 8 1 1 2 2 3 3 4 4 1 1 1 1 1 1 1 1 4 4 3 3 2 2 1 1 2 3 4 5 6 7 8 9 2 2 3 3 4 4 5 5 2 2 2 2 2 2 2 2 5 5 4 4 3 3 2 2 This paper presents the constrained directional enhancement filter 3 4 5 6 7 8 9 10 3 3 4 4 5 5 6 6 3 3 3 3 3 3 3 3 6 6 5 5 4 4 3 3 4 5 6 7 8 9 10 11 4 4 5 5 6 6 7 7 4 4 4 4 4 4 4 4 7 7 6 6 5 5 4 4 designed for the AV1 royalty-free video codec. The in-loop filter 5 6 7 8 9 10 11 12 5 5 6 6 7 7 8 8 5 5 5 5 5 5 5 5 8 8 7 7 6 6 5 5 is based on a non-linear low-pass filter and is designed for vector- 6 7 8 9 10 11 12 13 6 6 7 7 8 8 9 9 6 6 6 6 6 6 6 6 9 9 8 8 7 7 6 6 7 8 9 10 11 12 13 14 7 7 8 8 9 9 10 10 7 7 7 7 7 7 7 7 10 10 9 9 8 8 7 7 ization efficiency.
    [Show full text]
  • The Internet Needs a Competitive, Royalty-Free Video Codec James Bankoski, Matthew Frost and Adrian Grange
    SIP (2017),vol.6,e13,page1of7©TheAuthors,2017. This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited. doi:10.1017/ATSIP.2017.14 industrial technology advances The internet needs a competitive, royalty-free video codec james bankoski, matthew frost and adrian grange In this paper, we present the argument in favor of an open source, a royalty-free video codec that will keep pace with the evolution of video traffic. Additionally, we argue that the availability of a state-of-the-art, royalty-free codec levels the playing field, allowing small content owners, and application developerso t compete with the larger companies that operate in this space. Keywords: Royalty free, MPEG, Alliance for Open Media, Video compression Received 15 August 2017; Revised 2 November 2017; Accepted 2 November 2017 I. INTRODUCTION II. BACKGROUND Given the central role that the internet plays in modern life, Thefirstdigitalvideocodecstandards,H.120(1984)and internet bandwidth should be considered a scarce resource, H.261(1988),weredevelopedattheInternationalTelecom- demanding careful consideration as to how we consume it munication Union (ITU) and primarily intended for video- to maximize its utility. phone and video-conferencing use-cases. Both standards Global internet traffic is predicted to increase nearly were royalty-free. It was not until the 1995 development of threefold between 2016 and 2021, and the proportion of MPEG-2/H.262 that standards defined video codecs were video traffic is expected to rise from 73 to 82 over the monetized when the US Department of Justice approved same period [1], driven in part by the move of content from MPEG-LA [2], a company unaffiliated with either MPEG broadcast to online delivery channels and to the rapid evo- orISO,toformapatentpoolwiththeaimofcollecting lution of virtual reality applications.
    [Show full text]
  • Network Working Group T. Daede Internet-Draft J. Moffitt Intended Status: Informational Mozilla Expires: April 21, 2016 October 19, 2015
    Network Working Group T. Daede Internet-Draft J. Moffitt Intended status: Informational Mozilla Expires: April 21, 2016 October 19, 2015 Video Codec Testing and Quality Measurement draft-daede-netvc-testing-02 Abstract This document describes guidelines and procedures for evaluating an internet video codec specified at the IETF. This covers subjective and objective tests, test conditions, and materials used for the test. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on April 21, 2016. Copyright Notice Copyright (c) 2015 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust’s Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
    [Show full text]
  • Visual Coding"
    INTERNATIONAL TELECOMMUNICATION UNION STUDY GROUP 16 TELECOMMUNICATION TD 215 R1 (WP3/16) STANDARDIZATION SECTOR STUDY PERIOD 2013-2016 English only Original: English Question(s): 6/16 Geneva, 12-23 October 2015 TD Source: Rapporteur Q6/16 Title: Report of Question 6/16 "Visual coding" CONTENTS 1 RESULTS ................................................................................................................................................................ 2 1.1 SUMMARY ............................................................................................................................................................. 2 1.1.1 Recommendations for Approval .................................................................................................................. 2 1.1.2 Recommendations proposed for Consent in accordance with Rec. A.8. ..................................................... 2 1.1.3 Other documents for Approval .................................................................................................................... 2 1.1.4 Question 6/16 summary .............................................................................................................................. 2 1.2 QUESTION 6/16-VISUAL CODING ........................................................................................................................... 3 1.2.1 Documentation ............................................................................................................................................ 3 1.2.2 Report
    [Show full text]
  • Daala: Building a Next-Generation Video Codec from Unconventional Technology
    Daala: Building A Next-Generation Video Codec From Unconventional Technology Jean-Marc Valin, Timothy B. Terriberry, Nathan E. Egge, Thomas Daede, Yushin Cho, Christopher Montgomery, Michael Bebenita Mozilla, Mountain View, USA Xiph.Org Foundation, USA [email protected] Abstract—Daala is a new royalty-free video codec that attempts 0.6 to compete with state-of-the-art royalty-bearing codecs. To do so, it must achieve good compression while avoiding all of their 0.4 patented techniques. We use technology that is as different as possible from traditional approaches to achieve this. This paper describes the technology behind Daala and discusses where it 0.2 fits in the newly created AV1 codec from the Alliance for Open Media. We show that Daala is approaching the performance level 0 of more mature, state-of-the art video codecs and can contribute Amplitude to improving AV1. -0.2 I. INTRODUCTION -0.4 Daala [1] is a video codec designed to explore a set of atypical techniques, outlined in Section II, to avoid the patent -0.6 0 1 2 3 4 5 6 7 thickets built around most current codecs. Some of these Sample techniques are new to Daala, while others already existed, but are not used in popular standards. Although Daala is not yet Figure 1: 1D synthesis basis functions for the lapped 4 × 4 a competitive codec on its own, some of the techniques it DCT. uses are currently being integrated in the Alliance for Open Media (AOM) [2] codec, AV1, which we discuss in Section III. Section IV presents results obtained with both Daala and AV1.
    [Show full text]
  • Perceptually-Driven Video Coding with the Daala Video Codec
    Perceptually-Driven Video Coding with the Daala Video Codec Yushin Choa,b, Thomas J. Daedea,b, Nathan E. Eggea,b, Guillaume Martresa,b, Tristan Matthewsa,b, Christopher Montgomerya,b, Timothy B. Terriberrya,b, and Jean-Marc Valina,b aXiph.Org Foundation, 21 College Hill Road, Somerville, MA 02144, USA bMozilla Corporation, 331 E. Evelyn Ave., Mountain View, CA 94041, USA ABSTRACT The Daala project is a royalty-free video codec that attempts to compete with the best patent-encumbered codecs. Part of our strategy is to replace core tools of traditional video codecs with alternative approaches, many of them designed to take perceptual aspects into account, rather than optimizing for simple metrics like PSNR. This paper documents some of our experiences with these tools, which ones worked and which did not. We evaluate which tools are easy to integrate into a more traditional codec design, and show results in the context of the codec being developed by the Alliance for Open Media. Keywords: AOM, AV1, Daala, compression, codec, video, perceptual 1. INTRODUCTION Designing a royalty-free video codec is a challenging problem.1 Using tools and techniques that differ substantially from those used by the common commercially-licensed codecs can reduce that challenge. Over the last several years in the Daala project,2 we have experimented with a number of such tools. Some techniques were long known by other researchers,3{5 but not commonly used. However, we thought they might not have been sufficiently developed at the time to be competitive with the usual approaches, or might be worth re-evaluating in a modern context due to changes in engineering trade-offs.
    [Show full text]
  • A Comparison of JEM and AV1 with HEVC: Coding Tools, Coding Efficiency and Complexity
    A Comparison of JEM and AV1 with HEVC: Coding Tools, Coding Efficiency and Complexity Thorsten Laude, Yeremia Gunawan Adhisantoso, Jan Voges, Marco Munderloh, and Jorn¨ Ostermann Institut fur¨ Informationsverarbeitung, Leibniz Universitat¨ Hannover, Germany Email: flaude, gunawan, voges, munderloh, offi[email protected] Abstract—The current state-of-the-art for standardized video as contenders), it is of great interest to assess and compare codecs is High Efficiency Video Coding (HEVC) which was devel- their performance. This comparison can be performed in terms oped jointly by ISO/IEC and ITU-T. Recently, the development of coding efficiency but also in terms of computationally of two contenders for the next generation of standardized video codecs began: ISO/IEC and ITU-T advance the development complexity (run times for the encoding and decoding, as well of the Joint Exploration Model (JEM), a possible successor of as memory requirements). For JEM and HM, a straightfor- HEVC, while the Alliance for Open Media pushes forward the ward comparability is given because both codecs share the video codec AV1. It is asserted by both groups that their codecs same foundation (with JEM being an extension of HM) and achieve superior coding efficiency over the state-of-the-art. In Common Test Conditions are defined to configure both codecs this paper, we discuss the distinguishing features of JEM and AV1 and evaluate their coding efficiency and computational similarly [3]. To include AV1 in a fair comparison is more complexity under well-defined and balanced test conditions. Our challenging because its structure and working principle are main findings are that JEM considerably outperforms HM and fundamentally different compared to HM and JEM.
    [Show full text]