A Comprehensive Video Codec Comparison Thorsten Laude, Yeremia Gunawan Adhisantoso, Jan Voges, Marco Munderloh and Jörn Ostermann
Total Page：16
File Type：pdf, Size：1020Kb
SIP (2019),vol.8,e30,page1of16©TheAuthors,2019. This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work. doi:10.1017/ATSIP.2019.23 overview paper A Comprehensive Video Codec Comparison thorsten laude, yeremia gunawan adhisantoso, jan voges, marco munderloh and jörn ostermann In this paper, we compare the video codecs AV1 (version 1.0.0-2242 from August 2019), HEVC (HM and x265), AVC (x264), theexplorationsoftwareJEMwhichisbasedonHEVC,andtheVVC(successorofHEVC)testmodelVTM(version4.0from February 2019) under two fair and balanced configurations: All Intra for the assessment of intra coding and Maximum Coding Efficiency with all codecs being tuned for their best coding efficiency settings. VTM achieves the highest coding efficiency inboth configurations,followedbyJEMandAV1.Theworstcodingefficiencyisachievedbyx264andx265,evenintheplacebopreset for highest coding efficiency. AV1 gained a lot in terms of coding efficiency compared to previous versions and now outperforms HM by 24 BD-Rate gains. VTM gains 5 over AV1 in terms of BD-Rates. By reporting separate numbers for JVET and AOM test sequences, it is ensured that no bias in the test sequences exists. When comparing only intra coding tools, it is observed that the complexity increases exponentially for linearly increasing coding efficiency. Keywords: AV1, Codec comparison, HEVC, Video coding, VVC Received 28 May 2019; Revised 22 October 2019 I. INTRODUCTION (IETF) as Internet Video Codec (NetVC). The finalization of the standardization process was scheduled for 2017 but For several decades, the market for standardized video initially delayed until the end of 2018 [16]. At the time of codecs was dominated by the standardization groups ISO, writing this manuscript, the official status of NetVC is that IEC, and ITU-T: MPEG-1 [1], MPEG-2/H.262 [2], H.263, the requirements for the standards were finalized in March MPEG-4 Visual [3], and Advanced Video Coding (AVC, 2019 and that the submission of the codec specification for also referred to as MPEG-4 Part 10 and H.264) [4,5]are approval is aspired for December 2019 [17]. some standards in this line. In 2013, the steady improve- Concurrently, ISO/IEC and ITU-T established the Joint mentofvideocodingalgorithmsresultedinHighEfficiency Video Exploration Team (JVET) in October 2015 to explore Video Coding (HEVC) which was standardized as MPEG- technologies for a potential HEVC successor. For this pur- H Part 2 by ISO/IEC and as H.265 by ITU-T [6]. A reference pose, a reference software called Joint Exploration Model implementation of HEVC is available with the HM software (JEM) was developed which includes a variety of novel [7]. Compared to its predecessor standard AVC, HEVC con- coding tools [18]. In the process of the JVET activities, it siderably increases the coding efficiency. Depending on the wasrevealedthatthenewtestmodelprovidessufficient selected configuration, HEVC achieves a 40–60 bit rate evidence to justify to formally start a new standardization reduction while maintaining the same visual quality [8,9]. project [19]. The new standard is referred to as Versatile After the finalization of HEVC, the research for further Video Coding (VVC) and is planned to be finalized in 2020. improvements continued [10,11]. The Versatile Test Model (VTM) [20]wasestablishedto More recently, new participants entered the market for assess the performance of VVC. video codecs. Among the proposed codecs are VP8 [12], The purpose of the reference implementations and test VP9 [13], Daala [14], and Thor [15]. The participants respon- modelsHM,JEM,andVTMistoenabletheevaluation sible for these codecs and many more participants (e.g. of new coding tools and to demonstrate one exemplary Amazon, Facebook, Intel, Microsoft, Netflix) joined their and straight-forward implementation of the correspond- efforts in the Alliance for Open Media (AOM) to develop ing standard. Not much optimization, e.g. for fast encod- the video codec AV1. Furthermore, AV1 is a contender ing, was performed for these implementations. It is safe for standardization by the Internet Engineering Task Force to assume that it is therefore unlikely that these reference implementations will be deployed in real-world products. Institut für Informationsverarbeitung, Leibniz University Hannover, Appelstr. 9A, Instead, highly optimized encoders are used. Therefore, Hannover 30167, Germany we also evaluate the codecs x264 and x265 which imple- Corresponding author: ment the AVC and HEVC standards, respectively. For these Thorsten Laude Email: [email protected] two codecs, two presets are used. The medium preset is a Downloaded from https://www.cambridge.org/core. IP address: 170.106.35.229, on 01 Oct 2021 at 14:18:21, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/ATSIP.2019.23 1 2 thorsten laude et al. typical trade-off between coding efficiency and computa- We structure our answer to this question in the fol- tional resource requirements while the placebo preset max- lowing four parts: choice of codec implementation, codec imizes the coding efficiency at the cost of a considerable configuration, metrics, and test sequences. amount of complexity [21]. Given these eight codec implementations – HM as state-of-the-art, JEM, VTM, and AV1 as contenders, and A) Codec implementations x264 (medium and placebo) as well as x265 (medium and placebo) as optimized encoders – it is of great interest to Thedifficultyofcomparingvideocodecsstartswiththe assess and compare their performance. This comparison difference between video coding standards and particular can be performed in terms of coding efficiency but also in encoder implementations of these standards. The standards terms of computational complexity. are only long text documents which cannot be evaluated For some codecs, e.g. HM and JEM, straightforward in simulations. Only the implementations can be used for comparability is given because both codecs share the same simulations. However, two encoder implementations pro- foundation (with JEM being an extension of HM) and Com- ducing bitstreams compliant with the same standard can mon Test Conditions are defined to configure both codecs be very different. One could distinguish between refer- similarly [22]. To include AV1 or optimized encoders in ence implementations like HM and optimized encoders like a fair comparison is more challenging because their soft- x265. ware structures and working principles are fundamentally different. This also explains why existing comparisons of HEVC with JEM, VP8, VP9, or AV1 in the literature come B) Encoder configurations to different conclusions [13,23,24]. In this paper, we compare the codecs under well-defined Depending on the application and available computational and balanced conditions. First, we analyze the difficulty of resources, encoders can be configured in many different comparing video codecs in Section II.Anoverviewofthe ways.Amongthechoicestobemadearerestrictionsdur- technologies in the codecs is given in Section III.Basedon ing the rate-distortion optimization [32]forpartitioning the analysis in the preceding sections, we introduce our two options to be tested, the decision which coding tools should codec configurations which we use for the comparison in be enabled, and for parameters of the coding tools like Section IV.InSectionV and in Section VI,wecomparethe motion estimation search range. The x264 and x265 imple- performance of the codecs in terms of coding efficiency and mentations allow the configuration of coding tools by pre- complexity, respectively. Section VII concludes the paper. sets. Depending on the selected preset, a different trade-off between computational complexity and coding efficiency is made. When comparing the fastest preset (ultrafast) with II. ON THE DIFFICULTY OF the most efficient preset (placebo), the bit rate can differ by COMPARING VIDEO CODECS 179 for a 720p video encoded at the same quality [21]. Also, the tuning of the encoder can vary, e.g. it can Our motivation for this manuscript emerged at the Pic- be tuned for PSNR or some subjective criterion. Only if ture Coding Symposium (PCS) 2018 where we presented thecodecsaretunedforthesamecriterionandifthis our codec comparison work [24] together with three other criterion corresponds to the metric used for the evaluation, codec comparison works [25–27]. These four works com- theresultsaremeaningful.Thisis,forexample,thecase paredthesamevideocodingstandards.Indoingso,the ifthecodecsaretunedforPSNRandBD-Ratesareusedfor findings of the works are quite different: for example, in one the evaluation. work [24] HEVC is considerably better than AV1 while it is The group of pictures (GOP) structure is an important the other way around in another work [27]. aspect of the encoder configuration as well to ensure a fair The observation of inconclusive results is sustained when comparison. Depending on the available reference pictures, other published works are studied. For example, Feldmann the efficiency of motion-compensated prediction can vary finds that AV1 is up to 43 better than AV1 [28]