DOCSLIB.ORG

Versatile Video Coding Standard

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Versatile Video Coding Standard PAPER UNDER REVIEW, 2021 1 Versatile Video Coding Standard: A Review from Coding Tools to Consumers Deployment Wassim Hamidouche Member, IEEE, Thibaud Biatek, Mohsen Abdoli, Edouard Franc¸ois, Fernando Pescador Senior Member, IEEE, Milosˇ Radosavljevic,´ Daniel Menard and Mickael Raulet Senior Member, IEEE Abstract—The amount of video content and the number of bit resolution) and higher frame rates (100/120 frames per applications based on multimedia information increase each day. second). All of these features must be integrated in devices The development of new video coding standards is a challenge with low resources and limited batteries. Therefore, a balance to increase the compression rate and other important features with a reasonable increase in the computational load. Video between complexity of the algorithms and efficiency in the Experts Team (JVET) of ITU-T and the JCT group within implementations is a challenge in the development of new ISO/IEC have worked together to standardize the Versatile Video consumer electronic devices. Coding, approved finally in July 2020 as ITU-T H.266 — MPEG- Taking into account this situation, the joint collaborative I - Part 3 (ISO/IEC 23090-3) standard. This paper overviews some interesting consumer electronic use cases, the compression team on video coding (JCT-VC) of ITU-T and the JCT tools described in the standard, the current available real time group within ISO/IEC started working in 2010 [4] on the implementations and the first industrial trials done with this development of more efficient video coding standards. An standard. example of the success of this collaboration was the high Index Terms—Encoding/decoding, Applica- efficiency video coding (HEVC) [5] standard. This latter [5] tion/implementation, Versatile Video Coding, Real Time reduces the bit-rate of the previous video standard advanced video codecs. video coding (AVC) [6] in a 50% for similar visual quality [7]. Presently, Versatile Video Coding [8] is the most recent video I. INTRODUCTION standard and therefore the one that defines the current state-of- HE last two decades have witnessed exciting devel- the-art. The challenge of this new video standard is to ensure T opments in Consumer Electronics Applications. In this that ubiquitous, embedded, resource-constrained systems are framework, the multimedia applications, and more specifically able to process in real-time to the requirements imposed those in charge of video encoding, broadcasting, storage and by the increasingly complex and computationally demanding decoding, play a key role. Video content represents today consumer electronics video applications. around 82% of the global Internet traffic according to a Versatile Video Coding (VVC) [8] has important improve- study recently conducted by Cisco [1] and video streaming ments compared to its predecessors, although it is also based represents 58% of the Internet traffic [2]. All these new trends on the conventional hybrid prediction/transform video coding will increase the part of video traffic, storage requirement and design scheme. VVC has achieved 50% [9], [10] bitrate especially its energy footprint. For instance, video stream- reduction compared to HEVC by implementing a set of new ing contributes today to 1% of the global greenhouse gas tools and features distributed over the main modules of the emissions, which represent the emissions of a country like traditional hybrid prediction/transform coding scheme. On the Spain [3]. It is expected that in 2025, CO2 emissions induced other hand, the complexity of both encoder and decoder has by video streaming will reach the global CO2 emissions of been increased [11] as is explained in further sections of this cars [3]. paper. arXiv:2106.14245v1 [eess.IV] 27 Jun 2021 The impressive consumption of multimedia contents in The VVC standard has been published and, at present, different consumer electronic products (mobile devices, smart several research institutions and companies are working on ◦ TVs, video consoles, immersive and 360 video or augmented efficient implementations that will be included in new con- and virtual reality devices) requires more efficient video sumer electronics devices very soon. This paper reports in coding algorithms to reduce the bandwidth and the storage further sections some efficient implementations and trials done capabilities while increasing the video quality. Nowadays, recently in real scenarios. the mass market products demand videos with higher reso- The remainder of this paper is organized as follows. Sec- lutions (greater than 4K) with higher quality (HDR or 10- tionII describes some use cases and the integration of the Wassim Hamidouche and Daniel Menard are with Univ. Rennes, standard with other standards included in the video ecosystem. INSA Rennes, CNRS, IETR - UMR 6164, Rennes, France (e-mail: Sections III andIV outline the basic tools of the VVC [email protected] and [email protected]). Thibaud Biatek, Mohsen Abdoli and Mickael Raulet are with ATEME, standard and the complexity of the algorithm, respectively. Rennes, France. In SectionV, some state-of-the-art implementations for using Edouard Franc¸ois and Milosˇ Radosavljevic´ are with InterDigital, Cesson- consumer electronic devices are reported and, first commercial Sevign´ e,´ France. F. Pescador is with CITSEM at Universidad Politecnica´ de Madrid, Madrid, trials are then presented in SectionVI. Finally, Section VII Spain (e-mail: [email protected]). concludes the paper. PAPER UNDER REVIEW, 2021 2 Frame partitioning Transform & quantization Transform Quantization CU Multi-type Tree Decoder Inv. quantization Preprocessing LMCS (Forward Prediction Inv. transform luma sampling) Inter blocks LMCS (Chroma Entropy coding residue scaling) CABAC Intra prediction Input video LMCS (Inverse Bitstream frame luma sampling) Combined inter- intra prediction Deblocking lter Fig. 1: Potentials of VVC from two different points of view: LMCS (Forward improving existing services and enabling emerging ones. luma sampling) SAO Inter prediction ALF, CC-ALF Motion II. USE-CASES AND STANDARD INTEGRATION estimation The need for more efficient codecs has arisen from different Decoded picture buer sectors of the video delivery ecosystem, as the choice of codec plays a critical role in their success during the coming years. Fig. 2: VVC encoder block diagram. This includes different applications on different transport mediums and VVC is consistently being considered as one of the main options. From the use-case point of view, VVC investigating the adoption of new codecs for 5G applications. specification potentially covers a significantly wider range of Currently, three videos codecs are being characterized, namely applications, compared to previous video codecs. This aspect VVC, AV1 and EVC. In TR26.955 [12], these codecs are is likely to have a positive impact on the deployment cost and investigated for several scenarios, such as HD-Streaming, 4K- interoperability issue of solutions based on VVC. Thanks to TV, Screen-Content, Messaging, Social-Sharing and Online- its versatility and high capacity of addressing the upcoming Gaming. compression challenges, VVC can be used both for improving To limit the risk of reproducing the same licensing uncer- existing video communication applications and enabling new tainty as HEVC, VVC has taken a different approach. First, the ones relying on emerging technologies illustrated in Figure1. media coding industry forum (MC-IF) has been created to deal To properly address market needs and be deployed at scale, with all non-technical issues related to VVC such as licensing VVC shall be referenced and adopted by application-oriented and commercial development. Second, the specification of standards developing organization (SDO) specifications. Or- supplemental enhancement information (SEI) messages has ganizations such as digital video broadcast (DVB), 3rd gen- been shifted to a dedicated specification called VSEI (Versatile eration partnership project (3GPP) or advanced television SEI), published as ITU-T H.274 or ISO/IEC-23002-7. Finally, systems committee standards (ATSC) are defining receivers’ VVC has defined in its high level syntax (HLS) a structure capabilities for broadcast and broadband applications and are named adaptation parameter set (APS) enabling to switch tools thus critical to foster VVC adoption in the ecosystem. Apart off in a normative way in case licensing of specific IP would from its intrinsic performance (complexity and compression), be an issue. the successful adoption of a new video codec also relies on Finally, the integration of the VVC with all these standards its licensing structure. and initiatives will allow its use in different consumer elec- DVB, which is a set of international open standards for dig- tronic devices. ital television, is currently working to include next generation video coding solutions in the DVB specification. In late 2020, III. VVC CODING TOOLS before starting the standarization activities, DVB organized a workshop on new video codecs. During this workshop, five In this section, we give a brief description of the VVC potential codecs were presented and discussed as candidates to coding tools to understand the improvements regarding its address DVB customers’ needs: VVC, essential video coding predecessors. Figure2 illustrates the block diagram of a VVC (EVC), alliance for open media (AOM) video (AV1), low encoder. This latter relies on a conventional hybrid predic- complexity enhancement
Load more

Recommended publications
  • Versatile Video Coding – the Next-Generation Video Standard of the Joint Video Experts Team
    31.07.2018 Versatile Video Coding – The Next-Generation Video Standard of the Joint Video Experts Team Mile High Video Workshop, Denver July 31, 2018 Gary J. Sullivan, JVET co-chair Acknowledgement: Presentation prepared with Jens-Rainer Ohm and Mathias Wien, Institute of Communication Engineering, RWTH Aachen University 1. Introduction Versatile Video Coding – The Next-Generation Video Standard of the Joint Video Experts Team 1 31.07.2018 Video coding standardization organisations • ISO/IEC MPEG = “Moving Picture Experts Group” (ISO/IEC JTC 1/SC 29/WG 11 = International Standardization Organization and International Electrotechnical Commission, Joint Technical Committee 1, Subcommittee 29, Working Group 11) • ITU-T VCEG = “Video Coding Experts Group” (ITU-T SG16/Q6 = International Telecommunications Union – Telecommunications Standardization Sector (ITU-T, a United Nations Organization, formerly CCITT), Study Group 16, Working Party 3, Question 6) • JVT = “Joint Video Team” collaborative team of MPEG & VCEG, responsible for developing AVC (discontinued in 2009) • JCT-VC = “Joint Collaborative Team on Video Coding” team of MPEG & VCEG , responsible for developing HEVC (established January 2010) • JVET = “Joint Video Experts Team” responsible for developing VVC (established Oct. 2015) – previously called “Joint Video Exploration Team” 3 Versatile Video Coding – The Next-Generation Video Standard of the Joint Video Experts Team Gary Sullivan | Jens-Rainer Ohm | Mathias Wien | July 31, 2018 History of international video coding standardization
    [Show full text]
  • Overview of Technologies for Immersive Visual Experiences: Capture, Processing, Compression, Standardization and Display
    Overview of technologies for immersive visual experiences: capture, processing, compression, standardization and display Marek Domański Poznań University of Technology Chair of Multimedia Telecommunications and Microelectronics Poznań, Poland 1 Immersive visual experiences • Arbitrary direction of viewing System : 3DoF • Arbitrary location of viewer - virtual navigation - free-viewpoint television • Both System : 6DoF Virtual viewer 2 Immersive video content Computer-generated Natural content Multiple camera around a scene also depth cameras, light-field cameras Camera(s) located in a center of a scene 360-degree cameras Mixed e.g.: wearable cameras 3 Video capture Common technical problems • Synchronization of cameras Camera hardware needs to enable synchronization Shutter release error < Exposition interval • Frame rate High frame rate needed – Head Mounted Devices 4 Common task for immersive video capture: Calibration Camera parameter estimation: • Intrinsic – the parameters of individual cameras – remain unchanged by camera motion • Extrinsic – related to camera locations in real word – do change by camera motion (even slight !) Out of scope of standardization • Improved methods developed 5 Depth estimation • By video analysis – from at least 2 video sequences – Computationally heavy – Huge progress recently • By depth cameras – diverse products – Infrared illuminate of the scene – Limited resolution mostly Out of scope of standardization 6 MPEG Test video sequences • Large collection of video sequences with depth information
    [Show full text]
  • Versatile Video Coding (VVC)
    Versatile Video Coding (VVC) on the final stretch Benjamin Bross Fraunhofer Heinrich Hertz Institute, Berlin ITU Workshop on “The future of media” © Geneva, Switzerland, 8 October 2019 Versatile Video Coding (VVC) Joint ITU-T (VCEG) and ISO/IEC (MPEG) project Coding Efficiency Versatility 50% over H.265/HEVC Screen content HD / UHD / 8K resolutions Adaptive resolution change 10bit / HDR Independent sub-pictures 2 VVC – Coding Efficiency History of Video Coding Standards H.261 JPEG H.265 / H.264 / H.262 / (1991) (1990) PSNR MPEG-HEVC MPEG-4 AVC MPEG-2 (dB) (2013) (2003) (1995) 40 38 Bit-rate Reduction: 50% 35 36 34 32 30 28 0 100 200 300 bit rate (kbit/s) 3 VVC – Coding Efficiency History of Video Coding Standards H.261 JPEG H.265 / H.264 / H.262 / (1991) (1990) PSNR MPEG-HEVC MPEG-4 AVC MPEG-2 (dB) (2013) (2003) (1995) 40 38 36 Do we need more efficient video coding? 34 32 30 28 0 100 200 300 bit rate (kbit/s) 4 VVC – Coding Efficiency Jevons Paradox "The efficiency with which a resource is used tends to increase (rather than decrease) the rate of consumption of that resource." 5 VVC – Coding Efficiency Target for the final VVC standard H.262 / H.261 JPEG H.??? / H.265 / H.264 / MPEG-2 (1991) (1990) PSNR MPEG-VVC MPEG-HEVC MPEG-4 AVC (dB) (2013) (2003) (1995) 40 38 36 35 Bit-rate Reduction Target: 50% 34 32 30 28 0 100 200 300 bit rate (kbit/s) 6 VVC – Timeline 2015 Oct. – Exploration Phase • Joint Video Exploration Team (JVET) of ITU-T VCEG and ISO/IEC MPEG established October ‘15 in Geneva • Joint Video Exploration Model (JEM) as software playground to explore new coding tools • 34% bitrate savings for JEM relative to HEVC provided evidence to start a new joint standardization activity with a… 2017 Oct.
    [Show full text]
  • On Audio-Visual File Formats
    On Audio-Visual File Formats Summary • digital audio and digital video • container, codec, raw data • different formats for different purposes Reto Kromer • AV Preservation by reto.ch • audio-visual data transformations Film Preservation and Restoration Hyderabad, India 8–15 December 2019 1 2 Digital Audio • sampling Digital Audio • quantisation 3 4 Sampling • 44.1 kHz • 48 kHz • 96 kHz • 192 kHz digitisation = sampling + quantisation 5 6 Quantisation • 16 bit (216 = 65 536) • 24 bit (224 = 16 777 216) • 32 bit (232 = 4 294 967 296) Digital Video 7 8 Digital Video Resolution • resolution • SD 480i / SD 576i • bit depth • HD 720p / HD 1080i • linear, power, logarithmic • 2K / HD 1080p • colour model • 4K / UHD-1 • chroma subsampling • 8K / UHD-2 • illuminant 9 10 Bit Depth Linear, Power, Logarithmic • 8 bit (28 = 256) «medium grey» • 10 bit (210 = 1 024) • linear: 18% • 12 bit (212 = 4 096) • power: 50% • 16 bit (216 = 65 536) • logarithmic: 50% • 24 bit (224 = 16 777 216) 11 12 Colour Model • XYZ, L*a*b* • RGB / R′G′B′ / CMY / C′M′Y′ • Y′IQ / Y′UV / Y′DBDR • Y′CBCR / Y′COCG • Y′PBPR 13 14 15 16 17 18 RGB24 00000000 11111111 00000000 00000000 00000000 00000000 11111111 00000000 00000000 00000000 00000000 11111111 00000000 11111111 11111111 11111111 11111111 00000000 11111111 11111111 11111111 11111111 00000000 11111111 19 20 Compression Uncompressed • uncompressed + data simpler to process • lossless compression + software runs faster • lossy compression – bigger files • chroma subsampling – slower writing, transmission and reading • born
    [Show full text]
  • Videocodecs in a Nutshell
    Videocodecs in a Nutshell Wie funktioniert’s und was gibt’s neues? darkbit 14.08.2021 14.08.21 1 Gliederung ● Motivation ● Grundlagen moderner Videocodecs am Beispiel von HEVC ● Neue Videocodecs – AOMedia Video 1 (AV1) – Versatile Video Coding (VVC) – Essential Video Coding (EVC) – Low Complexity Enhancement Video Codec (LC EVC) ● Vergleich der Codecs ● Ausblick 14.08.21 Videocodecs in a Nutshell - darkbit 2 Motivation ● Rohvideo ist groß ● Bsp: Spielfilm auf DVD – Pixel hat je 8 Bit für die RGB-Komponenten => 24 bits pro Pixel – Übliche Auflösung 720x576 Pixel => 414‘720 Pixel => ~9,953Mbit pro Bild – 25 Bilder pro Sekunde => 248,825 Mbit/s – 90 Minuten Spielfilmlänge => 1,344 Tbit (168 Gbyte) pro Film – Wir haben aber nur max. 8,5 Gbyte pro DVD (Dual-Layer) – Wir müssen unser Video um mind. 95% komprimieren! ● Ziel eines Videocodecs: Möglichst hohe Kompression bei möglichst geringer visuellen Qualitätseinbußungen. 14.08.21 Videocodecs in a Nutshell - darkbit 3 Motivation CC-BY 3.0 - Blender Foundation CC-BY-SA 3.0 - Rdikeman Anforderungen On-Demand Video Live Video Kompression möglichst hoch auf Kanalbandbreite Encodingspeed irrelevant in Echtzeit Decodingspeed in Echtzeit in Echtzeit Bitrate adaptiv konstant 14.08.21 Videocodecs in a Nutshell - darkbit 4 State of the Art Videocodecs Veröffentlichung Codec Bitraten-reduktion Anwendungen Mai 1996 H.262, MPEG-2 Part 2 DVD, SDTV, Blu-Ray März 2003 AVC (H.264, MPEG-4 -50% gegenüber HDTV, Webvideo, Part 10) H.262 WebRTC, Blu-Ray September 2008 (seit VP8 -5% gegenüber AVC Webvideo, 2010 lizenzfrei) WebRTC Mai 2013 VP9 -20% gegenüber AVC Webvideo, WebRTC Dezember 2013 HEVC (H.265, MPEG-H -20% gegenüber AVC UHD Blu-Ray Part 2) 14.08.21 Videocodecs in a Nutshell - darkbit 5 Patentproblematiken Quelle: Jonatan Samuelsson und Per Hermansson.
    [Show full text]
  • VOL. E100-C NO. 6 JUNE 2017 the Usage of This PDF File Must Comply
    VOL. E100-C NO. 6 JUNE 2017 The usage of this PDF file must comply with the IEICE Provisions on Copyright. The author(s) can distribute this PDF file for research and educational (nonprofit) purposes only. Distribution by anyone other than the author(s) is prohibited. IEICE TRANS. ELECTRON., VOL.E100–C, NO.6 JUNE 2017 643 PAPER A High-Throughput and Compact Hardware Implementation for the Reconstruction Loop in HEVC Intra Encoding Yibo FAN†a), Member, Leilei HUANG†, Zheng XIE†, and Xiaoyang ZENG†, Nonmembers SUMMARY In the newly finalized video coding standard, namely high 4×4, 8×8, 16×16, 32×32 and 64×64 with 35 possible pre- efficiency video coding (HEVC), new notations like coding unit (CU), pre- diction modes in intra prediction. Although several fast diction unit (PU) and transformation unit (TU) are introduced to improve mode decision designs have been proposed, still a consid- the coding performance. As a result, the reconstruction loop in intra en- coding is heavily burdened to choose the best partitions or modes for them. erable amount of candidate PU modes, PU partitions or TU In order to solve the bottleneck problems in cycle and hardware cost, this partitions are needed to be traversed by the reconstruction paper proposed a high-throughput and compact implementation for such a loop. reconstruction loop. By “high-throughput”, it refers to that it has a fixed It can be inferred that the reconstruction loop in intra throughput of 32 pixel/cycle independent of the TU/PU size (except for 4×4 TUs). By “compact”, it refers to that it fully explores the reusability prediction has become a bottleneck in cycle and hardware between discrete cosine transform (DCT) and inverse discrete cosine trans- cost.
    [Show full text]
  • Amphion Video Codecs in an AI World
    Video Codecs In An AI World Dr Doug Ridge Amphion Semiconductor The Proliferance of Video in Networks • Video produces huge volumes of data • According to Cisco “By 2021 video will make up 82% of network traffic” • Equals 3.3 zetabytes of data annually • 3.3 x 1021 bytes • 3.3 billion terabytes AI Engines Overview • Example AI network types include Artificial Neural Networks, Spiking Neural Networks and Self-Organizing Feature Maps • Learning and processing are automated • Processing • AI engines designed for processing huge amounts of data quickly • High degree of parallelism • Much greater performance and significantly lower power than CPU/GPU solutions • Learning and Inference • AI ‘learns’ from masses of data presented • Data presented as Input-Desired Output or as unmarked input for self-organization • AI network can start processing once initial training takes place Typical Applications of AI • Reduce data to be sorted manually • Example application in analysis of mammograms • 99% reduction in images send for analysis by specialist • Reduction in workload resulted in huge reduction in wrong diagnoses • Aid in decision making • Example application in traffic monitoring • Identify areas of interest in imagery to focus attention • No definitive decision made by AI engine • Perform decision making independently • Example application in security video surveillance • Alerts and alarms triggered by AI analysis of behaviours in imagery • Reduction in false alarms and more attention paid to alerts by security staff Typical Video Surveillance
    [Show full text]
  • Encoding H.264 Video for Streaming and Progressive Download
    W4: KEY ENCODING SKILLS, TECHNOLOGIES TECHNIQUES STREAMING MEDIA EAST - 2019 Jan Ozer www.streaminglearningcenter.com [email protected]/ 276-235-8542 @janozer Agenda • Introduction • Lesson 5: How to build encoding • Lesson 1: Delivering to Computers, ladder with objective quality metrics Mobile, OTT, and Smart TVs • Lesson 6: Current status of CMAF • Lesson 2: Codec review • Lesson 7: Delivering with dynamic • Lesson 3: Delivering HEVC over and static packaging HLS • Lesson 4: Per-title encoding Lesson 1: Delivering to Computers, Mobile, OTT, and Smart TVs • Computers • Mobile • OTT • Smart TVs Choosing an ABR Format for Computers • Can be DASH or HLS • Factors • Off-the-shelf player vendor (JW Player, Bitmovin, THEOPlayer, etc.) • Encoding/transcoding vendor Choosing an ABR Format for iOS • Native support (playback in the browser) • HTTP Live Streaming • Playback via an app • Any, including DASH, Smooth, HDS or RTMP Dynamic Streaming iOS Media Support Native App Codecs H.264 (High, Level 4.2), HEVC Any (Main10, Level 5 high) ABR formats HLS Any DRM FairPlay Any Captions CEA-608/708, WebVTT, IMSC1 Any HDR HDR10, DolbyVision ? http://bit.ly/hls_spec_2017 iOS Encoding Ladders H.264 HEVC http://bit.ly/hls_spec_2017 HEVC Hardware Support - iOS 3 % bit.ly/mobile_HEVC http://bit.ly/glob_med_2019 Android: Codec and ABR Format Support Codecs ABR VP8 (2.3+) • Multiple codecs and ABR H.264 (3+) HLS (3+) technologies • Serious cautions about HLS • DASH now close to 97% • HEVC VP9 (4.4+) DASH 4.4+ Via MSE • Main Profile Level 3 – mobile HEVC (5+)
    [Show full text]
  • 2021 TIW DRAFT AGENDA Revi
    24th ITEA Test and Training Instrumentation Workshop Innovating for Tomorrow’s Challenges 5/5/21 REV I 11-May First Day – Tutorials 8:00 a.m. – 12:00 p.m. Morning Tutorials Basics of Aircraft Instrumentation Systems Bruce Johnson, NAWCAD This course will cover a wide variety of topics related to Aircraft Instrumentation. Data, Telemetry, Instrumentation System Block Diagram, Standards, Data Requirements, Transducers / Specifications, Video, 1553 Bus, Using Requirements to Configure an Analog Data Channel, Creating a PCM Map to Obtain a Sample Rate, Telemetry Bandwidth, Record Time, GPS, Audio, Telemetry Attributes Transfer Standard (TMATS), and Measurement Uncertainty - Interpreting the Results. This is great introduction for new hires or a refresher for current employees. IRIG 106-17 Chapter 7 Packet Telemetry Downlink Basis and Implementation Fundamentals Johnny Pappas, Safran Data Systems, Inc. This course will focus on presenting information to establish a basic understanding of the 2017 release of the IRIG 106, Chapter 7, Packet Telemetry Downlink Standard. It will also focus on the implementation of airborne and ground system hardware and methods to handle IRIG 106, Chapter 7, Packet Telemetry data. The presentation will address the implementation of special features necessary to support legacy RF Transmission, data recording, RF Receiving, Ground Reproduction, and Chapter 10 data processing methods. Predictive Analytics for Performance Assessment Mark J. Kiemele, Air Academy Associates Design of Experiments (DOE) is a method that can and should be used not only in the design and development of systems, but also in the modeling and validation of system performance. Building useful prediction models and then validating them can ease the burden of making procurement decisions.
    [Show full text]
  • Dynamic Resource Management of Network-On-Chip Platforms for Multi-Stream Video Processing
    DYNAMIC RESOURCE MANAGEMENT OF NETWORK-ON-CHIP PLATFORMS FOR MULTI-STREAM VIDEO PROCESSING Hashan Roshantha Mendis Doctor of Engineering University of York Computer Science March 2017 2 Abstract This thesis considers resource management in the context of parallel multiple video stream de- coding, on multicore/many-core platforms. Such platforms have tens or hundreds of on-chip processing elements which are connected via a Network-on-Chip (NoC). Inefficient task allo- cation configurations can negatively affect the communication cost and resource contention in the platform, leading to predictability and performance issues. Efficient resource management for large-scale complex workloads is considered a challenging research problem; especially when applications such as video streaming and decoding have dynamic and unpredictable workload characteristics. For these type of applications, runtime heuristic-based task mapping techniques are required. As the application and platform size increase, decentralised resource management techniques are more desirable to overcome the reliability and performance bot- tlenecks in centralised management. In this work, several heuristic-based runtime resource management techniques, targeting real-time video decoding workloads are proposed. Firstly, two admission control approaches are proposed; one fully deterministic and highly predictable; the other is heuristic-based, which balances predictability and performance. Secondly, a pair of runtime task mapping schemes are presented, which make use of limited known application properties, communication cost and blocking-aware heuristics. Combined with the proposed deterministic admission con- troller, these techniques can provide strict timing guarantees for hard real-time streams whilst improving resource usage. The third contribution in this thesis is a distributed, bio-inspired, low-overhead, task re-allocation technique, which is used to further improve the timeliness and workload distribution of admitted soft real-time streams.
    [Show full text]
  • Parameter Optimization in H.265 Rate-Distortion by Single Frame
    https://doi.org/10.2352/ISSN.2470-1173.2019.11.IPAS-262 © 2019, Society for Imaging Science and Technology Parameter optimization in H.265 Rate-Distortion by single- frame semantic scene analysis Ahmed M. Hamza; University of Portsmouth Abdelrahman Abdelazim; Blackpool and the Fylde College Djamel Ait-Boudaoud; University of Portsmouth Abstract with Q being the quantization value for the source. The relation is The H.265/HEVC (High Efficiency Video Coding) codec and derived based on several assumptions about the source probability its 3D extensions have crucial rate-distortion mechanisms that distribution within the quantization intervals, and the nature of help determine coding efficiency. We have introduced in this the rate-distortion relations themselves (constantly differentiable work a new system of Lagrangian parameterization in RDO cost throughout, etc.). The value initially used for c in the literature functions, based on semantic cues in an image, starting with the was 0.85. This was modified and made adaptive in subsequent current HEVC formulation of the Lagrangian hyper-parameter standards including HEVC/H.265. heuristics. Two semantic scenery flag algorithms are presented Our work here investigates further adaptations to the rate- and tested within the Lagrangian formulation as weighted factors. distortion Lagrangian by semantic algorithms, made possible by The investigation of whether the semantic gap between the coder recent frameworks in computer vision. and the image content is holding back the block-coding mecha- nisms as a whole from achieving greater efficiency has yielded a Adaptive Lambda Hyper-parameters in HEVC positive answer. Early versions of the rate-distortion parameters in H.263 were replaced with more sophisticated models in subsequent stan- Introduction dards.
    [Show full text]
  • IP-Soc Shanghai 2017 ALLEGRO Presentation FINAL
    Building an Area-optimized Multi-format Video Encoder IP Tomi Jalonen VP Sales www.allegrodvt.com Allegro DVT Founded in 2003 Privately owned, based in Grenoble (France) Two product lines: 1) Industry de-facto standard video compliance streams Decoder syntax, performance and error resilience streams for H.264|MVC, H.265/SHVC, VP9, AVS2 and AV1 System compliance streams 2) Leading semiconductor video IP Multi-format encoder IP for H.264, H.265, VP9, JPEG Multi-format decoder IP for H.264, H.265, VP9, JPEG WiGig IEEE 802.11ad WDE CODEC IP 2 Evolution of Video Coding Standards International standards defined by standardization bodies such as ITU-T and ISO/IEC H.261 (1990) MPEG-1 (1993) H.262 / MPEG-2 (1995) H.263 (1996) MPEG-4 Part 2 (1999) H.264 / AVC / MPEG-4 Part 10 (2003) H.265 / HEVC (2013) Future Video Coding (“FVC”) MPEG and ISO "Preliminary Joint Call for Evidence on Video Compression with Capability beyond HEVC.” (202?) Incremental improvements of transform-based & motion- compensated hybrid video coding schemes to meet the ever increasing resolution and frame rate requirements 3 Regional Video Standards SMPTE standards in the US VC-1 (2006) VC-2 (2008) China Information Industry Department standards AVS (2005) AVS+ (2012) AVS2.0 (2016) 4 Proprietary Video Formats Sorenson Spark On2 VP6, VP7 RealVideo DivX Popular in the past partly due to technical merits but mainly due to more suitable licensing schemes to a given application than standard video video formats with their patent royalties. 5 Royalty-free Video Formats Xiph.org Foundation
    [Show full text]