DOCSLIB.ORG

Constant Bit-Rate Control Efficiency with Fast Motion Estimation in H.264/Avc Video Coding Standard

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Constant Bit-Rate Control Efficiency with Fast Motion Estimation in H.264/Avc Video Coding Standard CONSTANT BIT-RATE CONTROL EFFICIENCY WITH FAST MOTION ESTIMATION IN H.264/AVC VIDEO CODING STANDARD Daniele Alfonso, Daniele Bagni, Luca Celetto and Simone Milani* Advanced System Technology labs, STMicroelectronics, Agrate Brianza, Italy (Europe) * Department of Information Engineering, University of Padova, Italy (Europe) emails: [email protected], [email protected] ABSTRACT greatly reduces the computation with respect to FSBM with- The emerging H.264/AVC video coding standard provides out visible quality losses at SD-TV resolution. ρ significant enhancements in compression efficiency with The JVT-D030/E069 and -domain algorithms are pre- respect to its ancestors in the MPEG and H.26x families. In sented in sections 2 and 3 respectively, whereas section 4 this paper, we analyse the performance of two different Con- introduces the motion estimation method. The numerical stant Bit-Rate control methods, suitable for H.264/AVC en- results are showed in section 5, followed by our conclusions coding at SD-TV resolution, where the motion estimation is in section 6. performed by a fast proprietary predictive algorithm. 2. JVT-D030/E069 RATE CONTROL 1. INTRODUCTION The JVT-D030/E069 [4, 5] is a CBR control method, devel- The H.264/AVC video coding standard [1] greatly outper- oped from the widely known TM5 [6], originally conceived forms its ancestors in the MPEG and H.26x families, provid- for MPEG-2 video coding. The main difference between the ing about 50% more compression at equal quality [2, 3] in two algorithms consists in the definition of luminance mac- all kinds of applications, from low-bitrate streaming to high- roblock activity, which is the Sum of Absolute Differences quality storage. after prediction (either Intra or Inter) for JVT-D030/E069, Until now, the performance of H.264/AVC was analysed and variance for TM5. mostly in an ideal environment, with pure variable bitrate (at Given a target bitrate, the JVT algorithm determines fixed quantization values) and using the common Full-Search the QP quantizer for each macroblock operating at three Block-Matching (FSBM) algorithm for motion estimation. In levels: Group-Of-Pictures (GOP), frame/field picture and practical applications, however, we need to impose some macroblock. In comparison with the original version [4, 5], constraints on the encoding process, i.e. on the number of we changed the experimental constants, in order to opti- bits generated per time unit in case of limited bandwidth, and mally adapt the algorithm to SD-TV video formats. on the overall computational complexity for real-time encod- ing. The former goal can be obtained by a rate control algo- 2.1 Rate control at GOP level rithm, which adapts the quantization of the residual coding in The target number of bits for each GOP is order to match a target bitrate, whereas the computational BitRate R = N ⋅ + R (1) complexity can be reduced by adopting a fast algorithm to PictureRate prev perform motion estimation, which is one of the most resource where N is the number of frames in the GOP, i.e. the dis- consuming tasks in the whole encoding process. tance between two Intra coded pictures, and Rprev is the num- Indeed, the relationship between rate control and motion ber of exceeding bits after the encoding of the previous estimation is an important topic of investigation, because the GOP, which is initialised to zero for the first GOP of the two systems interact with each other, determining the overall sequence. performance of the encoder. In fact, if the motion estimation is not efficient, the prediction error will be greater, forcing 2.2 Rate control at frame/field level the rate control to raise the quantization step, in the attempt At the beginning of each frame or field (respectively in case to match the target bitrate. This will negatively affect both of progressive or interlaced source video), the target number the achieved quality and the motion estimation of successive of bits for each I, P and B picture is computed as in the fol- frames, as the motion search is performed referencing the lowing: reconstructed frames, which are less correlated to the current N X N X BitRate one due to the coarser quantization. T = maxR /1+ p p + b b , i × In this paper, we consider two Constant Bit-Rate (CBR) K p X i Kb X i 8 PictureRate ρ control algorithms, named JVT-D030/E069 and -domain, N K X BitRate T = maxR / N + b p b , (2) while using a proprietary motion estimation technique that p p × Kb X p 8 PictureRate 1971 • n N pKb X p BitRate d0 is the initial virtual buffer occupancy, respectively set T = maxR / N + , i p i b i b b × as d =20·r/31, d =K ·d and d =K ·d for I, P and B K p X b 8 PictureRate 0 0 p 0 0 b 0 pictures at the beginning of the encoding process; where R is defined in (1) and Np and Nb are the number of • remaining P and B frames or fields in the GOP, respectively. Bm-1 is the number of bits generated by encoding the first After the encoding of each frame, R is updated as m-1 macroblocks in the picture (composed of MB_CNT macroblocks in total). R = R − S where S is the number of bits used to encode the current 3. ρ-DOMAIN RATE CONTROL frame, and Np and Nb are decremented by one if the current frame or field was of P- or B-type respectively. Traditional rate control algorithms, as also the JVT- Xi, Xp and Xb define the content complexity of the differ- D030/E069, operate in the so-called q-domain, where rate ent picture types. They are initialised with and distortion characteristic curves of the encoder are de- Xi = ( 155 · BitRate ) / 115 termined as a function of the quantization step. However, an Xp = ( 100 · BitRate ) / 115 alternative is to consider them as functions of the percentage Xb = 0.9 · Xp of null quantized transform coefficients, indicated by ρ. This and after the encoding of each frame they are updated as is named ρ-domain analysis, and an efficient rate control 1 method exploiting this theory was presented in [7] and prop- X {}= ⋅ S ⋅QP i, p,b 2 avg erly adapted to H.264/AVC in [8]. where QPavg is the average quantizer for the current picture. The number of bits R for each picture can be expressed Finally, Kp=1.1 and Kb=1.4 define the relative complex- as a function of ρ through the relation ity of I pictures with respect to P and B ones. ρ = λ ⋅ ρ + λ R( ) 1 2 (3) With the approximation that no bits are sent if all coeffi- 2.3 Rate control at macroblock level cients are null, equation (3) becomes Before encoding each macroblock, an initial quantizer is R(ρ) = λ ⋅ (1− ρ) (4) chosen according to the following formula Given a target bit budget Tn for the n-th frame, the d n × 31 corresponding ρ can be determined from (4) as = m + n Qm dq T r ρ = 1− n (5) n λ where r is a constant called reaction parameter, defined as The parameter λ can be estimated from the results of the bit _ rate r = 10⋅ encoding of previous frames, whereas the percentage of null frame _ rate coefficients ρ can be written as a function of the quantizer dq is named delta parameter and it depends from the activ- QP in the following way ity of the current macroblock in the following way ρ ∆ = (6) ( ) ∑ px (QP) − floor(AvgAct / act −1), 0 < act / AvgAct <= 1/ 2 QP <∆ m m = < < dq 0, 1/ 2 actm / AvgAct 2 where px(q) is the probability distribution of transform coef- − >= ficients x and [-∆,+∆] is the quantization step interval asso- floor(actm / AvgAct) 1, actm / AvgAct 2 AvgAct is the average activity for the current picture and ciated with the null reconstructed value. For the probability distribution, we empirically have chosen a laplacian- actm the activity of the current m-th macroblock. At the be- ginning of the sequence we set AvgAct =2000, impulsive function defined as i 2 − q AvgActp=1500 and AvgActb=800. = α ⋅δ + β px (q) (q) e (7) We would remark that the Intra/Inter prediction could be being δ(q) the Dirac impulse. performed twice for each macroblock in JVT-D030/E069. In Writing (6) in integral form and considering (7) we obtain a first step, we derive the activity using as temporary QP the − 2 ∆ α 1 β one of the previously encoded macroblock. Hence, we en- ρ(∆) = + ⋅1− e +α +α code the macroblock with the final QP computed by the rate 1 1 control, when it differs from the temporary one. Experimen- from which we have tally we found that every macroblock is Intra/Inter predicted β ∆ = − ()+α − ρ ()+α (8) about 1.25 times, on the average, for target bitrates in the ln 1 n 1 ranges from 2 to 7 Mbit/s at SD-TV resolution. 2 The coefficients α and β can be accurately approximated by The dm parameter indicates the occupation of the virtual buffer for picture type n=i,p,b, and it is specified as relating them to the average activity of the picture in the polynomial form: n = n + − × − d d B − T (m 1) / MB _ CNT α(AvgAct) = α + α ⋅ log( AvgAct) + α ⋅ log log( AvgAct) m 0 m 1 n 0 1 2 where: β = β + β ⋅ + β ⋅ 2 (AvgAct) 0 1 AvgAct 2 (AvgAct) • i p b dm , dm , and dm represent the fullness of virtual buffers and then tabulated.
Load more

Recommended publications
  • Digital Speech Processing— Lecture 17
    Digital Speech Processing— Lecture 17 Speech Coding Methods Based on Speech Models 1 Waveform Coding versus Block Processing • Waveform coding – sample-by-sample matching of waveforms – coding quality measured using SNR • Source modeling (block processing) – block processing of signal => vector of outputs every block – overlapped blocks Block 1 Block 2 Block 3 2 Model-Based Speech Coding • we’ve carried waveform coding based on optimizing and maximizing SNR about as far as possible – achieved bit rate reductions on the order of 4:1 (i.e., from 128 Kbps PCM to 32 Kbps ADPCM) at the same time achieving toll quality SNR for telephone-bandwidth speech • to lower bit rate further without reducing speech quality, we need to exploit features of the speech production model, including: – source modeling – spectrum modeling – use of codebook methods for coding efficiency • we also need a new way of comparing performance of different waveform and model-based coding methods – an objective measure, like SNR, isn’t an appropriate measure for model- based coders since they operate on blocks of speech and don’t follow the waveform on a sample-by-sample basis – new subjective measures need to be used that measure user-perceived quality, intelligibility, and robustness to multiple factors 3 Topics Covered in this Lecture • Enhancements for ADPCM Coders – pitch prediction – noise shaping • Analysis-by-Synthesis Speech Coders – multipulse linear prediction coder (MPLPC) – code-excited linear prediction (CELP) • Open-Loop Speech Coders – two-state excitation
    [Show full text]
  • Audio Coding for Digital Broadcasting
    Recommendation ITU-R BS.1196-7 (01/2019) Audio coding for digital broadcasting BS Series Broadcasting service (sound) ii Rec. ITU-R BS.1196-7 Foreword The role of the Radiocommunication Sector is to ensure the rational, equitable, efficient and economical use of the radio- frequency spectrum by all radiocommunication services, including satellite services, and carry out studies without limit of frequency range on the basis of which Recommendations are adopted. The regulatory and policy functions of the Radiocommunication Sector are performed by World and Regional Radiocommunication Conferences and Radiocommunication Assemblies supported by Study Groups. Policy on Intellectual Property Right (IPR) ITU-R policy on IPR is described in the Common Patent Policy for ITU-T/ITU-R/ISO/IEC referenced in Resolution ITU-R 1. Forms to be used for the submission of patent statements and licensing declarations by patent holders are available from http://www.itu.int/ITU-R/go/patents/en where the Guidelines for Implementation of the Common Patent Policy for ITU-T/ITU-R/ISO/IEC and the ITU-R patent information database can also be found. Series of ITU-R Recommendations (Also available online at http://www.itu.int/publ/R-REC/en) Series Title BO Satellite delivery BR Recording for production, archival and play-out; film for television BS Broadcasting service (sound) BT Broadcasting service (television) F Fixed service M Mobile, radiodetermination, amateur and related satellite services P Radiowave propagation RA Radio astronomy RS Remote sensing systems S Fixed-satellite service SA Space applications and meteorology SF Frequency sharing and coordination between fixed-satellite and fixed service systems SM Spectrum management SNG Satellite news gathering TF Time signals and frequency standards emissions V Vocabulary and related subjects Note: This ITU-R Recommendation was approved in English under the procedure detailed in Resolution ITU-R 1.
    [Show full text]
  • Challenges in Relaying Video Back to Mission Control
    Challenges in Relaying Video Back to Mission Control CONTENTS 1 Introduction ..................................................................................................................................................................................... 2 2 Encoding ........................................................................................................................................................................................... 3 3 Time is of the Essence .............................................................................................................................................................. 3 4 The Reality ....................................................................................................................................................................................... 5 5 The Tradeoff .................................................................................................................................................................................... 5 6 Alleviations and Implementation ......................................................................................................................................... 6 7 Conclusion ....................................................................................................................................................................................... 7 White Paper Revision 2 July 2020 By Christopher Fadeley Using a customizable hardware-accelerated encoder is essential to delivering the high
    [Show full text]
  • Hikvision H.264+ Encoding Technology
    WHITE PAPER Hikvision H.264+ Encoding Technology Encoding Improvement / Higher Transmission / Efficiency Storage Savings 2 Contents 1. Introduction .............................................................................................. 3 2. Background ............................................................................................... 3 3. Key Technologies .................................................................................... 4 3.1 Predictive Encoding ........................................................................ 4 3.2 Noise Suppression.......................................................................... 8 3.3 Long-Term Bitrate Control........................................................... 9 4. Applications ............................................................................................ 11 5. Conclusion............................................................................................... 11 Hikvision H.264+ Encoding Technology 3 1. INTRODUCTION As the global market leader in video surveillance products, Hikvision Digital Technology Co., Ltd., continues to strive for enhancement of its products through application of the latest in technology. H.264+ Advanced Video Coding (AVC) optimizes compression beyond the current H.264 standard. Through the combination of intelligent analysis technology with predictive encoding, noise suppression, and long-term bitrate control, Hikvision is meeting the demand for higher resolution at reduced bandwidths. Our customers will benefit
    [Show full text]
  • TMS320C6000 U-Law and A-Law Companding with Software Or The
    Application Report SPRA634 - April 2000 TMS320C6000t µ-Law and A-Law Companding with Software or the McBSP Mark A. Castellano Digital Signal Processing Solutions Todd Hiers Rebecca Ma ABSTRACT This document describes how to perform data companding with the TMS320C6000 digital signal processors (DSP). Companding refers to the compression and expansion of transfer data before and after transmission, respectively. The multichannel buffered serial port (McBSP) in the TMS320C6000 supports two companding formats: µ-Law and A-Law. Both companding formats are specified in the CCITT G.711 recommendation [1]. This application report discusses how to use the McBSP to perform data companding in both the µ-Law and A-Law formats. This document also discusses how to perform companding of data not coming into the device but instead located in a buffer in processor memory. Sample McBSP setup codes are provided for reference. In addition, this application report discusses µ-Law and A-Law companding in software. The appendix provides software companding codes and a brief discussion on the companding theory. Contents 1 Design Problem . 2 2 Overview . 2 3 Companding With the McBSP. 3 3.1 µ-Law Companding . 3 3.1.1 McBSP Register Configuration for µ-Law Companding. 4 3.2 A-Law Companding. 4 3.2.1 McBSP Register Configuration for A-Law Companding. 4 3.3 Companding Internal Data. 5 3.3.1 Non-DLB Mode. 5 3.3.2 DLB Mode . 6 3.4 Sample C Functions. 6 4 Companding With Software. 6 5 Conclusion . 7 6 References . 7 TMS320C6000 is a trademark of Texas Instruments Incorporated.
    [Show full text]
  • Bit Rate Adaptation Using Linear Quadratic Optimization for Mobile Video Streaming
    applied sciences Article Bit Rate Adaptation Using Linear Quadratic Optimization for Mobile Video Streaming Young-myoung Kang 1 and Yeon-sup Lim 2,* 1 Network Business, Samsung Electronics, Suwon 16677, Korea; [email protected] 2 Department of Convergence Security Engineering, Sungshin Women’s University, Seoul 02844, Korea * Correspondence: [email protected]; Tel.: +82-2-920-7144 Abstract: Video streaming application such as Youtube is one of the most popular mobile applications. To adjust the quality of video for available network bandwidth, a streaming server provides multiple representations of video of which bit rate has different bandwidth requirements. A streaming client utilizes an adaptive bit rate (ABR) scheme to select a proper representation that the network can support. However, in mobile environments, incorrect decisions of an ABR scheme often cause playback stalls that significantly degrade the quality of user experience, which can easily happen due to network dynamics. In this work, we propose a control theory (Linear Quadratic Optimization)- based ABR scheme to enhance the quality of experience in mobile video streaming. Our simulation study shows that our proposed ABR scheme successfully mitigates and shortens playback stalls while preserving the similar quality of streaming video compared to the state-of-the-art ABR schemes. Keywords: dynamic adaptive streaming over HTTP; adaptive bit rate scheme; control theory 1. Introduction Video streaming constitutes a large fraction of current Internet traffic. In particular, video streaming accounts for more than 65% of world-wide mobile downstream traffic [1]. Commercial streaming services such as YouTube and Netflix are implemented based on Citation: Kang, Y.-m.; Lim, Y.-s.
    [Show full text]
  • A Predictive H.263 Bit-Rate Control Scheme Based on Scene
    A PRJ3DICTIVE H.263 BIT-RATE CONTROL SCHEME BASED ON SCENE INFORMATION Pohsiang Hsu and K. J. Ray Liu Department of Electrical and Computer Engineering University of Maryland at College Park College Park, Maryland, USA ABSTRACT is typically constant bit rate (e.g. PSTN). Under this cir- cumstance, the encoder’s output bit-rate must be regulated For many real-time visual communication applications such to meet the transmission bandwidth in order to guaran- as video telephony, the transmission channel in used is typ- tee a constant frame rate and delay. Given the channel ically constant bit rate (e.g. PSTN). Under this circum- rate and a target frame rate, we derive the target bits per stance, the encoder’s output bit-rate must be regulated to frame by distribute the channel rate equally to each hme. meet the transmission bandwidth in order to guarantee a Ffate control is typically based on varying the quantization constant frame rate and delay. In this work, we propose a paraneter to meet the target bit budget for each frame. new predictive rate control scheme based on the activity of Many rate control scheme for the block-based DCT/motion the scene to be coded for H.263 encoder compensation video encoders have been proposed in the past. These rate control algorithms are typically focused on MPEG video coding standards. Rate-Distortion based rate 1. INTRODUCTION control for MPEG using Lagrangian optimization [4] have been proposed in the literature. These methods require ex- In the past few years, emerging demands for digital video tensive,rate-distortion measurements and are unsuitable for communication applications such as video conferencing, video real time applications.
    [Show full text]
  • A Packetization and Variable Bitrate Interframe Compression Scheme for Vector Quantizer-Based Distributed Speech Recognition
    A Packetization and Variable Bitrate Interframe Compression Scheme For Vector Quantizer-Based Distributed Speech Recognition Bengt J. Borgstrom¨ and Abeer Alwan Department of Electrical Engineering, University of California, Los Angeles [email protected], [email protected] Abstract For example, the ETSI standard [2] packetization scheme sim- ply concatenates 2 adjacent 44-bit source coded frames, and We propose a novel packetization and variable bitrate com- transmits the resulting signal along with header information. pression scheme for DSR source coding, based on the Group The proposed algorithm performs further compression using of Pictures concept from video coding. The proposed algorithm interframe correlation. The high time correlation present in simultaneously packetizes and further compresses source coded speech has previously been studied for speech coding [3] and features using the high interframe correlation of speech, and is DSR coding [4]. compatible with a variety of VQ-based DSR source coders. The The proposed algorithm allows for lossless compression of algorithm approximates vector quantizers as Markov Chains, the quantized speech features. However, the algorithm is also and empirically trains the corresponding probability parame- robust to various degrees of lossy compression, either through ters. Feature frames are then compressed as I-frames, P-frames, VQ pruning or frame puncturing. VQ pruning refers to ex- or B-frames, using Huffman tables. The proposed scheme can clusion of low probability VQ codebook labels prior to Huff- perform lossless compression, but is also robust to lossy com- man coding, and thus excludes longer Huffman codewords, pression through VQ pruning or frame puncturing. To illustrate and frame puncturing refers to the non-transmission of certain its effectiveness, we applied the proposed algorithm to the ETSI frames, which drastically reduces the final bitrate.
    [Show full text]
  • AVC to the Max: How to Configure Encoder
    Contents Company overview …. ………………………………………………………………… 3 Introduction…………………………………………………………………………… 4 What is AVC….………………………………………………………………………… 6 Making sense of profiles, levels, and bitrate………………………………………... 7 Group of pictures and its structure..………………………………………………… 11 Macroblocks: partitioning and prediction modes….………………………………. 14 Eliminating spatial redundancy……………………………………………………… 15 Eliminating temporal redundancy……...……………………………………………. 17 Adaptive quantization……...………………………………………………………… 24 Deblocking filtering….….…………………………………………………………….. 26 Entropy encoding…………………………………….……………………………….. 2 8 Conclusion…………………………………………………………………………….. 29 Contact details..………………………………………………………………………. 30 2 www.elecard.com Company overview Elecard company, founded in 1988, is a leading provider of software products for encoding, decoding, processing, monitoring and analysis of video and audio data in 9700 companies various formats. Elecard is a vendor of professional software products and software development kits (SDKs); products for in - depth high - quality analysis and monitoring of the media content; countries 1 50 solutions for IPTV and OTT projects, digital TV broadcasting and video streaming; transcoding servers. Elecard is based in the United States, Russia, and China with 20M users headquarters located in Tomsk, Russia. Elecard products are highly appreciated and widely used by the leaders of IT industry such as Intel, Cisco, Netflix, Huawei, Blackmagic Design, etc. For more information, please visit www.elecard.com. 3 www.elecard.com Introduction Video compression is the key step in video processing. Compression allows broadcasters and premium TV providers to deliver their content to their audience. Many video compression standards currently exist in TV broadcasting. Each standard has different properties, some of which are better suited to traditional live TV while others are more suited to video on demand (VoD). Two basic standards can be identified in the history of video compression: • MPEG-2, a legacy codec used for SD video and early digital broadcasting.
    [Show full text]
  • Adaptive Bitrate Streaming Over Cellular Networks: Rate Adaptation and Data Savings Strategies
    Adaptive Bitrate Streaming Over Cellular Networks: Rate Adaptation and Data Savings Strategies Yanyuan Qin, Ph.D. University of Connecticut, 2021 ABSTRACT Adaptive bitrate streaming (ABR) has become the de facto technique for video streaming over the Internet. Despite a flurry of techniques, achieving high quality ABR streaming over cellular networks remains a tremendous challenge. First, the design of an ABR scheme needs to balance conflicting Quality of Experience (QoE) metrics such as video quality, quality changes, stalls and startup performance, which is even harder under highly dynamic bandwidth in cellular network. Second, streaming providers have been moving towards using Variable Bitrate (VBR) encodings for the video content, which introduces new challenges for ABR streaming, whose nature and implications are little understood. Third, mobile video streaming consumes a lot of data. Although many video and network providers currently offer data saving options, the existing practices are suboptimal in QoE and resource usage. Last, when the audio and video arXiv:2104.01104v2 [cs.NI] 14 May 2021 tracks are stored separately, video and audio rate adaptation needs to be dynamically coordinated to achieve good overall streaming experience, which presents interesting challenges while, somewhat surprisingly, has received little attention by the research community. In this dissertation, we tackle each of the above four challenges. Firstly, we design a framework called PIA (PID-control based ABR streaming) that strategically leverages PID control concepts and novel approaches to account for the various requirements of ABR streaming. The evaluation results demonstrate that PIA outperforms state-of-the-art schemes in providing high average bitrate with signif- icantly lower bitrate changes and stalls, while incurring very small runtime overhead.
    [Show full text]
  • Comments on `Area and Power Efficient DCT Architecture for Image
    Cintra and Bayer EURASIP Journal on Advances in Signal EURASIP Journal on Advances Processing (2017) 2017:50 DOI 10.1186/s13634-017-0486-8 in Signal Processing RESEARCH Open Access Comments on ‘Area and power efficient DCT architecture for image compression’ by Dhandapani and Ramachandran Renato J. Cintra1* and Fábio M. Bayer2 Abstract In [Dhandapani and Ramachandran, “Area and power efficient DCT architecture for image compression”, EURASIP Journal on Advances in Signal Processing 2014, 2014:180] the authors claim to have introduced an approximation for the discrete cosine transform capable of outperforming several well-known approximations in literature in terms of additive complexity. We could not verify the above results and we offer corrections for their work. Keywords: DCT approximations, Low-complexity transforms 1 Introduction 2 Criticisms In a recent paper [1], a low-complexity transformation 2.1 Inverse transformation was introduced, which is claimed to be a good approxima- The authors of [1] claim that inverse transformation T−1 tion to the discrete cosine transform (DCT). We wish to is given by evaluate this claim. ⎡ ⎤ The introduced transformation is given by the following 10001000 ⎢ ⎥ matrix: ⎢ −1100−11 0 0⎥ ⎢ ⎥ ⎢ 00100010⎥ ⎡ ⎤ ⎢ ⎥ 1 ⎢ 00−11 0 0−11⎥ 10000001 · ⎢ ⎥ . ⎢ ⎥ 2 ⎢ 00−11 0 0 1−1 ⎥ ⎢ 11000011⎥ ⎢ ⎥ ⎢ ⎥ ⎢ 001000−10⎥ ⎢ 00100100⎥ ⎣ ⎦ ⎢ ⎥ −11001−10 0 ⎢ 00111100⎥ T = ⎢ ⎥ . 1000−10 0 0 ⎢ 00 1 1−1 −10 0⎥ ⎢ ⎥ ⎢ 00 1 0 0−10 0⎥ ⎣ ⎦ However, simple computation reveal that this is not 110000−1 −1 accurate, being the actual inverse given by: 1000000−1 ⎡ ⎤ 10000001 ⎢ ⎥ We aim at analyzing the above matrix and showing that ⎢ −1100001−1 ⎥ ⎢ ⎥ it does not consist of a meaningful approximation for ⎢ 00100100⎥ ⎢ ⎥ the8-pointDCT.Inthefollowing,weadoptedthesame − 1 ⎢ 00−11 1−10 0⎥ T 1 = · ⎢ ⎥ .
    [Show full text]
  • Neural Rate Control for Video Encoding Using Imitation Learning
    Neural Rate Control for Video Encoding using Imitation Learning Hongzi Mao†? Chenjie Gu† Miaosen Wang† Angie Chen† Nevena Lazic† Nir Levine† Derek Pang* Rene Claus* Marisabel Hechtman* Ching-Han Chiang* Cheng Chen* Jingning Han* †DeepMind *Google ?MIT CSAIL Abstract module manages the distribution of available bandwidth, usually constrained by the network condition. It directly In modern video encoders, rate control is a critical com- determines how many bits to spend to encode each video ponent and has been heavily engineered. It decides how frame, by assigning a quantization parameter (QP). The many bits to spend to encode each frame, in order to opti- goal of rate control is to maximize the encoding efficiency mize the rate-distortion trade-off over all video frames. This (often measured by the Bjontegaard delta rate) as well as is a challenging constrained planning problem because of maintaining the bitrate under a user-specified target bitrate. the complex dependency among decisions for different video Rate control is a constrained planning problem and can frames and the bitrate constraint defined at the end of the be formulated as a Partially Observable Markov Decision episode. Process. For a particular frame, the QP assignment decision We formulate the rate control problem as a Partially depends on (1) the frame’s spatial and temporal complexity, Observable Markov Decision Process (POMDP), and ap- (2) the previously encoded frames this frame refers to, and ply imitation learning to learn a neural rate control pol- (3) the complexity of future frames. In addition, the bitrate icy. We demonstrate that by learning from optimal video constraint imposes an episodic constraint on the planning encoding trajectories obtained through evolution strategies, problem.
    [Show full text]