DOCSLIB.ORG

Motion Estimation and Compensation

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Motion Estimation and Compensation Video Codec Design Iain E. G. Richardson Copyright q 2002 John Wiley & Sons, Ltd ISBNs: 0-471-48553-5 (Hardback); 0-470-84783-2 (Electronic) Motion Estimation and Compensation 6.1 INTRODUCTION In the video coding standards described in Chapters 4 and 5, blocks of image samples or residual data are compressed using a block-based transform (such as the DCT) followed by quantisationand entropy encoding. There is limitedscope for improved compression performance in the later stages of encoding (DCT, quantisation and entropy coding), since the operation of the DCT and the codebook for entropy coding are specified by the relevant video coding standard. However, there is scope for significant performance improvement in the design of the first stage of a video CODEC (motion estimation and compensation). Efficient motion estimation reduces the energy in the motion-compensated residual frame and can dramatically improve compression performance. Motion estimation can bevery computationally intensive and so this compression performance may be at the expense of high computational complexity. This chapter describes the motion estimation and compen- sation process in detail and discusses implementation alternatives and trade-offs. The motion estimation and compensation functions have many implications for CODEC performance. Key performance issues include: 0 Coding performance (how efficient is the algorithm at minimising the residual frame?) 0 Complexity (does the algorithm makeeffective use of computation resources, how easy is it to implement in software or hardware?) 0 Storage and/or delay (does the algorithm introduce extra delay and/or require storage of multiple frames?) 0 ‘Side’ information(how much extrainformation, e.g. motion vectors, needs to be transmitted to the decoder?) 0 Error resilience (how does the decoder perform when errors occur during transmission?) These issues are interrelated and are potentially contradictory (e.g. better coding perfor- mance may lead to increased complexity and delay and poor error resilience) and different solutions are appropriate for different platforms and applications. The design and imple- mentation of motionestimation, compensation and reconstruction can be critical to the performance of a video coding application. 94 COMPENSATIONESTIMATIONMOTION AND 6.2 MOTION ESTIMATION AND COMPENSATION 6.2.1 Requirements for Motion Estimation and Compensation Motion estimation creates a model of the current frame based on available data in one or more previously encoded frames (‘reference frames’). These reference framesmay be ‘past’ frames (i.e. earlier than the current frame in temporal order) or ‘future’ frames (i.e. later in temporal order). The design goals for a motion estimation algorithm areto model the current frame as accurately as possible (since this gives better compression performance) whilst maintainingacceptable computational complexity. InFigure 6.1, the motion estimation module creates a model by modifying one or more reference frames to match the current frame asclosely as possible (according to a matching criterion).The current frame is motion compensated by subtracting the model from the frame to produce a motion-compensated residualframe. This is coded and transmitted, along with theinformation required for the decoder to recreate the model (typically a set of motion vectors). At the same time, the encoded residual is decoded and added to the model to reconstruct a decoded copy of the current frame (which may not be identical to the original frame because of coding losses). This reconstructed frame is stored to be used as a reference frame for further predictions. The residualframe (or displaced frame difference, DFD) is encoded and transmitted, together with any ‘side information’ (such as motion vectors) needed to recreate the model at the decoder. The ‘best’ compression performance is achieved when the size of the coded DFD and coded side information is minimised. The size of the coded DFD is related to the energy remaining in the DFD after motion compensation. Figure 6.2 shows a previous, current and residual frame (DFD) without motion compensation: there is clearly a significant amount of energy present around the boundaries of moving objects (the girl and the bicycle in this case).It should be possible to reducethis energy (and improve compression performance) using motion estimation and compensation. Motion Current frame compensation Encode residual $. Reconstructed Reconstruction 4 frame Decode residual d Figure 6.1 Motion estimation and compensation block diagram MOTION ESTIMATIONMOTION AND COMPENSATION 95 Figure 6.2 (a)Previous frame; (b) current (c) frame; (c) DFD (no motion compensation) 6.2.2 Block Matching In the popular video coding standards (H.261, H.263, MPEG-1, MPEG-2 and MPEG-4), motion estimation and compensation are carried out8 onx 8 or 16 x 16 blocksin the current frame. Motion estimation of complete blocks is known as block matching. For each block of luminance samples (say 16 x 16) in the current frame, the motion estimation algorithm searches a neighbouring area of the reference frame for a ‘matching’ 16 x 16 area. The best match is the one that minimises the energyof the difference between the current 16 x 16 block and the matching 16 x 16 area. The area in which the search is carried out may be centred around the position of the current 16 x 16 block, because (a) there is likely to be a good match in the immediate areaof the current block due to thehigh similarity (correlation) between subsequent frames and (b) it would be computationally intensive to search the whole of the reference frame. Figure 6.3 illustrates the block matching process. The current ‘block’ (in this case, 3x 3 pixels) is shown on the left and thisblock is comparedwith the same position in the reference frame (shown by the thick line in the centre) and the immediate neighbouring positions (+/-l pixelin each direction). The mean squarederror (MSE) betweenthe 96 MOTION ESTIMATION AND COMPENSATION CurrentReferenceblock area Positions (x,y) Figure 6.3 Current 3 x 3 block and 5 x 5 reference area current block and the same position in the reference frame (position (0, 0)) is given by {(l - 4)* + (3 - 2)2 + (2 - 3)2+ (6 - 4)* + (4 - 2)’ + (3 - 2)’ + (5 - 4)’ + (4 - 3)’ + (3 - 3)2}/9 = 2.44 The complete set of MSE values for each search position is listed in Table 6.1 and shown graphically in Figure 6.4. Of the nine candidate positions, (- 1, l) gives the smallest MSE and hence the ‘best’ match. In this example, the best ‘model’ for the current block (i.e. the best prediction) is the 3 x 3 region in position (- l, 1). A video encoder carries out this process for each block in the current frame 1. Calculate the energyof the difference betweenthe current block and a set of neighbouring regions in the reference frame. 2. Select the region that gives the lowest error (the ‘matching region’). 3. Subtract the matching region from the current block to produce a difference block. 4. Encode and transmit the difference block. 5. Encode and transmit a ‘motion vector’ that indicates the position of the matching region, relative to the current block position (in the above example, the motionvector is ( - I, 1). Steps 1 and 2 above correspond to motion estimation and step 3 to motion compensation. The video decoder reconstructs the block as follows: 1. Decode the difference block and motion vector. 2. Add the difference block to the matching region in the reference frame (i.e. the region ‘pointed’ to by the motion vector). Table 6.1 MSE values for block matching example Position(x,y) (-1,-1) (0, -1) (1, -1) (-1,O) (0,O) (1, 0) (-1, 1) (0, 1) (1, 1) MSE 4.672.78 2.89 2.443.22 3.33 0.22 5.332.56 M OTION ESTIMATIONMOTION AND COMPENSATION 97 (b) Figure 6.4 MSE map: (a) surface plot; (b) pseudocolour plot 6.2.3 MinimisingDifference Energy The name‘motion estimation’ is misleading because the process does not necessarily identify ‘true’ motion, instead it attempts to find a matching region in the reference frame MOTION ESTIMATION AND COMPENSATION l, I 2t3 4- 5- 6- e-- 7 8 “0 2 4 6 8 10 12 Figure 6.5 16 x 16 block motion vectors thatminimises the energy of thedifference block. Where there is clearly identifiable linearmotion, such as largemoving objects or global motion (camera panning, etc.), motion vectors produced in this way should roughly correspond to the movement of blocks between the reference and the current frames. However, where the motion is less obvious (e.g. small moving objects that do not correspond to complete blocks, irregular motion, etc.), the ‘motion vector’ may not indicate genuine motion but rather the positionof a good match. Figure6.5 shows the motion vectors produced by motionestimation for each of the 16 x 16 blocks (‘macroblocks’)of the frame in Figure 6.2. Most of the vectors do correspond to motion: the girl and bicycle are movingto the left and so the vectors point to therighr (i.e. to the region the objects have movedfrorn). There is an anomalous vectorin the middle (it is larger than the rest and points diagonally upwards). This vector does not correspond to ‘true’ motion, it simply indicates that the best match can be found in this position. There are many possible variations on the basic block matching process, some of which will be described later in this chapter. Alternative measures of DFD energy may be used (to reduce the computation required to calculate MSE). Varying block sizes, or irregular- shaped regions, can be more efficient at matching ‘true’ motion than fixed 16 x 16 blocks. A better match may be found by searching within two or more reference frames (rather than just one). The order of searching neighbouring regions can have a significant effect on matching efficiency and computational complexity.
Load more

Recommended publications
  • On Computational Complexity of Motion Estimation Algorithms in MPEG-4 Encoder
    On Computational Complexity of Motion Estimation Algorithms in MPEG-4 Encoder Muhammad Shahid This thesis report is presented as a part of degree of Master of Science in Electrical Engineering Blekinge Institute of Technology, 2010 Supervisor: Tech Lic. Andreas Rossholm, ST-Ericsson Examiner: Dr. Benny Lovstrom, Blekinge Institute of Technology Abstract Video Encoding in mobile equipments is a computationally demanding fea- ture that requires a well designed and well developed algorithm. The op- timal solution requires a trade off in the encoding process, e.g. motion estimation with tradeoff between low complexity versus high perceptual quality and efficiency. The present thesis works on reducing the complexity of motion estimation algorithms used for MPEG-4 video encoding taking SLIMPEG motion estimation algorithm as reference. The inherent prop- erties of video like spatial and temporal correlation have been exploited to test new techniques of motion estimation. Four motion estimation algo- rithms have been proposed. The computational complexity and encoding quality have been evaluated. The resulting encoded video quality has been compared against the standard Full Search algorithm. At the same time, reduction in computational complexity of the improved algorithm is com- pared against SLIMPEG which is already about 99 % more efficient than Full Search in terms of computational complexity. The fourth proposed algorithm, Adaptive SAD Control, offers a mechanism of choosing trade off between computational complexity and encoding quality in a dynamic way. Acknowledgements It is a matter of great pleasure to express my deepest gratitude to my ad- visors Dr. Benny L¨ovstr¨om and Andreas Rossholm for all their guidance, support and encouragement throughout my thesis work.
    [Show full text]
  • 1. in the New Document Dialog Box, on the General Tab, for Type, Choose Flash Document, Then Click______
    1. In the New Document dialog box, on the General tab, for type, choose Flash Document, then click____________. 1. Tab 2. Ok 3. Delete 4. Save 2. Specify the export ________ for classes in the movie. 1. Frame 2. Class 3. Loading 4. Main 3. To help manage the files in a large application, flash MX professional 2004 supports the concept of _________. 1. Files 2. Projects 3. Flash 4. Player 4. In the AppName directory, create a subdirectory named_______. 1. Source 2. Flash documents 3. Source code 4. Classes 5. In Flash, most applications are _________ and include __________ user interfaces. 1. Visual, Graphical 2. Visual, Flash 3. Graphical, Flash 4. Visual, AppName 6. Test locally by opening ________ in your web browser. 1. AppName.fla 2. AppName.html 3. AppName.swf 4. AppName 7. The AppName directory will contain everything in our project, including _________ and _____________ . 1. Source code, Final input 2. Input, Output 3. Source code, Final output 4. Source code, everything 8. In the AppName directory, create a subdirectory named_______. 1. Compiled application 2. Deploy 3. Final output 4. Source code 9. Every Flash application must include at least one ______________. 1. Flash document 2. AppName 3. Deploy 4. Source 10. In the AppName/Source directory, create a subdirectory named __________. 1. Source 2. Com 3. Some domain 4. AppName 11. In the AppName/Source/Com directory, create a sub directory named ______ 1. Some domain 2. Com 3. AppName 4. Source 12. A project is group of related _________ that can be managed via the project panel in the flash.
    [Show full text]
  • An Overview of Block Matching Algorithms for Motion Vector Estimation
    Proceedings of the Second International Conference on Research in DOI: 10.15439/2017R85 Intelligent and Computing in Engineering pp. 217–222 ACSIS, Vol. 10 ISSN 2300-5963 An Overview of Block Matching Algorithms for Motion Vector Estimation Sonam T. Khawase1, Shailesh D. Kamble2, Nileshsingh V. Thakur3, Akshay S. Patharkar4 1PG Scholar, Computer Science & Engineering, Yeshwantrao Chavan College of Engineering, India 2Computer Science & Engineering, Yeshwantrao Chavan College of Engineering, India 3Computer Science & Engineering, Prof Ram Meghe College of Engineering & Management, India 4Computer Technology, K.D.K. College of Engineering, India [email protected], [email protected],[email protected], [email protected] Abstract–In video compression technique, motion estimation is one of the key components because of its high computation complexity involves in finding the motion vectors (MV) between the frames. The purpose of motion estimation is to reduce the storage space, bandwidth and transmission cost for transmission of video in many multimedia service applications by reducing the temporal redundancies while maintaining a good quality of the video. There are many motion estimation algorithms, but there is a trade-off between algorithms accuracy and speed. Among all of these, block-based motion estimation algorithms are most robust and versatile. In motion estimation, a variety of fast block based matching algorithms has been proposed to address the issues such as reducing the number of search/checkpoints, computational cost, and complexities etc. Due to its simplicity, the Fig. 1. Classification of Frames block-based technique is most popular. Motion estimation is only known for video coding process but for solving real life redundancies.
    [Show full text]
  • Motion Estimation at the Decoder Sven Klomp and Jorn¨ Ostermann Leibniz Universit¨At Hannover Germany
    5 Motion Estimation at the Decoder Sven Klomp and Jorn¨ Ostermann Leibniz Universit¨at Hannover Germany 1. Introduction All existing video coding standards, such as MPEG-1,2,4 or ITU-T H.26x, perform motion estimation at the encoder in order to exploit temporal dependencies within the video sequence. The estimated motion vectors are transmitted and used by the decoder to assemble a prediction of the current frame. Since only the prediction error and the motion information are transmitted, instead of intra coding the pixel values, compression is achieved. Due to block-based motion estimation, accurate compensation at object borders can only be achieved with small block sizes. Large blocks may contain several objects which move in different directions. Thus, accurate motion estimation and compensation is not possible, as shown by Klomp et al. (2010a) using prediction error variance, and small block sizes are favourable. However, the smaller the block, the more motion vectors have to be transmitted, resulting in a contradiction to bit rate reduction. 4.0 3.5 3.0 2.5 MV Backward MV Forward MV Bidirectional 2.0 RS Backward Rate (bit/pixel) RS Forward 1.5 RS Bidirectional 1.0 0.5 0.0 2 4 8 16 32 Block Size Fig. 1. Data rates of residual (RS) and motion vectors (MV) for different motion compensation techniques (Kimono sequence). 782 Effective Video Coding for MultimediaVideo Applications Coding These characteristics can be observed in Figure 1, where the rates for the residual and the motion vectors are plotted for different block sizes and three prediction techniques.
    [Show full text]
  • Motion Vector Forecast and Mapping (MV-Fmap) Method for Entropy Coding Based Video Coders Julien Le Tanou, Jean-Marc Thiesse, Joël Jung, Marc Antonini
    Motion Vector Forecast and Mapping (MV-FMap) Method for Entropy Coding based Video Coders Julien Le Tanou, Jean-Marc Thiesse, Joël Jung, Marc Antonini To cite this version: Julien Le Tanou, Jean-Marc Thiesse, Joël Jung, Marc Antonini. Motion Vector Forecast and Mapping (MV-FMap) Method for Entropy Coding based Video Coders. MMSP’10 2010 IEEE International Workshop on Multimedia Signal Processing, Oct 2010, Saint Malo, France. pp.206. hal-00531819 HAL Id: hal-00531819 https://hal.archives-ouvertes.fr/hal-00531819 Submitted on 3 Nov 2010 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Motion Vector Forecast and Mapping (MV-FMap) Method for Entropy Coding based Video Coders Julien Le Tanou #1, Jean-Marc Thiesse #2, Joël Jung #3, Marc Antonini ∗4 # Orange Labs 38 rue du G. Leclerc, 92794 Issy les Moulineaux, France 1 [email protected] {2 jeanmarc.thiesse,3 joelb.jung}@orange-ftgroup.com ∗ I3S Lab. University of Nice-Sophia Antipolis/CNRS 2000 route des Lucioles, 06903 Sophia Antipolis, France 4 [email protected] Abstract—Since the finalization of the H.264/AVC standard between motion vectors of neighboring frames and blocks, we and in order to meet the target set by both ITU-T and MPEG propose in this paper a method for motion vector coding based to define a new standard that reaches 50% bit rate reduction on a motion vector residuals forecast followed by an adaptive compared to H.264/AVC, many tools have efficiently improved the texture coding and the motion compensation accuracy.
    [Show full text]
  • 11.2 Motion Estimation and Motion Compensation 421
    11.2 Motion Estimation and Motion Compensation 421 vertical component to the enhancement filter, making the overall filter separable with 3 3 support. × 11.2 MOTION ESTIMATION AND MOTION COMPENSATION Motion compensation (MC) is very useful in video filtering to remove noise and enhance signal. It is useful since it allows the filter or coder to process through the video on a path of near-maximum correlation based on following motion trajectories across the frames making up the image sequence or video. Motion compensation is also employed in all distribution-quality video coding formats, since it is able to achieve the smallest prediction error, which is then easier to code. Motion can be characterized in terms of either a velocity vector v or displacement vector d and is used to warp a reference frame onto a target frame. Motion estimation is used to obtain these displacements, one for each pixel in the target frame. Several methods of motion estimation are commonly used: • Block matching • Hierarchical block matching • Pel-recursive motion estimation • Direct optical flow methods • Mesh-matching methods Optical flow is the apparent displacement vector field d .d1,d2/ we get from setting (i.e., forcing) equality in the so-called constraint equationD x.n1,n2,n/ x.n1 d1,n2 d2,n 1/. (11.2–1) D − − − All five approaches start from this basic equation, which is really just an ide- alization. Departures from the ideal are caused by the covering and uncovering of objects in the viewed scene, lighting variation both in time and across the objects in the scene, movement toward or away from the camera, as well as rotation about an axis (i.e., 3-D motion).
    [Show full text]
  • Diapositivo 1
    VC 14/15 – TP16 Video Compression Mestrado em Ciência de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informáticos Miguel Tavares Coimbra Outline • The need for compression • Types of redundancy • Image compression • Video compression VC 14/15 - TP16 - Video Compression Topic: The need for compression • The need for compression • Types of redundancy • Image compression • Video compression VC 14/15 - TP16 - Video Compression Images are great! VC 14/15 - TP16 - Video Compression But... Images need storage space... A lot of space! Size: 1024 x 768 pixels RGB colour space 8 bits per color = 2,6 MBytes VC 14/15 - TP16 - Video Compression What about video? • VGA: 640x480, 3 bytes per pixel -> 920KB per image. • Each second of video: 23 MB • Each hour of vídeo: 83 GB The death of Digital Video VC 14/15 - TP16 - Video Compression What if... ? • We exploit redundancy to compress image and video information? – Image Compression Standards – Video Compression Standards • “Explosion” of Digital Image & Video – Internet media – DVDs – Digital TV – ... VC 14/15 - TP16 - Video Compression Compression • Data compression – Reduce the quantity of data needed to store the same information. – In computer terms: Use fewer bits. • How is this done? – Exploit data redundancy. • But don’t we lose information? – Only if you want to... VC 14/15 - TP16 - Video Compression Types of Compression • Lossy • Lossless – We do not obtain an – We obtain an exact exact copy of our copy of our compressed data after compressed data after decompression. decompression. – Very high compression – Lower compression rates. rates. – Increased degradation – Freely compress / with sucessive decompress images. compression / It all depends on what we decompression.
    [Show full text]
  • Motion Compensation on DCT Domain
    EURASIP Journal on Applied Signal Processing 2001:3, 147–162 © 2001 Hindawi Publishing Corporation Motion Compensation on DCT Domain Ut-Va Koc Lucent Technologies Bell Labs, 600 Mountain Avenue, Murray Hill, NJ 07974, USA Email: [email protected] K. J. Ray Liu Department of Electrical and Computer Engineering and Institute for Systems Research, University of Maryland, College Park, MD 20742, USA Email: [email protected] Received 21 May 2001 and in revised form 21 September 2001 Alternative fully DCT-based video codec architectures have been proposed in the past to address the shortcomings of the conven- tional hybrid motion compensated DCT video codec structures traditionally chosen as the basis of implementation of standard- compliant codecs. However, no prior effort has been made to ensure interoperability of these two drastically different architectures so that fully DCT-based video codecs are fully compatible with the existing video coding standards. In this paper, we establish the criteria for matching conventional codecs with fully DCT-based codecs. We find that the key to this interoperability lies in the heart of the implementation of motion compensation modules performed in the spatial and transform domains at both the encoder and the decoder. Specifically,if the spatial-domain motion compensation is compatible with the transform-domain motion compensation, then the states in both the coder and the decoder will keep track of each other even after a long series of P-frames. Otherwise, the states will diverge in proportion to the number of P-frames between two I-frames. This sets an important criterion for the development of any DCT-based motion compensation schemes.
    [Show full text]
  • A Dynamic Motion Vector Referencing Scheme for Video Coding
    A DYNAMIC MOTION VECTOR REFERENCING SCHEME FOR VIDEO CODING Jingning Han, Yaowu Xu, and James Bankoski WebM Codec Team, Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043 Emails: fjingning,yaowu,[email protected] ABSTRACT poral neighbors. Prior research exploits such correlations to improve coding efficiency [2]-[4]. Video codecs exploit temporal redundancy in video signals, direct mode through the use of motion compensated prediction, to achieve A derived motion vector mode named is pro- superior compression performance. The coding of motion posed in [5] for bi-directional predicted blocks. Unlike the vectors takes a large portion of the total rate cost. Prior re- conventional inter block that needs to send the motion vector search utilizes the spatial and temporal correlation of the mo- residual to the decoder, this mode only infers motion vector tion field to improve the coding efficiency of the motion in- from previously coded blocks. To determine the motion vec- formation. It typically constructs a candidate pool composed tor for each reference frame, the scheme checks the neigh- of a fixed number of reference motion vectors and allows the boring blocks in the order of above, left, top-right, and top- codec to select and reuse the one that best approximates the left, picks the first one that has the same reference frame, and motion of the current block. This largely disconnects the en- reuses its motion vector. A rate-distortion optimization ap- tropy coding process from the block’s motion information, proach that allows the codec to select between two reference and throws out any information related to motion consistency, motion vector candidates is proposed in [6].
    [Show full text]
  • COMPRESSIVE TECHNIQUES APPLICABLE for VIDEO COMPRESSION Jayshree R
    COMPRESSIVE TECHNIQUES APPLICABLE FOR VIDEO COMPRESSION Jayshree R. Pansare1, Ketki R. Jadhav2 1,2 Department of Computer Engineering, M.E.S. College of Engineering, Pune, India Abstract there lot of techniques for compression of Communication over internet is becoming images/ videos, finding better compression important part of our lives. Now days techniques for diversified applications or millions of peoples spend their time on you improving performance of existing techniques is tube, video communication and video still concern of lot of researchers. conferencing etc. Videos require large Images/videos are either get compressed by amount of bandwidth and transmission time. using lossless compression methods consist of Lot of video compression standards, Run length encoding (RLE), LZW ( Lempel Ziv- techniques and algorithms had been Welch) Coding, Huffman coding, Area Coding developed to reduce data quantity and gain or using lossy compression techniques consist of expected quality as possible as can. Video Transformation coding, Fractal segmentation, compressions technologies are now become Vector quantization, Sub band coding and Block the important part of the way we create, truncation coding. Even though most of us want consume and communicate visual image representation of content rather than text information. Paper reviews no of video for better understanding and visual appearance, compression techniques. It divides those videos turned out to be standards now days and techniques based on used techniques and used used by lot of applications such as DVD, digital concepts in survey papers. And shows that for TV, HDTV, Video calls and teleconferencing. which type of videos these techniques are Because of lot of advancement in network useful.
    [Show full text]
  • Lecture 13 Review Questions.Pdf
    Multimedia communications ECP 610 Omar A. Nasr [email protected] May, 2015 1 Take home messages from the course: Google, Facebook, Microsoft and other content providers would like to take over the whole stack! Google 2 3 4 5 6 7 Multimedia communications involves: Media coding (Speech, Audio, Images, Video coding) Media transmission Media coding (compression) Production models (For speech, music, image, video) Perception models Auditory system Masking Ear sensitivity to different frequency ranges Visual system Brightness vs color Low frequency vs High frequency Quality of Service and Perception QoS for voice QoS for video 8 Speech production system 9 10 Perception system 11 12 Digital Image Representation (3 Bit Quantization) CS 414 - Spring 2009 Image Representations Black and white image single color plane with 1 bits Gray scale image single color plane with 8 bits Color image three color planes each with 8 bits RGB, CMY, YIQ, etc. Indexed color image single plane that indexes a color table Compressed images TIFF, JPEG, BMP, etc. 4 gray levels 2gray levels Image Representation Example 24 bit RGB Representation (uncompressed) 128 135 166 138 190 132 129 255 105 189 167 190 229 213 134 111 138 187 128 138 135 190 166 132 129 189 255 167 105 190 229 111 213 138 134 187 Color Planes Techniques used in coding: Loss-Less compression : Huffman coding Variable length, prefix, uniquely decodable code Main objective: optimally assign different number of bits to symbols having different frequencies 16 Discrete
    [Show full text]
  • NEW METHODS for MOTION ESTIMATION with APPLICATIONS to LOW COMPLEXITY VIDEO COMPRESSION a Thesis Submitted to the Faculty Of
    NEW METHODS FOR MOTION ESTIMATION WITH APPLICATIONS TO LOW COMPLEXITY VIDEO COMPRESSION A Thesis Submitted to the Faculty of Purdue University by Zhen Li In Partial Ful¯llment of the Requirements for the Degree of Doctor of Philosophy December 2005 ii To my parents, Meilian Jin and Yanzong Li; To my wife, Limin Liu; and in memory of my grandmother, Zhugu Cai. iii ACKNOWLEDGMENTS I would like to thank my advisor, Professor Edward J. Delp, for his guidance and support. I truly appreciate the research freedom he allowed. And I am deeply grateful for his reminder to keep focused when I was zigzagging with various research topics, and his advice to look deeper into one challenging problem rather than playing around with many problems. I would also like to thank my Doctoral Committee: Professors Charles A. Bouman, Michael D. Zoltowski, and Dongyan Xu, for their advice, encouragement, and in- sights. I am grateful to the organizations which supported the research in this disser- tation, in particular the Indiana Twenty-First Century Research and Technology Fund. I appreciate the support and friendship of all my colleagues in the Video and Image Processing (VIPER) lab. It has been a joyful and memorable journey with these excellent people and scholars: Dr. Eduardo Asbun, Dr. Gregory Cook, Dr. Paul Salama, Dr. Lauren Christopher, Dr. Eugene Lin, Dr. Zoe Yuxin Liu, Dr. Yajie Sun, Dr. Jinwha Yang, Dr. Cuneyt Taskiran, Dr. Sahng-Gyu Park, Hyung Cook Kim, Jennifer Talavage, Hwayoung Um, Anthony Martone, Aravind Mikkili- neni, Oriol Guitart, Michael Igarta, Liang Liang, Ying Chen, Fengqing Zhu, Ashok Mariappan and Hakeem Ogunleye.
    [Show full text]