DOCSLIB.ORG

Lec 09 - Video Processing I Motion Estimation & Compensation

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Lec 09 - Video Processing I Motion Estimation & Compensation ECE 5578 Multimedia Communication Lec 09 - Video Processing I Motion Estimation & Compensation Zhu Li Dept of CSEE, UMKC Office: FH560E, Email: [email protected], Ph: x 2346. http://l.web.umkc.edu/lizhu slides created with WPS Office Linux and EqualX LaTex equation editor Z. Li Multimedia Communciation, 2018 p.1 Outline Lecture 07 Re-Cap Arithmetic Coding HW Color Space and Sampling Motion Estimation and Compensation . Optical Flow (pixel based) . Block Based Motion Estimation . Fast Block Motion Estimation Z. Li Multimedia Communciation, 2018 p.2 Scalar Quantization Uniform Quantizer & Distortions Distortion Metrics: ¥ M bi . MSE = ˆ - 2 = - 2 d ò (x x) f (x)dx å ò (yi x) f (x)dx i=1 -¥ bi-1 . Uniform Q MSE: D 1 2 1 1 1 d = M ò ( x - D / 2) dx = D3 = D2 M D 0 D 12 12 Z. Li Multimedia Communciation, 2018 p.3 Scalar Quantization Non-Uniform Scalar Quantization f(x) . Intuition: denser sampling at higher pdf region ¥ M bk = ˆ - 2 = - 2 d({bk , yk }) ò (x x) f (x)dx å ò (yk x) f (x)dx k =1 -¥ bk-1 x . Formulation: minimize d() over {bk, yk}, take Lagrangian and via KKT condition: ¶d 2 2 = 0 Þ (yi - bi ) f (bi ) - (yi+1 - bi ) f (bi ) = 0 ¶bi yi + yi+1 Þ bi = 2 bi ò x f (x)dx ¶d b i-1 y is the centroid ! = 0 Þ yi = E{X | X Î I i }= i ¶y bi i ò f (x)dx bi-1 Z. Li Multimedia Communciation, 2018 p.4 Vector Quantizer A more optimal solution, better approx. R-D info theoretical boundary kmeans() . % desired rate . R=8; . [indx, vq_codebook]=kmeans(x, 2^R); kd-tree implementation . [kdt.indx, kdt.leafs, kdt.mbox]=buildVisualWordList(x, 2^R); . [node, prefix_code]=searchVisualWordList(q, kdt.indx, kdt.leafs); Z. Li Multimedia Communciation, 2018 p.5 Arithmetic Coding Quantization/ExpGolomb Binarization . Imfilter(im, f1) . Res = res(: ) – mean(res(: )) . ResQ = fix(Res/Delta); Arithmetic Coding . SFU code, thanks for Prof. Jie Liang ! Binary Image Adaptive Arithmetic Coding . Re-use HEVC BAC DNA sequence coding (bonus) . Re-use HEVC BAC Will have more Lab sessions to help if necessary, don’t worry. Z. Li Multimedia Communciation, 2018 p.6 Outline Lecture 07 Re-Cap Color Space and Sampling Motion Estimation and Compensation . Optical Flow (pixel based) . Block Based Motion Estimation . Fast Block Motion Estimation Z. Li Multimedia Communciation, 2018 p.7 Digital Video Basics Frame 1 51 71 91 111 Neighboring frames are usually very similar: . Prediction coding will be very efficient. Typical video coding methods: Motion compensated hybrid video coding: most popular approach Block-based motion estimation/compensation Block transform (DCT) Wavelet based video coding (AVC Scalable Video Coding (Shi- Tah Hsiang, Motorola Lab) Model-based video coding: (MPEG4 object based coding) Use analysis/synthesis techniques. Encode model parameters for the decoder to synthesize. Z. Li Multimedia Communciation, 2018 p.8 Color Space Conversion RGB to YUV conversion: . Kind of backward compatibility with old BW TVs . Color does not carry as much info. Z. Li Multimedia Communciation, 2018 p.9 Color Space Conversion Matlab example . Play YUV sequences: ffplay -s '1920x1080' -pix_fmt yuv420p -f rawvideo Beauty_1920x1080_120fps_420_8bit_YUV.yuv . Access YUV sequences ffmpeg -f rawvideo -s cif -i stefan_cif.yuv -vcodec png stefan_yuv_%03d.png f0 = imread(‘Stefan_yuv_100.png '); Z. Li Multimedia Communciation, 2018 p.10 Color Space: Down-sampling RGB components of an image have strong correlation. Can be converted to YUV space for better compression. HVS is more sensitive to the details of brightness than color. Can down-sample color components to improve compression. Luma sample Chroma sample MPEG-1 MPEG-2 YUV 4:4:4 YUV 4:2:2 YUV 4:2:0 No downsampling • 2:1 horizontal downsampling •2:1 horizontal downsampling Of Chroma of chroma components of chroma components • 2 chroma samples for •1 chroma sample for every every 4 luma samples 4 luma samples Z. Li Multimedia Communciation, 2018 p.11 YUV 4:2:0 File Format In YUV 4:2:0, number of U and V samples are 1/4 of the Y samples YUV samples are stored separately: Image: YYYY…..Y UU…U VV…V Y (row by row in each channel) Video: YUV of frame 1, YUV of frame 2, …… U V CIF (Common Intermediate format): . 352 x 288 pixels for Y, 176 x 144 pixels for U, V QCIF (Quarter CIF): 176 x 144 pixels for Y, 88 x 72 pixels for U, V U: 88 x 72 V: 88 x 72 Y: 176 x 144 Z. Li Multimedia Communciation, 2018 p.12 Layered Structure for Video Data Video data layers: . Sequence layer Group of Picture (GOP) layer Picture (frame) layer Group of Block (GOB) or slice layer Macroblock (MB) layer block layer sub-block layer Block: usually 8x8 pixels (4x4 pixels in H.264) Macroblock (MB): 16 x 16 region An MB: . Includes 4 luma blocks, 2 chroma blocks. Group of Blocks (GOB) or slice: 8x8 Y1 Y2 Cr Cb . H.261: 3 rows of 11 MBs (176 x 48 pixels) Y3 Y4 o QCIF: 176 x 144 3 GOBs o CIF: 352 x 288 12 GOBs . Slice: More flexible def. in H.264 1 2 Picture layer: one frame 3 4 GOP layer: Group of frames 5 6 7 8 Sequence: entire video sequence. 1 9 10 One GOB 2 3 11 12 QCIF: 176 x 144 CIF: 352 x 288 Z. Li Multimedia Communciation, 2018 p.13 Outline Lecture 07 Re-Cap Color Space and Sampling Motion Estimation and Compensation . Optical Flow (pixel based) . Block Based Motion Estimation . Fast Block Motion Estimation Z. Li Multimedia Communciation, 2018 p.14 Key Idea in Video Coding Predictive coding: Predict each frame from the previous frame(s) and only encode the prediction error: . Pred. error has smaller energy and is easier to compress Prediction can be performed at Frame level, Macroblock level, Block level, or even sub-block level. x Current Frame y Previous frame Z. Li Multimedia Communciation, 2018 p.15 Motion Field Image velocity of a point moving in the scene vodt X f ' ro Z dr ri = o d Y Scene point velocity: vo vi t dt Image velocity: dr v = i i dt 1 ro Perspective projection: ri = f ' ro × Z Motion field dri (ro × Z)vo - (vo × Z)ro (ro ´ vo )´ Z vi = = f ' 2 = f ' 2 dt (ro × Z) (ro × Z) Z. Li Multimedia Communciation, 2018 p.16 Optical Flow Optical Flow: OPTICAL FLOW = apparent motion of brightness patterns . Ideally a projection of 3d obj motion to 2d image plane Z. Li Multimedia Communciation, 2018 p.17 Lucas-Kanade OF estimator Optical Flow . I(x, y, t) is the pixel value at location (x,y) on image plane at time t . Brightness constancy assumption: dx dy I(x + dt, y + dt,t +dt) = I(x, y,t) dt dt . Optical Flow Constraint Equation: dI ¶I dx ¶I dy ¶I = + + = 0 dt ¶x dt ¶y dt ¶t This leads to Lucas-Kanade algorithm (details skipped) . Optimal OF [u, v] should satisfy: Z. Li Multimedia Communciation, 2018 p.18 Block Based Motion Estimation (ME) For each ME unit (MB or block), find the best match in the previous frame (reference frame) . Upper-left corner of the block being encoded: (x0, y0) . Upper-left corner of the matched block in the reference frame: (x1, y1) . Motion vector (dx, dy): the offset of the two blocks: o (dx, dy) = (x1 – x0, y1 – y0) = (x1, y1) - (x0, y0) o (x0, y0) + (dx, dy) = (x1, y1) . Motion vector need to be sent to the decoder. (x1, y1) Prediction error: B (x0, y0) e = A – B. A Z. Li Multimedia Communciation, 2018 p.19 GOP, I, P, and B Frames …… …… I P P P … P P P I P P P … P P P GOP GOP GOP: Group of pictures (frames). I frames (Key frames): . Intra-coded frame, coded as a still image. Can be decoded directly. Used at GOP head, or at scene changes. Allow random access, improves error resilience. P frames: (Inter-coded frames) . Predicated from the previous frame. Z. Li Multimedia Communciation, 2018 p.20 GOP, I, P, and B Frames B frames: Bi-directional interpolated prediction frames . Predicted from both the previous frame and the next frame: more flexibilities better prediction. Useful when new objects come into the scene. Before H.264, B frames are not used as reference for future frames: . B frames can be coded with lower quality or can be discarded without affecting future frames. Allow temporal scalability Encoding/Decoding order: 1 4 2 3 7 5 6 Display order: 1 2 3 4 5 6 7 1 2 3 4 5 6 7 Need more buffers I B B P B B P Z. Li Multimedia Communciation, 2018 p.21 Basic Encoder Block Diagram Intra Input DCT Q Entropy frame Coding -1 Inter Q Pred. error I DCT Intra Recon Pred error Prediction Inter Recon. MC Memory Prediction Motion vectors ME Reconstructed Previous frame Motion compensation (MC): get the prediction of the current frame using MV information. Use reconstructed frame in the loop to prevent drifting. Z. Li Multimedia Communciation, 2018 p.22 Basic Decoder Block Diagram Intra Reconstructed frame -1 Entropy Q I DCT Decoding Recon Inter Pred error MC Memory Prediction Motion vectors Reconstructed Decoder is simpler than the encoder: Previous frame . No need to do motion estimation. Z. Li Multimedia Communciation, 2018 p.23 Motion Estimation For the k-th MB, the prediction error with MV (dx, dy): N -1 N -1 p e (k,dx,dy) = åå fn (xk + i, yk + j)- fn-1(xk + dx + i, yk + dy + j) i=0 j=0 xk , yk : The upper - left coordinate of the k - th MB; fn (x,y) : The (x, y)- th pixel in Frame n; N : ME block size (16 for MB level ME) p = 1: Sum of Absolute Difference (SAD) p = 2: Sum of Squared Difference (SSD) (-R, -R) Number of MV candidates: (2R+1) x (2R + 1) Objective: Find the MV that minimizes the pred error.
Load more

Recommended publications
  • On Computational Complexity of Motion Estimation Algorithms in MPEG-4 Encoder
    On Computational Complexity of Motion Estimation Algorithms in MPEG-4 Encoder Muhammad Shahid This thesis report is presented as a part of degree of Master of Science in Electrical Engineering Blekinge Institute of Technology, 2010 Supervisor: Tech Lic. Andreas Rossholm, ST-Ericsson Examiner: Dr. Benny Lovstrom, Blekinge Institute of Technology Abstract Video Encoding in mobile equipments is a computationally demanding fea- ture that requires a well designed and well developed algorithm. The op- timal solution requires a trade off in the encoding process, e.g. motion estimation with tradeoff between low complexity versus high perceptual quality and efficiency. The present thesis works on reducing the complexity of motion estimation algorithms used for MPEG-4 video encoding taking SLIMPEG motion estimation algorithm as reference. The inherent prop- erties of video like spatial and temporal correlation have been exploited to test new techniques of motion estimation. Four motion estimation algo- rithms have been proposed. The computational complexity and encoding quality have been evaluated. The resulting encoded video quality has been compared against the standard Full Search algorithm. At the same time, reduction in computational complexity of the improved algorithm is com- pared against SLIMPEG which is already about 99 % more efficient than Full Search in terms of computational complexity. The fourth proposed algorithm, Adaptive SAD Control, offers a mechanism of choosing trade off between computational complexity and encoding quality in a dynamic way. Acknowledgements It is a matter of great pleasure to express my deepest gratitude to my ad- visors Dr. Benny L¨ovstr¨om and Andreas Rossholm for all their guidance, support and encouragement throughout my thesis work.
    [Show full text]
  • 1. in the New Document Dialog Box, on the General Tab, for Type, Choose Flash Document, Then Click______
    1. In the New Document dialog box, on the General tab, for type, choose Flash Document, then click____________. 1. Tab 2. Ok 3. Delete 4. Save 2. Specify the export ________ for classes in the movie. 1. Frame 2. Class 3. Loading 4. Main 3. To help manage the files in a large application, flash MX professional 2004 supports the concept of _________. 1. Files 2. Projects 3. Flash 4. Player 4. In the AppName directory, create a subdirectory named_______. 1. Source 2. Flash documents 3. Source code 4. Classes 5. In Flash, most applications are _________ and include __________ user interfaces. 1. Visual, Graphical 2. Visual, Flash 3. Graphical, Flash 4. Visual, AppName 6. Test locally by opening ________ in your web browser. 1. AppName.fla 2. AppName.html 3. AppName.swf 4. AppName 7. The AppName directory will contain everything in our project, including _________ and _____________ . 1. Source code, Final input 2. Input, Output 3. Source code, Final output 4. Source code, everything 8. In the AppName directory, create a subdirectory named_______. 1. Compiled application 2. Deploy 3. Final output 4. Source code 9. Every Flash application must include at least one ______________. 1. Flash document 2. AppName 3. Deploy 4. Source 10. In the AppName/Source directory, create a subdirectory named __________. 1. Source 2. Com 3. Some domain 4. AppName 11. In the AppName/Source/Com directory, create a sub directory named ______ 1. Some domain 2. Com 3. AppName 4. Source 12. A project is group of related _________ that can be managed via the project panel in the flash.
    [Show full text]
  • Video Coding Standards
    Video coding standards Video signals represent sequences of images or frames which can be transmitted with a rate from 15 to 60 frames per second (fps), that provides the illusion of motion in the displayed signal. Unlike images they contain the so-called temporal redundancy. Temporal redundancy arises from repeated objects in consecutive frames of the video sequence. Such objects can remain, they can move horizontally, vertically, or any combination of directions (translation movement), they can fade in and out, and they can disappear from the image as they move out of view. Temporal redundancy Motion compensation A motion compensation technique is used to compensate the temporal redundancy of video sequence. The main idea of the this method is to predict the displacement of group of pixels (usually block of pixels) from their position in the previous frame. Information about this displacement is represented by motion vectors which are transmitted together with the DCT coded difference between the predicted and the original images. Motion compensation Image VLC DCT Quantizer Buffer - Dequantizer Coded DCT coefficients. IDCT Motion compensated predictor Motion Motion vectors estimation Decoded VL image Buffer Dequantizer IDCT decoder Motion compensated predictor Motion vectors Motion compensation Previous frame Current frame Set of macroblocks of the previous frame used to predict the selected macroblock of the current frame Motion compensation Previous frame Current frame Macroblocks of previous frame used to predict current frame Motion compensation Each 16x16 pixel macroblock in the current frame is compared with a set of macroblocks in the previous frame to determine the one that best predicts the current macroblock.
    [Show full text]
  • An Overview of Block Matching Algorithms for Motion Vector Estimation
    Proceedings of the Second International Conference on Research in DOI: 10.15439/2017R85 Intelligent and Computing in Engineering pp. 217–222 ACSIS, Vol. 10 ISSN 2300-5963 An Overview of Block Matching Algorithms for Motion Vector Estimation Sonam T. Khawase1, Shailesh D. Kamble2, Nileshsingh V. Thakur3, Akshay S. Patharkar4 1PG Scholar, Computer Science & Engineering, Yeshwantrao Chavan College of Engineering, India 2Computer Science & Engineering, Yeshwantrao Chavan College of Engineering, India 3Computer Science & Engineering, Prof Ram Meghe College of Engineering & Management, India 4Computer Technology, K.D.K. College of Engineering, India [email protected], [email protected],[email protected], [email protected] Abstract–In video compression technique, motion estimation is one of the key components because of its high computation complexity involves in finding the motion vectors (MV) between the frames. The purpose of motion estimation is to reduce the storage space, bandwidth and transmission cost for transmission of video in many multimedia service applications by reducing the temporal redundancies while maintaining a good quality of the video. There are many motion estimation algorithms, but there is a trade-off between algorithms accuracy and speed. Among all of these, block-based motion estimation algorithms are most robust and versatile. In motion estimation, a variety of fast block based matching algorithms has been proposed to address the issues such as reducing the number of search/checkpoints, computational cost, and complexities etc. Due to its simplicity, the Fig. 1. Classification of Frames block-based technique is most popular. Motion estimation is only known for video coding process but for solving real life redundancies.
    [Show full text]
  • CHAPTER 10 Basic Video Compression Techniques Contents
    Multimedia Network Lab. CHAPTER 10 Basic Video Compression Techniques Multimedia Network Lab. Prof. Sang-Jo Yoo http://multinet.inha.ac.kr The Graduate School of Information Technology and Telecommunications, INHA University http://multinet.inha.ac.kr Multimedia Network Lab. Contents 10.1 Introduction to Video Compression 10.2 Video Compression with Motion Compensation 10.3 Search for Motion Vectors 10.4 H.261 10.5 H.263 The Graduate School of Information Technology and Telecommunications, INHA University 2 http://multinet.inha.ac.kr Multimedia Network Lab. 10.1 Introduction to Video Compression A video consists of a time-ordered sequence of frames, i.e.,images. Why we need a video compression CIF (352x288) : 35 Mbps HDTV: 1Gbps An obvious solution to video compression would be predictive coding based on previous frames. Compression proceeds by subtracting images: subtract in time order and code the residual error. It can be done even better by searching for just the right parts of the image to subtract from the previous frame. Motion estimation: looking for the right part of motion Motion compensation: shifting pieces of the frame The Graduate School of Information Technology and Telecommunications, INHA University 3 http://multinet.inha.ac.kr Multimedia Network Lab. 10.2 Video Compression with Motion Compensation Consecutive frames in a video are similar – temporal redundancy Frame rate of the video is relatively high (30 frames/sec) Camera parameters (position, angle) usually do not change rapidly. Temporal redundancy is exploited so that not every frame of the video needs to be coded independently as a new image. The difference between the current frame and other frame(s) in the sequence will be coded – small values and low entropy, good for compression.
    [Show full text]
  • Motion Estimation at the Decoder Sven Klomp and Jorn¨ Ostermann Leibniz Universit¨At Hannover Germany
    5 Motion Estimation at the Decoder Sven Klomp and Jorn¨ Ostermann Leibniz Universit¨at Hannover Germany 1. Introduction All existing video coding standards, such as MPEG-1,2,4 or ITU-T H.26x, perform motion estimation at the encoder in order to exploit temporal dependencies within the video sequence. The estimated motion vectors are transmitted and used by the decoder to assemble a prediction of the current frame. Since only the prediction error and the motion information are transmitted, instead of intra coding the pixel values, compression is achieved. Due to block-based motion estimation, accurate compensation at object borders can only be achieved with small block sizes. Large blocks may contain several objects which move in different directions. Thus, accurate motion estimation and compensation is not possible, as shown by Klomp et al. (2010a) using prediction error variance, and small block sizes are favourable. However, the smaller the block, the more motion vectors have to be transmitted, resulting in a contradiction to bit rate reduction. 4.0 3.5 3.0 2.5 MV Backward MV Forward MV Bidirectional 2.0 RS Backward Rate (bit/pixel) RS Forward 1.5 RS Bidirectional 1.0 0.5 0.0 2 4 8 16 32 Block Size Fig. 1. Data rates of residual (RS) and motion vectors (MV) for different motion compensation techniques (Kimono sequence). 782 Effective Video Coding for MultimediaVideo Applications Coding These characteristics can be observed in Figure 1, where the rates for the residual and the motion vectors are plotted for different block sizes and three prediction techniques.
    [Show full text]
  • Motion Vector Forecast and Mapping (MV-Fmap) Method for Entropy Coding Based Video Coders Julien Le Tanou, Jean-Marc Thiesse, Joël Jung, Marc Antonini
    Motion Vector Forecast and Mapping (MV-FMap) Method for Entropy Coding based Video Coders Julien Le Tanou, Jean-Marc Thiesse, Joël Jung, Marc Antonini To cite this version: Julien Le Tanou, Jean-Marc Thiesse, Joël Jung, Marc Antonini. Motion Vector Forecast and Mapping (MV-FMap) Method for Entropy Coding based Video Coders. MMSP’10 2010 IEEE International Workshop on Multimedia Signal Processing, Oct 2010, Saint Malo, France. pp.206. hal-00531819 HAL Id: hal-00531819 https://hal.archives-ouvertes.fr/hal-00531819 Submitted on 3 Nov 2010 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Motion Vector Forecast and Mapping (MV-FMap) Method for Entropy Coding based Video Coders Julien Le Tanou #1, Jean-Marc Thiesse #2, Joël Jung #3, Marc Antonini ∗4 # Orange Labs 38 rue du G. Leclerc, 92794 Issy les Moulineaux, France 1 [email protected] {2 jeanmarc.thiesse,3 joelb.jung}@orange-ftgroup.com ∗ I3S Lab. University of Nice-Sophia Antipolis/CNRS 2000 route des Lucioles, 06903 Sophia Antipolis, France 4 [email protected] Abstract—Since the finalization of the H.264/AVC standard between motion vectors of neighboring frames and blocks, we and in order to meet the target set by both ITU-T and MPEG propose in this paper a method for motion vector coding based to define a new standard that reaches 50% bit rate reduction on a motion vector residuals forecast followed by an adaptive compared to H.264/AVC, many tools have efficiently improved the texture coding and the motion compensation accuracy.
    [Show full text]
  • Multiple Reference Motion Compensation: a Tutorial Introduction and Survey Contents
    Foundations and TrendsR in Signal Processing Vol. 2, No. 4 (2008) 247–364 c 2009 A. Leontaris, P. C. Cosman and A. M. Tourapis DOI: 10.1561/2000000019 Multiple Reference Motion Compensation: A Tutorial Introduction and Survey By Athanasios Leontaris, Pamela C. Cosman and Alexis M. Tourapis Contents 1 Introduction 248 1.1 Motion-Compensated Prediction 249 1.2 Outline 254 2 Background, Mosaic, and Library Coding 256 2.1 Background Updating and Replenishment 257 2.2 Mosaics Generated Through Global Motion Models 261 2.3 Composite Memories 264 3 Multiple Reference Frame Motion Compensation 268 3.1 A Brief Historical Perspective 268 3.2 Advantages of Multiple Reference Frames 270 3.3 Multiple Reference Frame Prediction 271 3.4 Multiple Reference Frames in Standards 277 3.5 Interpolation for Motion Compensated Prediction 281 3.6 Weighted Prediction and Multiple References 284 3.7 Scalable and Multiple-View Coding 286 4 Multihypothesis Motion-Compensated Prediction 290 4.1 Bi-Directional Prediction and Generalized Bi-Prediction 291 4.2 Overlapped Block Motion Compensation 294 4.3 Hypothesis Selection Optimization 296 4.4 Multihypothesis Prediction in the Frequency Domain 298 4.5 Theoretical Insight 298 5 Fast Multiple-Frame Motion Estimation Algorithms 301 5.1 Multiresolution and Hierarchical Search 302 5.2 Fast Search using Mathematical Inequalities 303 5.3 Motion Information Re-Use and Motion Composition 304 5.4 Simplex and Constrained Minimization 306 5.5 Zonal and Center-biased Algorithms 307 5.6 Fractional-pixel Texture Shifts or Aliasing
    [Show full text]
  • Overview: Motion-Compensated Coding
    Overview: motion-compensated coding Motion-compensated prediction Motion-compensated hybrid coding Motion estimation by block-matching Motion estimation with sub-pixel accuracy Power spectral density of the motion-compensated prediction error Rate-distortion analysis Loop filter Motion compensated coding with sub-pixel accuracy Rate-constrained motion estimation Bernd Girod: EE398B Image Communication II Motion Compensated Coding no. 1 Motion-compensated prediction previous frame stationary Δt background current frame x time t y moving object ⎛ d x ⎞ „Displacement vector“ ⎜ d ⎟ shifted ⎝ y ⎠ object Prediction for the luminance signal S(x,y,t) within the moving object: ˆ S (x, y,t) = S(x − dx , y − dy ,t − Δt) Bernd Girod: EE398B Image Communication II Motion Compensated Coding no. 2 Combining transform coding and prediction Transform domain prediction Space domain prediction Q Q T - T - T T −1 T −1 T −1 PT T PTS T T −1 T−1 PT PS Bernd Girod: EE398B Image Communication II Motion Compensated Coding no. 3 Motion-compensated hybrid coder Coder Control Control Data Intra-frame DCT DCT Coder - Coefficients Decoder Intra-frame Decoder 0 Motion- Compensated Intra/Inter Predictor Motion Data Motion Estimator Bernd Girod: EE398B Image Communication II Motion Compensated Coding no. 4 Motion-compensated hybrid decoder Control Data DCT Coefficients Decoder Intra-frame Decoder 0 Motion- Compensated Intra/Inter Predictor Motion Data Bernd Girod: EE398B Image Communication II Motion Compensated Coding no. 5 Block-matching algorithm search range in Subdivide current reference frame frame into blocks. Sk −1 Find one displacement vector for each block. Within a search range, find a best „match“ that minimizes an error measure.
    [Show full text]
  • Efficient Video Coding with Motion-Compensated Orthogonal
    Efficient Video Coding with Motion-Compensated Orthogonal Transforms DU LIU Master’s Degree Project Stockholm, Sweden 2011 XR-EE-SIP 2011:011 Efficient Video Coding with Motion-Compensated Orthogonal Transforms Du Liu July, 2011 Abstract Well-known standard hybrid coding techniques utilize the concept of motion- compensated predictive coding in a closed-loop. The resulting coding de- pendencies are a major challenge for packet-based networks like the Internet. On the other hand, subband coding techniques avoid the dependencies of predictive coding and are able to generate video streams that better match packet-based networks. An interesting class for subband coding is the so- called motion-compensated orthogonal transform. It generates orthogonal subband coefficients for arbitrary underlying motion fields. In this project, a theoretical lossless signal model based on Gaussian distribution is proposed. It is possible to obtain the optimal rate allocation from this model. Addition- ally, a rate-distortion efficient video coding scheme is developed that takes advantage of motion-compensated orthogonal transforms. The scheme com- bines multiple types of motion-compensated orthogonal transforms, variable block size, and half-pel accurate motion compensation. The experimental results show that this scheme outperforms individual motion-compensated orthogonal transforms. i Acknowledgements This thesis was carried out at Sound and Image Processing Lab, School of Electrical Engineering, KTH. I would like to express my appreciation to my supervisor Markus Flierl for the opportunity of doing this thesis. I am grateful for his patience and valuable suggestions and discussions. Many thanks to Haopeng Li, Mingyue Li, and Zhanyu Ma, who helped me a lot during my research.
    [Show full text]
  • Respiratory Motion Compensation Using Diaphragm Tracking for Cone-Beam C-Arm CT: a Simulation and a Phantom Study
    Hindawi Publishing Corporation International Journal of Biomedical Imaging Volume 2013, Article ID 520540, 10 pages http://dx.doi.org/10.1155/2013/520540 Research Article Respiratory Motion Compensation Using Diaphragm Tracking for Cone-Beam C-Arm CT: A Simulation and a Phantom Study Marco Bögel,1 Hannes G. Hofmann,1 Joachim Hornegger,1 Rebecca Fahrig,2 Stefan Britzen,3 and Andreas Maier3 1 Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg, 91058 Erlangen, Germany 2 Department of Radiology, Lucas MRS Center, Stanford University, Palo Alto, CA 94304, USA 3 Siemens AG, Healthcare Sector, 91301 Forchheim, Germany Correspondence should be addressed to Andreas Maier; [email protected] Received 21 February 2013; Revised 13 May 2013; Accepted 15 May 2013 Academic Editor: Michael W. Vannier Copyright © 2013 Marco Bogel¨ et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Long acquisition times lead to image artifacts in thoracic C-arm CT. Motion blur caused by respiratory motion leads to decreased image quality in many clinical applications. We introduce an image-based method to estimate and compensate respiratory motion in C-arm CT based on diaphragm motion. In order to estimate respiratory motion, we track the contour of the diaphragm in the projection image sequence. Using a motion corrected triangulation approach on the diaphragm vertex, we are able to estimate a motion signal. The estimated motion signal is used to compensate for respiratory motion in the target region, for example, heart or lungs.
    [Show full text]
  • LOW COMPLEXITY H.264 to VC-1 TRANSCODER by VIDHYA
    LOW COMPLEXITY H.264 TO VC-1 TRANSCODER by VIDHYA VIJAYAKUMAR Presented to the Faculty of the Graduate School of The University of Texas at Arlington in Partial Fulfillment of the Requirements for the Degree of MASTER OF SCIENCE IN ELECTRICAL ENGINEERING THE UNIVERSITY OF TEXAS AT ARLINGTON AUGUST 2010 Copyright © by Vidhya Vijayakumar 2010 All Rights Reserved ACKNOWLEDGEMENTS As true as it would be with any research effort, this endeavor would not have been possible without the guidance and support of a number of people whom I stand to thank at this juncture. First and foremost, I express my sincere gratitude to my advisor and mentor, Dr. K.R. Rao, who has been the backbone of this whole exercise. I am greatly indebted for all the things that I have learnt from him, academically and otherwise. I thank Dr. Ishfaq Ahmad for being my co-advisor and mentor and for his invaluable guidance and support. I was fortunate to work with Dr. Ahmad as his research assistant on the latest trends in video compression and it has been an invaluable experience. I thank my mentor, Mr. Vishy Swaminathan, and my team members at Adobe Systems for giving me an opportunity to work in the industry and guide me during my internship. I would like to thank the other members of my advisory committee Dr. W. Alan Davis and Dr. William E Dillon for reviewing the thesis document and offering insightful comments. I express my gratitude Dr. Jonathan Bredow and the Electrical Engineering department for purchasing the software required for this thesis and giving me the chance to work on cutting edge technologies.
    [Show full text]