FUNDAMENTALS OF DIGITAL VIDEO COMPRESSION MPEG Compression Streaming Video and Progressive Download © 2016 Pearson Education, Inc., Hoboken, NJ. All 1 rights reserved. In this lecture, you will learn: • Basic concepts of MPEG compression • Types of MPEG • Applications of MPEG • What GOP (group of pictures) in MPEG is • How GOP settings affect MPEG video file size • How GOP settings affect MPEG picture quality • What streaming video and progressive download are © 2016 Pearson Education, Inc., Hoboken, NJ. All 2 rights reserved. MPEG • Moving Pictures Experts Group Committee who derives standards for encoding video • Allow high compression • MPEG-1, MPEG-2, MPEG-4 © 2016 Pearson Education, Inc., Hoboken, NJ. All 3 rights reserved. What Happend to MPEG-3? • NOT MP3 (which is audio format) • Intended for HDTV • HDTV specifications was merged into MPEG-2 © 2016 Pearson Education, Inc., Hoboken, NJ. All 4 rights reserved. MPEG-1 • Video quality comparable to VHS • Originally intended for Web and CD-ROM playback • Frame sizes up to 352 × 240 pixels • Video format for VCD (VideoCD) before DVD became widespread © 2016 Pearson Education, Inc., Hoboken, NJ. All 5 rights reserved. MPEG-2 • Supports DVD-video, HDTV, HDV standards • For DVD video production: Export video into DVD MPEG-2 format • For HDV video production: Export video into HDV's MPEG-2 format © 2016 Pearson Education, Inc., Hoboken, NJ. All 6 rights reserved. MPEG-4 • Newer standard of MPEG family • Different encoding approach from MPEG-1 and MPEG-2 (will discuss after MPEG-1 and MPEG-2 compression in this lecture) © 2016 Pearson Education, Inc., Hoboken, NJ. All 7 rights reserved. MPEG-1 AND MPEG-2 How the Compression Works © 2016 Pearson Education, Inc., Hoboken, NJ. All 8 rights reserved. Properties of Typical Video • Neighboring frames are very similar • This property is called Temporal redundancy . • MPEG compression exploits temporal redundancy to reduce video file size • by looking for motion difference from one frame to the next • This technique is called motion compensation . © 2016 Pearson Education, Inc., Hoboken, NJ. All 9 rights reserved. Basic Ideas of Motion Compensation 1. An video frame image is read in as a reference frame 2. Read the next video frame image 3. Compare the image content between this current frame ( target frame ) with the reference frame • one block of pixels at a time © 2016 Pearson Education, Inc., Hoboken, NJ. All 10 rights reserved. Comparing Reference and Target Frames Case 1: If a pixel block is identical at the same location in both frames: • No need to encode the target frame's • Just save an instruction to refer to the block in the reference image— requires less space than encoding the whole pixel block © 2016 Pearson Education, Inc., Hoboken, NJ. All 11 rights reserved. Comparing Reference and Target Frames Case 2: If a pixel block at the same location in both frames is not identical: • Search for the reference image for a match (because the content may be moved to another location) • Case 2a: No match is found • Case 2b: A match is found © 2016 Pearson Education, Inc., Hoboken, NJ. All 12 rights reserved. Case 2a: No Match is Found • The whole pixel block is encoded • Thus, no saving in file size © 2016 Pearson Education, Inc., Hoboken, NJ. All 13 rights reserved. Case 2b: A Match is Found • The displacement information of the block is saved. • Displacement information: – A 2-dimensional value, indicating how much the block moves horizontally and vertically from the reference frame to the target frame – Called motion vector – Much smaller size than encoding the whole pixel blocks (thus the saving in file size) © 2016 Pearson Education, Inc., Hoboken, NJ. All 14 rights reserved. Illustration of the Ideas Let's consider the first 4 frames of a video Frame 1 Frame 2 Frame 3 Frame 4 The square grid cell in a frame represents a block of pixels. © 2016 Pearson Education, Inc., Hoboken, NJ. All 15 rights reserved. Illustration of the Ideas Frame 1 is read in. Frame 1 Frame 2 Frame 3 Frame 4 © 2016 Pearson Education, Inc., Hoboken, NJ. All 16 rights reserved. Illustration of the Ideas Frame 1 is used as a reference frame. Frame 1 Frame 2 Frame 3 Frame 4 The red color highlights the pixel blocks that are encoded. The whole frame of a reference frame is encoded. No saving in file size. © 2016 Pearson Education, Inc., Hoboken, NJ. All 17 rights reserved. Illustration of the Ideas Frame 2 is read in. Frame 1 Frame 2 Frame 3 Frame 4 © 2016 Pearson Education, Inc., Hoboken, NJ. All 18 rights reserved. Illustration of the Ideas Each pixel block in Frame 2 is searched in Frame 1 trying to find a match. Frame 1 Frame 2 Frame 3 Frame 4 © 2016 Pearson Education, Inc., Hoboken, NJ. All 19 rights reserved. Illustration of the Ideas Each pixel block in Frame 2 is searched in Frame 1 trying to find a match. Frame 1Frame Frame 2 Frame 3 Frame 4 The blue color highlights some pixel blocks that are found in Frame 1. The yellow color highlights some pixel blocks that are found in Frame 1 (but later in the process they are© 2016found Pearson in Education,the next Inc., refere Hoboken,nce NJ. All frame.) 20 rights reserved. Illustration of the Ideas Frame 3 is read in. Frame 1 Frame 2 Frame 3 Frame 4 © 2016 Pearson Education, Inc., Hoboken, NJ. All 21 rights reserved. Illustration of the Ideas Each pixel block in Frame 3 is searched in Frame 1 trying to find a match. Frame 1 Frame 2 Frame 3 Frame 4 The process is repeated similar to that for Frame 2. © 2016 Pearson Education, Inc., Hoboken, NJ. All 22 rights reserved. Illustration of the Ideas about Motion Vector Take Frames 1 and 2 as an example Frame 1 Frame 2 © 2016 Pearson Education, Inc., Hoboken, NJ. All 23 rights reserved. Illustration of the Ideas about Motion Vector Let's look at the pixel block showing the car's windshield Frame 1 Frame 2 © 2016 Pearson Education, Inc., Hoboken, NJ. All 24 rights reserved. Illustration of the Ideas about Motion Let'sVector arrange the frames vertically to see the displacement easier Frame 1 Frame 2 © 2016 Pearson Education, Inc., Hoboken, NJ. All 25 rights reserved. Illustration of the Ideas about Motion Let'sVector arrange the frames vertically to see the displacement easier Frame 1 Frame 2 © 2016 Pearson Education, Inc., Hoboken, NJ. All 26 rights reserved. Illustration of the Ideas about Motion Let'sVector arrange the frames vertically to see the displacement easier Frame 1 Frame 2 © 2016 Pearson Education, Inc., Hoboken, NJ. All 27 rights reserved. Illustration of the Ideas about Motion Let'sVector arrange the frames vertically to see the displacement easier Frame 1 Frame 2 The red line indicates the displacement of the pixel block. © 2016 Pearson Education, Inc., Hoboken, NJ. All 28 rights reserved. SO HOW ARE REFERENCE FRAMES CHOSEN? We will need to understand an important concept of group of pictures (GOP ) first. © 2016 Pearson Education, Inc., Hoboken, NJ. All 29 rights reserved. © 2016 Pearson Education, Inc., Hoboken, NJ. All 30 rights reserved. Motivation of Understanding GOP To understand: • the terminology related to GOP because – it is used by MPEG which in turns used by DVD video and HDTV – it comes up in video production application dialog boxes when you export video into MPEG format • the impact of the GOP parameters on video file size and quality • why some video editing programs may not support frame accurate editing of MPEG © 2016 Pearson Education, Inc., Hoboken, NJ. All 31 rights reserved. © 2016 Pearson Education, Inc., Hoboken, NJ. All 32 rights reserved. Group of Pictures (GOP) • Specifies the grouping structure of frames • Frame types in a structure: – I-frames – P-frames – B-frames • An MPEG video contains 1 or more repeating GOPs © 2016 Pearson Education, Inc., Hoboken, NJ. All 33 rights reserved. Number of Frames in a GOP • Each GOP in a video has a fixed number of frames • The number is the N parameter • Example: • DVD-compliant MPEG-3: N = 15 • HDV: N = 15 © 2016 Pearson Education, Inc., Hoboken, NJ. All 34 rights reserved. I-frames • Stands for Intraframes • Encoded only using the information within the frame-- intracoding . – Use spatial compression • Similar to JPEG compression – No temporal compression • Least compressed of the three types • Reference frame in the previous 4-frame example © 2016 Pearson Education, Inc., Hoboken, NJ. All 35 rights reserved. I-frames • Each GOP starts with an I-frame • Each GOP has only one I-frame © 2016 Pearson Education, Inc., Hoboken, NJ. All 36 rights reserved. P-frames • Stands for predicted frames • Encoded using the information from the previous I- or P-frames as the reference frame if a match of a pixel block is found • The 2 nd and 3 rd frames in the previous 4-video example © 2016 Pearson Education, Inc., Hoboken, NJ. All 37 rights reserved. B-frames • Stands for bidirectional frames • Between I- and P-frames • Encoded using information from the previous and subsequent I- and/or P-frame as the reference frames © 2016 Pearson Education, Inc., Hoboken, NJ. All 38 rights reserved. Example GOP DVD-compliant MPEG-2: I B B P B B P B B P B B P B B N = 15 i.e., number of frames in a GOP = 15 © 2016 Pearson Education, Inc., Hoboken, NJ. All 39 rights reserved. Example GOP DVD-compliant MPEG-2: I B B P B B P B B P B B P B B M = 3 i.e., number of frames between non-B-frames plus one = 2+1 = 3 © 2016 Pearson Education, Inc., Hoboken, NJ. All 40 rights reserved.