Overview: Video Coding Standards

„ Video coding standards: applications and common structure „ Relevant standards organizations „ ITU-T Rec. H.261 „ ITU-T Rec. H.263 „ ISO/IEC MPEG-1 „ ISO/IEC MPEG-2 „ ISO/IEC MPEG-4 „ Recent progress: H.264/AVC

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 1 ISO/IEC MPEG

„ MPEG-1 Standard (1991) (ISO/IEC 11172) z Target bit-rate about 1.5 Mbps z Typical image format CIF, no interlace z Frame rate 24 ... 30 fps z Main application: video storage for multimedia (e.g., on CD-ROM) „ MPEG-2 Standard (1994) (ISO/IEC 13818) z Extension for interlace, optimized for TV resolution (NTSC: 704 x 480 Pixel) z Image quality similar to NTSC, PAL, SECAM at 4 - 8 Mbps z HDTV at 20 Mbps „ MPEG-4 Standard (1999) (ISO/IEC 14496) z Object based coding z Wide-range of applications, with choices of interactivity, scalability, error resilience, etc.

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 2 MPEG-1/2: GOP Structure

„ "Group of Pictures" = “GOP“, GOP structure is very flexible

I-Picture P-Picture P-Picture

B-Pictures

time 1 3 4 2 6 7 8 5

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 3 MPEG-1 Encoder

Pre- Picture Video DCT Weighting Quantization VLC Buffer processing reordering - multiplex

Inverse quantization Motion vectors, Inverse macroblock info, Video in weighting start codes

Inverse DCT Bitstream

+ zero Picture store 1 Motion 1/2 + compensation

Picture store 2

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4 MPEG-1: coding of I-pictures

„ I-pictures: intraframe coded „ 8x8 DCT „ Arbitrary weighting matrix for coefficients „ Differential coding of DC-coefficients „ Uniform quantization „ Zig-zag-scan, run-level-coding „ Entropy coding „ Unfortunately, not quite JPEG

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 5 MPEG-1: coding of P-pictures

„ Motion-compensated prediction from an encoded I-picture or P-picture (DPCM) „ Half-pel accuracy of motion compensation, bilinear interpolation „ One displacement vector per macroblock „ Differential coding of displacement vectors „ Coding of prediction error with 8x8-DCT, uniform threshold quantization, zig-zag-scan as in I-pictures

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 6 MPEG-1: coding of B-pictures

„ Motion-compensated prediction from two consecutive P- or I-pictures z either • only forward prediction (1 vector/macroblock) z or • only backward prediction (1 vector/macroblock) z or • Average of forward and backward prediction = interpolation (2 vectors/macroblock) „ Half-pel accuracy of motion compensation, bilinear interpolation „ Coding of prediction error with 8x8-DCT, uniform quantization, zig-zag-scan as in I-pictures

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 7 MPEG-2 vs. MPEG-1

„ Efficiently compress interlaced digital video at broadcast quality z Field/frame pictures z Chroma sampling z New prediction modes z Field/frame DCT z Additional scan patterns for DCT coefficients z Motion compensation with blocks of size 16x8 pels „ Improved coding efficiency by different quantization, VLC tables „ Various scalability modes

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 8 Coding of Interlaced Video (1)

„ Frame and field picture structures

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 9 Coding of Interlaced Video (2)

„ Field prediction for field pictures

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 10 Coding of Interlaced Video (3)

„ Field prediction for frame pictures

16 16

8

16

8

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 11 Coding of Interlaced Video (4)

„ Dual prime for P pictures

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 12 Coding of Interlaced Video (5)

„ Field/frame DCT „ Alternate Scan

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 13 MPEG-4

„ Support highly interactive multimedia applications as well as traditional applications „ Advanced functionalities: interactivity, scalability, error resilience… „ Coding of natural and synthetic audio and video, as well as graphics „ Enable the multiplexing of audiovisual objects and composition in a scene

¾ Video on LANs, Internet video ‘TV/film’ ¾ Wireless video AV-data ¾ Video database ¾ Interactive home shopping ¾ Video e-mail, home movies ‘Computer’ ‘Telecom’ Wireless ¾ Virtual reality games, flight Interactivity simulation, multi-viewpoint training

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 14 MPEG-4: Scene with audiovisual objects

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 15 MPEG-4: Video coding

„ Basic video coding z Definition of Video Object (VO), Video Object Layer (VOL), Video Object Plane (VOP) z Improved coding efficiency vs. MPEG-1/2 • Based on H.263 baseline • Global motion compensation • Sprites • Quarter pixel motion compensation

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 16 MPEG-4: Video coding

„ Object-based video coding z Binary shape coding z α-map shape coding z Padding for block-based DCT of texture z Shape-adaptive DCT „ DWT for still texture coding „ Mesh animation, face and body animation

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 17 Shape Adaptive DCT

Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 18