Course Presentation

Multimedia Systems III (Video Coding Standards)

Mahdi Amiri

December 2015 Sharif University of Technology Video Coding Standards Motivation Motivation

Page 1 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards Standardization Organizations Two organizations have dominated video compression standardization. ITU-T Video Coding Experts Group ( VCEG ) International Union –Telecommunications Standardization Sector (ITU-T, a United Nations Organization, formerly CCITT), Study Group 16, Question 6. ISO/IEC Moving Picture Experts Group ( MPEG ) International Standardization Organization and International Electrotechnical Commission, Joint Technical Committee Number 1, Subcommittee 29, Working Group 11.

Page 2 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards Dynamics VCEG is older and more focused on conventional (esp. low-delay) video coding goals (e.g. good compression and packet-loss/error resilience) MPEG is larger and takes on more ambitious goals (e.g. “object oriented video”, “synthetic-natural hybrid coding”, and ) Sometimes the major organizations team up (e.g. ISO, IEC and ITU teamed up for both MPEG-2 and JPEG) Relatively little industry consortium activity (DV and organizations that tweak the video coding standards in minor ways, such as DVD, 3GPP, 3GPP2, SMPTE, IETF, etc.) Growing activity for outside of formal standardization (e.g., , Real Networks, Quicktime)

Page 3 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards The Scope of Picture and Video Coding Standardization A Video standard specifically do not define an encoder; rather, they define the output that an encoder should produce. A decoding method is defined in each standard (only the Bitstream Syntax and Decoding Process are standardized): e.g. use IDCT, but not how to implement the IDCT. Permits optimization beyond the obvious. Permits complexity reduction for implementability. Provides no guarantees of Quality - only interoperability.

Ensuring interoperability : Enabling communication between devices made by different manufacturers

Page 4 Multimedia Systems, Mahdi Amiri, Video III Video Quality Evaluation Objective A computer judges the distortion between . Attempts to model a human observer. There is currently no standard method.

Page 5 Multimedia Systems, Mahdi Amiri, Video III Video Quality Evaluation Objective Metrics: PSNR Peak Signal-To-Noise Ratio (PSNR). Used widely in evaluating coding performance. Purely mathematical difference. Can be tricked quite easily.

Root Mean Squared Error (RMSE)

255 = 2^n – 1 n: the number of bits per image sample

Page 6 Multimedia Systems, Mahdi Amiri, Video III Video Quality Evaluation PSNR, Example

Original PSNR 35.4 [dB] PSNR 29.0 [dB]

Page 7 Multimedia Systems, Mahdi Amiri, Video III ABC AB

Original PSNR 45.53 [dB]

PSNR 36.81 [dB] PSNR 31.45 [dB] Page 8 Multimedia Systems, Mahdi Amiri, Video III Video Quality Evaluation Mahalanobis Distance The Mahalanobis distance differs from Euclidean Prasanta Chandra Mahalanobis distance in that it takes into account the 1893-1972 Euclid, Floruit 300 BC (Statue of Euclid in the Oxford University correlations of the data set and is scale-invariant. Museum of Natural History)

T Ref.: www.aiaccess.net T −1 ()()x −μ x − μ ()()x −μ Σ xμ − Euclidian distance (Squared) Mahalanobis distance (Squared)

Page 9 Multimedia Systems, Mahdi Amiri, Video III Video Quality Evaluation Objective Metrics: PSNR How to trick PSNR Take a natural image Give more bits to areas you look at more Give less bits to areas you look at less Subjective rating will be high, PSNR low

Original Attention Map Example Test (High subjective rating, low PSNR)

Page 10 Multimedia Systems, Mahdi Amiri, Video III Video Quality Evaluation Subjective: MOS Mean Opinion Score (MOS) A human “subject” rates the video on a scale. A numerical indication of the perceived quality of the media received after being transmitted and eventually compressed using . MOS is expressed in one number, from 1 to 5, 1 being the worst and 5 the best.

Page 11 Multimedia Systems, Mahdi Amiri, Video III Video Quality Evaluation Subjective In the ITU recommendations, there are many subjective quality test methods. Absolute Category Rating ( ACR ) Degradation Category Rating ( DCR ) The Double-Stimulus Continuous Quality-Scale method ( DSCQS )

Page 12 Multimedia Systems, Mahdi Amiri, Video III Video Quality Evaluation Subjective: ACR and DCR Absolute Category Rating ( ACR ) No reference sequence. Subjects are asked to rate the quality of the presentation based on the level of the quality they have in their opinion for it after viewing or listening it (Single Stimulus). Degradation Category Rating ( DCR ) Known reference sequence. Test sequences are presented in pairs. The first stimulus presented in each pair is always the source reference without any impairments (Double Stimulus).

Ref.: www.irisa.fr/armor/lesmembres/Mohamed /Thesis/node147.html

Page 13 Multimedia Systems, Mahdi Amiri, Video III Double Stimulus Continuous Quality Scale Method (DSCQS) Video Quality Evaluation Subjective: DSCQS

Unknown reference sequence.

For having fidelity test the observers are not told which is the reference sequence.

Ref.: www.irisa.fr/armor/lesmembres/Mohamed /Thesis/node147.html

Page 14 Multimedia Systems, Mahdi Amiri, Video III R-D Curve of Video Codecs ABC

Page 15 Multimedia Systems, Mahdi Amiri, Video III Reminder CIF-size image R-D Curve of Video Codecs 352 ×288 ABC

Page 16 Multimedia Systems, Mahdi Amiri, Video III R-D Curve of Video Codecs ABC

Page 17 Multimedia Systems, Mahdi Amiri, Video III R-D Curve of Video Codecs

R-D PerformanceABC of MPEG Codecs

50

48

46

44

42

PSNR(Y) 40

38

36

34

32 350 450 550 650 750 850 950 1050 (kbps)

MPEG-1 MPEG-2 MPEG-4 H.264

Page 18 Multimedia Systems, Mahdi Amiri, Video III R-D Curve of Video Codecs ABC

Page 19 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards

Gary J. Sullivan, Ph.D. ITU-T VCEG Rapporteur/Chairman ISO/IEC MPEG Video Rapporteur/Co-Chairman ITU/ISO/IEC JVT Rapporteur/Co-Chairman

VIDEO CODECS STANDARDIZATION HISTORY

Page 20 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards History

The Society of Motion Picture and Engineers, SMPTE (pron. simpti) is an internationally en.wikipedia.org/wiki/Data_compression recognized standards organizations founded in 1916 (en.wikipedia.org/wiki/SMPTE). Page 21 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards Major Video Compression Applications

Page 22 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards Moving Picture Experts Group (MPEG) A working group of ISO/IEC in charge of the development of standards for coded representation of digital audio and video and related data. Established in 1988 26 years of activity The number of independent standards: more than 125

Ref.: en.wikipedia.org/wiki/Moving_Picture_Experts_Group Page 23 Multimedia Systems, Mahdi Amiri, Video III MPEG-1 The standard on which such products as Video CD and MP3 are based MPEG-2 The standard on which such products as set top boxes and DVD are Videobased; Coding Standards MPEG-4 The standard for multimedia for the fixed and ; MPEG-7 The standard for description and search of audio and visual content; MPEG-21 The Multimedia Framework; MPEG-A The standard providing application-specific formats by integrating multiple MPEG technologies; MPEG-B A collection of Systems specific standards MPEG- A collection of Video specific standards MPEG-D A collection of Audio specific standards MPEG-E A standard (M3W) providing support to download and execution of multimedia applications MPEG-H A standard (HEVC) providing a significantly increased video compression performance MPEG-M A standard (MXM) for packaging and reusability of MPEG technologies MPEG-U A standard for rich-media user interface MPEG-V A standard for interchange with virtual worlds

Page 24 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards Video Coding Experts Group (VCEG) Part of study group 16 (Multimedia coding, systems and applications) of the ITU-T. Established in 1984

H.120 The first coding standard H.261 Was the first practical digital video coding standard. H.262 It is identical in content to the video part of the ISO/IEC MPEG-2 standard. H.263 Provided a suitable replacement for H.261 at all bitrates. H.263v2 Also known as H.263+, Enhanced robustness against data loss in the transmission channel. H.264 The ITU-T H.264 standard and the ISO/IEC MPEG-4 Part 10 standard (formally, ISO/IEC 14496- 10) are technically identical. H.265 Not yet developed; expected 2012 or later. H.271 Video back channel messages for conveyance of status information and requests from a video receiver to a video .

Page 25 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards H.120 The First Digital Video Coding Standard ITU-T (ex-CCITT) Rec. H.120: 1984 v1 (1984) had conditional replenishment, DPCM, scalar quantization , variable-length coding, switch for quincunx sampling v2 (1988) added and background prediction Operated at 1.544 (NTSC) and 2.048 (PAL) Mbits/s Few units made, essentially not in use today Conditional Replenishment : Can signal to leave a block area of the image unchanged, or replace it with new data (using a threshold value). Quincunx sampling : In a digital video system, a sampling structure with an array of samples where alternate rows of pixel samples are displaced horizontally in the grid by half of the pitch of the pixel samples along the remaining rows.

Page 26 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards H.261 ITU-T, completed in 1990, The first widespread practical success Video and teleconferencing over ISDN (Integrated Services Digital Network) Embodying typical structure that dominates today Combination of DPCM and DCT Motion Compensation p x 64kbps (64-2048 kbps) Still in use, although mostly as a backward-compatibility feature – overtaken by H.263

Page 27 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards MPEG-1 MPEG-1: “Coding of moving pictures and associated audio for digital storage media” (1992) , and Motion Compensation , Differential Coding, DCT, Quantization, Entropy Coding Video on digital storage media (CD-ROM) Target was VHS Quality at 1.5MBits/s (at 352x240 resolution) Basis of Video-CD MP3 (MPEG-1 Layer 3) 16 bits Sampling rate - 32, 44.1, or 48 kHz Bitrate – 32 to 320 kbps De facto - 44.1 kHz sample rate, 192 kbps bitrate

Page 28 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards MPEG-1 Only supports progressive pictures. Adds bi-directional motion prediction to H.261 design. Adds half-pixel motion estimation (See next slide). Slice-structured coding. DC-only “D” pictures. Superior quality to H.261 when operated a higher bit rates ( > 1 Mbps for CIF 352x288 resolution). Now mostly overtaken by MPEG-2.

Page 29 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards MPEG-1, Half-Pixel ME Half-Pixel (coarse-fine) Motion Estimation Algorithm 1) Coarse step: Perform integer motion estimation on blocks; find best integer-pixel MV 2) Fine step: Refine estimate to find best half-pixel MV a) Spatially interpolate the selected region in reference frame b) Compare current block to interpolated reference frame block c) Choose the integer or half-pixel offset that provides best match Typically, bilinear interpolation is used for spatial interpolation

Page 30 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards MPEG-2 MPEG-2: “Generic coding of Moving Pictures and Associated Audio”. and storage. Satellite TV, DVD, Digital TV. Ubiquity in hardware implies that it will be here for a long time. Transition to HDTV has taken over 10 years and is not finished yet Different profiles and levels allow for quality control. Bitrates: Typ. 4-9 MBits/s (Not especially useful below 4 Mbps, normal range of use 5-30 Mbps).

Page 31 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards MPEG-2 Support for interlaced scan, various picture sampling formats, user defined quantization . Essentially same as MPEG-1 for progressive- scan pictures. Various forms of scalability (SNR, Spatial, Temporal and hybrid). Base Layer: Basic quality requirement, For SDTV. Enhanced Layer: High quality service, For HDTV.

Page 32 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards MPEG-2 Profiles and Levels Goal: To enable more efficient implementations for different applications (interoperability points) Profile : Represents a sub-set of the encoding techniques (address the problem of code complexity and processing power). Level : Represents adjusting the settings of any chosen technique in each profile (address the problem of , max. resolution, and memory issues on the decoder side).

Page 33 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards Bitrate allocation CBR – Constant BitRate Streaming media uses this. Easier to implement. VBR – Variable BitRate DVD’s use this. Allocate more bits for complex scenes . Usually requires 2-pass coding. This is worth it, because you assume that you encode once, decode many times.

Page 34 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards MPEG Container Format Container format is a file format that can contain data compressed by standard codecs. 2 types for MPEG Program Stream (PS) – Designed for reasonably reliable media, such as disks. Transport Stream (TS) – Designed for lossy links, such as networks or broadcast antennas.

Page 35 Multimedia Systems, Mahdi Amiri, Video III Audio formats supported Video Coding Standards Video Container Formats

Video formats supported

Comparison of video container formats Ref.: en.wikipedia.org/wiki/Comparison_of_video_container_formats Page 36 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards MPEG-3 ? Originally developed for HDTV, but abandoned when MPEG-2 was determined to be sufficient.

Page 37 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards H.263 ITU-T Rec. H.263 (v1: 1995): The next generation of video coding performance, developed by ITU-T – the current premier ITU-T video standard (has overtaken H.261 as dominant videoconferencing ) Video telephony over PSTN (public switched network) Wins by a factor of two at very low rates Version 2 (late 1997 / early 1998) & version 3 (2000) later developed with a large number of new features H.263+ & H.263++ (Extensions to H.263)

Page 38 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards MPEG-4 MPEG-4: “Coding of audio-visual objects”. Started as very low-bitrate project. Contains the H.263 baseline design and adds many creative new extras: Coding of media objects ( Segmented coding of shapes ). Bitrate: variable. Synthetic/Semi-synthetic objects. XMT: Like HTML, but to build videos. First standard with Intellectual Property Management.

Page 39 Multimedia Systems, Mahdi Amiri, Video III Part Number Title Description

Describes synchronization and of video and audio. For example Part 1 ISO/IEC 14496-1 Systems Video CodingTransport Standards stream.

A compression codec for visual data (video, still textures, synthetic images, etc.). Part 2 ISO/IEC 14496-2 Visual MPEG-4One of the many "profiles" in Part 2 is the Advanced Simple Profile (ASP). A set of compression codecs for perceptual coding of audio signals, including some Part 3 ISO/IEC 14496-3 Audio variations of (AAC) as well as other audio/speech coding tools.

Part 4 ISO/IEC 14496-4 Conformance Describes procedures for testing conformance to other parts of the standard.

Part 5 ISO/IEC 14496-5 Reference Software Provides software for demonstrating and clarifying the other parts of the standard.

Delivery Multimedia Part 6 ISO/IEC 14496-6 Integration Framework (DMIF). Optimized Reference Provides examples of how to make improved implementations (e.g., in relation to Part 7 ISO/IEC 14496-7 Software Part 5).

Carriage on IP Part 8 ISO/IEC 14496-8 Specifies a method to carry MPEG-4 content on IP networks. networks

Provides hardware designs for demonstrating how to implement the other parts of Part 9 ISO/IEC 14496-9 Reference Hardware the standard.

ISO/IEC 14496- Advanced Video Part 10 A codec for video signals which is technically identical to the ITU-T H.264 standard. 10 Coding (AVC) http://en.wikipedia.org/wiki/MPEG-4

Page 40 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards MPEG-4, Object Based Coding Extension of MPEG-1/2-type to code arbitrarily shaped objects

[MPEG Committee] Basic Idea: Extend Block-DCT and Block-ME/MC- prediction to code arbitrarily shaped objects.

Page 41 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards MPEG-4, Sprite Coding Sprite: Large background image. Hypothesis: Same background exists for many frames, changes resulting from camera motion and occlusions. One possible coding strategy: 1. Code & transmit entire sprite once. 2. Only transmit camera motion parameters for each subsequent frame. Significant coding gain for some scenes.

Page 42 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards MPEG-4, Sprite Coding

[MPEG Committee]

Page 43 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards H.264 or MPEG-4 Part 10 (AVC) MPEG-4 Part 10: / H.264. Designed by a Joint MPEG and VCEG. Claims 50% bitrate savings to MPEG-2, 30% over MPEG-4! Bitrate: 10’s to 100’s kb/s. Variable Block Size, Multiple Reference Frames, Integer Transform, Intra Prediction, In-loop Deblocking Filtering, 1/4-pel Resolution Motion Estimation, ASO (Arbitrary Slice Ordering), FMO (Flexible Ordering). Enhanced entropy coding. CAVLC (Context Adaptive Variable Length Coding) CABAC (Context Adaptive Binary Arithmetic Codes) Increased complexity relative to prior standards.

Page 44 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards, H.264 Integer Transform MPEG-2, MPEG-4

MPEG-4 AVC

Find out more about integer transform by looking for "lifting scheme"

Page 45 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards, H.264 Variable Block Size The fixed block size may not be suitable for all motion objects Improve the flexibility of comparison Reduce the error of comparison 7 types of blocks for selection

16 x 16 16 x 8 8 x 16 8 x 8 0 0 1 0 0 1 1 2 3 8 x 8 8 x 4 4 x 8 4 x 4 0 0 1 0 0 1 1 2 3

Page 46 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards, H.264 Variable Block Size Residual (without MC) showing optimum choice of partitions

Page 47 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards, H.264 Multiple Reference Frames The neighboring frames are not the most similar in some cases. The B-frame can be reference frame. B-frame is close to the target frame in many situations.

Page 48 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards, H.264 Deblocking Filter There are severe blocking artifacts. 4*4 transforms and block-based motion compensation. Result in bit rate savings of around 6~9%. Improve subjective quality and PSNR of the decoded picture.

Without Filter With AVC Deblocking Filter Page 49 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards, H.264 FMO (Flexible Macroblock Ordering) Slice (composed in FMO)  Enhance Robustness to data loss

Subdivision of a picture into Subdivision of a QCIF frame into slices when not using FMO slices when utilizing FMO

Page 50 Multimedia Systems, Mahdi Amiri, Video III H.264, Profiles ABC ABC http://en.wikipedia.org/wiki/MPEG-4_AVC.htm

Page 51 Multimedia Systems, Mahdi Amiri, Video III H.264, Profiles ABC ABC

Page 52 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards, H.265

H.265 or HEVC MP: Main Profile High Efficiency Video Coding (HEVC) / H.265. HP: High Profiel Under joint development by MPEG and VCEG. ASP: Advanced Simple Profile HLP: High Latency Profile Claims 50% bitrate savings to H.264. Two of the key features where HEVC was improved compared to H.264/MPEG-4 AVC was: Support for higher resolution video. Improved parallel processing methods. en.wikipedia.org/wiki/High_Efficiency_Video_Coding

Page 53 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards, H.265 H.265 or HEVC

Typical HEVC video encoder (with decoder modeling elements shaded in light gray). Ref.: G. J. Sullivan, J.-R. Ohm, W.-J. Han, and T. Wiegand, "Overview of the High Efficiency Video Coding (HEVC) Standard", IEEE Trans. Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1649-1668, Dec. 2012. Page 54 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards, H.265 H.265 or HEVC

Page 55 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards, H.265 H.265 or HEVC

(Input MB can be coded using intra-picture prediction: Prediction derived from spatially adjacent MBs).

Page 56 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards, H.265 H.265 or HEVC

Page 57 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards, H.265 H.265 Parallel Processing

H.265/HEVC PARALLELIZATION CONSIDERATIONS NOTABLE CHANGES FROM H.264 WaveFront Parallel Processing: Each row of largest CU blocks can be encoded in parallel, with a two block lag to row above.

Page 58 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards, VP9 VP9 Open and royalty free video coding format developed by Google . Previous name: Next Gen Open Video (NGOV) and VPNext. Extended from VP8 and will be extended to VP10. Aiming for better compression efficiency than h.265. Initial release: Dec. 2013.

Page 59 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards, VP9 VP9 vs. H.264

Page 60 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards, VP9 VP9 vs. H.264

Page 61 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards, VP9 VP9 vs. H.265

VP9 vs. h.265

Page 62 Multimedia Systems, Mahdi Amiri, Video III Scalable Video Coding Motivation Basic situation: 1. Diverse receivers may request the same video Different bandwidths, spatial resolutions, frame rates, computational capabilities 2. Heterogeneous networks and a priori unknown network conditions Wired and links, time-varying bandwidths When you originally code the video you don’t know which client or network situation will exist in the future. Probably have multiple different situations, each requiring a different compressed bitstream. Need a different compressed video matched to each situation. Possible solutions: 1. Compress & store MANY different versions of the same video. 2. Real-time (e.g. decode/re-encode). 3. Scalable coding.

Page 63 Multimedia Systems, Mahdi Amiri, Video III Scalable Video Coding Type of Scalability

The basic types of scalability in video coding

Page 64 Multimedia Systems, Mahdi Amiri, Video III Scalable Video Coding Temporal Scalability Based on the use of B-frames to refine the temporal resolution. B-frames are dependent on other frames. However, no other frame depends on a B-frame. Each B-frame may be discarded without affecting other frames.

Page 65 Multimedia Systems, Mahdi Amiri, Video III Scalable Video Coding Spatial Scalability Based on refining the spatial resolution Base layer is low resolution version of video. Enhanced (Enh1) contains coded difference between upsampled base layer and original video. Also called: Pyramid coding .

Page 66 Multimedia Systems, Mahdi Amiri, Video III Scalable Video Coding Quality Scalability Based on refining the amplitude resolution. Base layer uses a coarse quantizer. Enh1applies a finer quantizer to the difference between the original DCT coefficients and the coarsely quantized base layer coefficients. Also called: SNR Scalability.

Note: Base & enhancement layers are at the same spatial resolution.

Page 67 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards Independent players Efforts from independent companies in video compression. Main competitors: RealNetworks (www.real.com) Microsoft Windows Media (WM) (www.microsoft.com) Apple Quicktime (www.apple.com) Google (VP9) On2 (www.on2.com) (Acquired by Google on Feb. 2010) Mozilla () Cisco (Thor)

Page 68 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards Independent players

Ref.: aomedia.org

Page 69 Multimedia Systems, Mahdi Amiri, Video III MPEG & Multimedia Application MPEG-7 MPEG-7: “Multimedia Content Description Interface” (2001) It uses XML to store metadata, and can Describing audio/video be attached to timecode in order to tag particular events, or synchronize lyrics Applications: to a song, for example . Indexing of video databases. Search & Retrieval. Browsing.

Ref.: en.wikipedia.org/wiki/MPEG-7 Independence between description and content. Page 70 Multimedia Systems, Mahdi Amiri, Video III MPEG & Multimedia Application MPEG-21 Aims at defining an open framework for multimedia applications . MPEG-21 is based on two essential concepts: Definition of a Digital Item (a fundamental unit of distribution and transaction). Users interacting with Digital Items.

Main objective of the MPEG-21: Is to define the technology needed to support users to exchange , access, consume, trade or manipulate Digital Items in an efficient and transparent way. Ref.: en.wikipedia.org/wiki/MPEG-21

Page 71 Multimedia Systems, Mahdi Amiri, Video III MPEG & Multimedia Application MPEG-21 and MPEG-7 Example

Metadata and Identifiers within an MPEG-21 Music Album

Page 72 Multimedia Systems, Mahdi Amiri, Video III MPEG & Multimedia Application MPEG-21, DIDL

Digital Item Declaration Language (DIDL)

Example:

A Personal Photo abstract terms and concepts to form a useful model for defining Digital Items.

Page 73 Multimedia Systems, Mahdi Amiri, Video III Multimedia Systems Video III (Video Coding Standards) More at: en.wikipedia.org/wiki/Comparison_of_video_codecs Thank You

Next Session: Multimedia Networks I

FIND OUT MORE AT... 1. http://ce.sharif.edu/~m_amiri/ 2. http://www.aictc.ir/

Page 74 Multimedia Systems, Mahdi Amiri, Video III