Course Presentation Multimedia Systems Video III (Video Coding Standards) Mahdi Amiri December 2015 Sharif University of Technology Video Coding Standards Motivation Motivation Page 1 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards Standardization Organizations Two organizations have dominated video compression standardization. ITU-T Video Coding Experts Group ( VCEG ) International Telecommunications Union –Telecommunications Standardization Sector (ITU-T, a United Nations Organization, formerly CCITT), Study Group 16, Question 6. ISO/IEC Moving Picture Experts Group ( MPEG ) International Standardization Organization and International Electrotechnical Commission, Joint Technical Committee Number 1, Subcommittee 29, Working Group 11. Page 2 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards Dynamics VCEG is older and more focused on conventional (esp. low-delay) video coding goals (e.g. good compression and packet-loss/error resilience) MPEG is larger and takes on more ambitious goals (e.g. “object oriented video”, “synthetic-natural hybrid coding”, and digital cinema) Sometimes the major organizations team up (e.g. ISO, IEC and ITU teamed up for both MPEG-2 and JPEG) Relatively little industry consortium activity (DV and organizations that tweak the video coding standards in minor ways, such as DVD, 3GPP, 3GPP2, SMPTE, IETF, etc.) Growing activity for internet streaming media outside of formal standardization (e.g., Microsoft, Real Networks, Quicktime) Page 3 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards The Scope of Picture and Video Coding Standardization A Video standard specifically do not define an encoder; rather, they define the output that an encoder should produce. A decoding method is defined in each standard (only the Bitstream Syntax and Decoding Process are standardized): e.g. use IDCT, but not how to implement the IDCT. Permits optimization beyond the obvious. Permits complexity reduction for implementability. Provides no guarantees of Quality - only interoperability. Ensuring interoperability : Enabling communication between devices made by different manufacturers Page 4 Multimedia Systems, Mahdi Amiri, Video III Video Quality Evaluation Objective A computer algorithm judges the distortion between videos. Attempts to model a human observer. There is currently no standard method. Page 5 Multimedia Systems, Mahdi Amiri, Video III Video Quality Evaluation Objective Metrics: PSNR Peak Signal-To-Noise Ratio (PSNR). Used widely in evaluating coding performance. Purely mathematical difference. Can be tricked quite easily. Root Mean Squared Error (RMSE) 255 = 2^n – 1 n: the number of bits per image sample Page 6 Multimedia Systems, Mahdi Amiri, Video III Video Quality Evaluation PSNR, Example Original PSNR 35.4 [dB] PSNR 29.0 [dB] Page 7 Multimedia Systems, Mahdi Amiri, Video III ABC AB Original PSNR 45.53 [dB] PSNR 36.81 [dB] PSNR 31.45 [dB] Page 8 Multimedia Systems, Mahdi Amiri, Video III Video Quality Evaluation Mahalanobis Distance The Mahalanobis distance differs from Euclidean Prasanta Chandra Mahalanobis distance in that it takes into account the 1893-1972 Euclid, Floruit 300 BC (Statue of Euclid in the Oxford University correlations of the data set and is scale-invariant. Museum of Natural History) T Ref.: www.aiaccess.net T −1 ()()x −μ x − μ ()()x −μ Σ xμ − Euclidian distance (Squared) Mahalanobis distance (Squared) Page 9 Multimedia Systems, Mahdi Amiri, Video III Video Quality Evaluation Objective Metrics: PSNR How to trick PSNR Take a natural image Give more bits to areas you look at more Give less bits to areas you look at less Subjective rating will be high, PSNR low Original Attention Map Example Test (High subjective rating, low PSNR) Page 10 Multimedia Systems, Mahdi Amiri, Video III Video Quality Evaluation Subjective: MOS Mean Opinion Score (MOS) A human “subject” rates the video on a scale. A numerical indication of the perceived quality of the media received after being transmitted and eventually compressed using codecs. MOS is expressed in one number, from 1 to 5, 1 being the worst and 5 the best. Page 11 Multimedia Systems, Mahdi Amiri, Video III Video Quality Evaluation Subjective In the ITU recommendations, there are many subjective quality test methods. Absolute Category Rating ( ACR ) Degradation Category Rating ( DCR ) The Double-Stimulus Continuous Quality-Scale method ( DSCQS ) Page 12 Multimedia Systems, Mahdi Amiri, Video III Video Quality Evaluation Subjective: ACR and DCR Absolute Category Rating ( ACR ) No reference sequence. Subjects are asked to rate the quality of the presentation based on the level of the quality they have in their opinion for it after viewing or listening it (Single Stimulus). Degradation Category Rating ( DCR ) Known reference sequence. Test sequences are presented in pairs. The first stimulus presented in each pair is always the source reference without any impairments (Double Stimulus). Ref.: www.irisa.fr/armor/lesmembres/Mohamed /Thesis/node147.html Page 13 Multimedia Systems, Mahdi Amiri, Video III Double Stimulus Continuous Quality Scale Method (DSCQS) Video Quality Evaluation Subjective: DSCQS Unknown reference sequence. For having fidelity test the observers are not told which is the reference sequence. Ref.: www.irisa.fr/armor/lesmembres/Mohamed /Thesis/node147.html Page 14 Multimedia Systems, Mahdi Amiri, Video III R-D Curve of Video Codecs ABC Page 15 Multimedia Systems, Mahdi Amiri, Video III Reminder CIF-size image R-D Curve of Video Codecs 352 ×288 ABC Page 16 Multimedia Systems, Mahdi Amiri, Video III R-D Curve of Video Codecs ABC Page 17 Multimedia Systems, Mahdi Amiri, Video III R-D Curve of Video Codecs R-D PerformanceABC of MPEG Codecs 50 48 46 44 42 PSNR(Y) 40 38 36 34 32 350 450 550 650 750 850 950 1050 Bit rate (kbps) MPEG-1 MPEG-2 MPEG-4 H.264 Page 18 Multimedia Systems, Mahdi Amiri, Video III R-D Curve of Video Codecs ABC Page 19 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards Gary J. Sullivan, Ph.D. ITU-T VCEG Rapporteur/Chairman ISO/IEC MPEG Video Rapporteur/Co-Chairman ITU/ISO/IEC JVT Rapporteur/Co-Chairman VIDEO CODECS STANDARDIZATION HISTORY Page 20 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards History The Society of Motion Picture and Television Engineers, SMPTE (pron. simpti) is an internationally en.wikipedia.org/wiki/Data_compression recognized standards organizations founded in 1916 (en.wikipedia.org/wiki/SMPTE). Page 21 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards Major Video Compression Applications Page 22 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards Moving Picture Experts Group (MPEG) A working group of ISO/IEC in charge of the development of standards for coded representation of digital audio and video and related data. Established in 1988 26 years of activity The number of independent standards: more than 125 Ref.: en.wikipedia.org/wiki/Moving_Picture_Experts_Group Page 23 Multimedia Systems, Mahdi Amiri, Video III MPEG-1 The standard on which such products as Video CD and MP3 are based MPEG-2 The standard on which such products as Digital Television set top boxes and DVD are Videobased; Coding Standards MPEG-4 The standard for multimedia for the fixed and mobile web; MPEG-7 The standard for description and search of audio and visual content; MPEG-21 The Multimedia Framework; MPEG-A The standard providing application-specific formats by integrating multiple MPEG technologies; MPEG-B A collection of Systems specific standards MPEG-C A collection of Video specific standards MPEG-D A collection of Audio specific standards MPEG-E A standard (M3W) providing support to download and execution of multimedia applications MPEG-H A standard (HEVC) providing a significantly increased video compression performance MPEG-M A standard (MXM) for packaging and reusability of MPEG technologies MPEG-U A standard for rich-media user interface MPEG-V A standard for interchange with virtual worlds Page 24 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards Video Coding Experts Group (VCEG) Part of study group 16 (Multimedia coding, systems and applications) of the ITU-T. Established in 1984 H.120 The first digital video coding standard H.261 Was the first practical digital video coding standard. H.262 It is identical in content to the video part of the ISO/IEC MPEG-2 standard. H.263 Provided a suitable replacement for H.261 at all bitrates. H.263v2 Also known as H.263+, Enhanced robustness against data loss in the transmission channel. H.264 The ITU-T H.264 standard and the ISO/IEC MPEG-4 Part 10 standard (formally, ISO/IEC 14496- 10) are technically identical. H.265 Not yet developed; expected 2012 or later. H.271 Video back channel messages for conveyance of status information and requests from a video receiver to a video sender. Page 25 Multimedia Systems, Mahdi Amiri, Video III Video Coding Standards H.120 The First Digital Video Coding Standard ITU-T (ex-CCITT) Rec. H.120: 1984 v1 (1984) had conditional replenishment, DPCM, scalar quantization , variable-length coding, switch for quincunx sampling v2 (1988) added motion compensation and background prediction Operated at 1.544 (NTSC) and 2.048 (PAL) Mbits/s Few units made, essentially not in use today Conditional Replenishment : Can signal to leave a block area of the image unchanged, or replace it with new data (using a threshold value). Quincunx sampling : In a digital video system, a sampling structure with an array of samples where