A Comparison of JEM and AV1 with HEVC Thorsten Laude Leibniz Universität Hannover Institut für Informationsverarbeitung Overview Coding Coding Complexity Tools Efficiency Thorsten Laude 2 [email protected] History of Video Codecs ISO/IEC/ITU-T 1980s 1990s 2000s 2010s H.120 MPEG-1 AVC HEVC H.261 MPEG-2/H.262 (MPEG-4 Part 10/ (MPEG-H JEM H.263 H.264) Part 2/ MPEG-4 Part 2 H.265) Contenders 1980s 1990s 2000s 2010s VC-X TrueMotion S/RT/2 Dirac VP9 Real Video VP3-7 Daala AV1 VP8 Thor Real Video Real Video Comparison of the latest video codecs (JEM/AV1) with HEVC Thorsten Laude 3 [email protected] On the Difficulty of Comparing Video Codecs AV1 is up to 43% better than HEVC HEVC is 30% better than AV1 Source: Feldmann, “Multi-Codec DASH Source: Grois et al., “Performance Dataset: An Evaluation of AV1, AVC, HEVC Comparison of AV1, JEM, VP9 and HEVC and VP9”, Bitmovin Blog, 2018 Encoders”, Proceedings of SPIE, 2017 “In terms of PSNR, the average BD-rate savings of AV1 relative to […] x264 high […] are […] 45.8% […]On the other hand, the encoding computational complexity […] was increased by factors of […] 5869.9x” Source: Liu, “AV1 beats x264 and libvpx-vp9 in practical Source: Akyazi and Ebrahimi, “Comparison of use cases”, Facebook Blog, 2018 compression efficiency between HEVC/H.265 and VP9 based on subjective assessments”, QoMEX, 2018 Thorsten Laude 4 [email protected] On the Difficulty of Comparing Video Codecs • Reference implementations Standard vs. Encoder (HM/JEM/aomenc) • Optimized encoders (x264/x265) • Computing resources Codecs perform differently good Codec Codec • Applications: e.g. Sequences for different content Comparison Configurations Broadcasting, VoD, Social Media • Tuning (e.g. PSNR, visual • Luma BD-rates, weighted BD-rates Metrics • Quality metrics: PSNR, SSIM, VMAF • Subjective Tests Thorsten Laude 5 [email protected] Test Conditions for this Comparison Class Sequence • Reference implementations Tango2 Standard vs. A1 Drums100 Encoder (HM/JEM/aomenc) (4K) Campfire ToddlerFountain2 CatRobot A2 TrafficFlow (4K) DaylightRoad2 Rollercoaster2 Kimono B ParkScene • HM/JEM: Common (1080p) Cactus BasketballDrive Codec Codec Test Conditions (CTC) Sequences BQTerrace Comparison Configurations • AV1 BasketballDrill --auto-alt-ref=1 --psnr C BQMall --tune=psnr --i420 -p 1 (WVGA) PartyScene -t 1 --fps=<?> --bit- RaceHorses depth=<?> --input-bit- BasketballPass depth=<?> --cq- D BQSquare level=<?> --kf-min- (WQVGA) BlowingBubbles dist=<?> --kf-max- RaceHorses dist=<?> -w <?> -h <?> E FourPeople (720p) Johnny Metrics • Luma BD-rates KristenAndSara • Quality metrics: PSNR F BasketballDrillText (Screen/ ChinaSpeed Mixed SlideEditing Content) SlideShow Thorsten Laude 6 [email protected] Coding Tools JEM AV1 Partitioning Partitioning • Quaternary and binary splits • Quaternary and binary splits • Bigger block size • Bigger block size Inter coding Inter coding • Overlapped block motion compensation • Overlapped block motion compensation • Higher order motion model • Higher order motion models • Sub-CU MV prediction • Wedge mode partitioning • Compound intra-inter prediction Intra coding Intra Coding • Additional directions • Directional, Paeth, Smooth prediction • Cross-component linear model • Intra block copy • Palette mode Transform coding Transform coding • Adaptive multiple transforms • DCT, DST, Identity • Non-separable secondary transform • Independent horizontal/vertical • Signal-dependent transform transforms Thorsten Laude 7 [email protected] Coding Efficiency 75% 55% 35% 15% rate - Better BD -5% -25% -45% -65% JEM vs. HM JEM vs. AV1 AV1 vs. HM JEM vs. HM JEM vs. AV1 AV1 vs.HM All-intra Random Access Thorsten Laude 8 [email protected] Encoder Runtimes 60 e.g. 10 50 Total CPU time: ≈ 1 decade frames/day 40 30 Better Complexity increase Complexity 20 10 0 JEM AV1 JEM AV1 All-intra Random Access Class A1 Class A2 Class B Class C Class D Class E Class F Overall HM Relative factors to HM, i.e. HM=1 Thorsten Laude 9 [email protected] Decoder Runtimes 14 12 10 8 Better 6 Complexity increase Complexity 4 2 0 JEM AV1 JEM AV1 All-intra Random Access Class A1 Class A2 Class B Class C Class D Class E Class F Overall HM Relative factors to HM, i.e. HM=1 Thorsten Laude 10 [email protected] Runtime-memory Complexity Thorsten Laude 11 [email protected] Trade-off Coding Efficiency vs. Complexity Better Better Thorsten Laude 12 [email protected] Summary Coding Efficiency Runtimes Comparison vs. HM Comparison vs. HM All intra (AI) Encoder JEM: 20% gain JEM: 39×(AI)/10×(RA) slower AV1: 4% gain AV1: 9×(AI)/32×(RA) slower Random Access (RA) Decoder JEM: 28% gain JEM: 3×(AI)/7×(RA) slower AV1: 38% loss AV1: 2×faster (AI)/same (RA) Closing remarks • Results are a snapshot of summer 2017 → AV1 finalization in March 2018 and JVET CfP evaluation in April 2018 • Since last summer, AV1 has gained additional 5% (based on 80 preliminary data points) • Complexity: Reference implementations vs. product implementations Thorsten Laude 13 [email protected] Details: Laude, T., Adhisantoso, Y. G., Voges, J., Munderloh, M., & Ostermann, J. (2018). A Comparison of JEM and AV1 with HEVC: Coding Tools , Coding Efficiency and Complexity. In Picture Coding Symposium (PCS). Dipl.-Ing. Thorsten Laude 14 [email protected].