A Comparison of JEM and AV1 with HEVC
Thorsten Laude
Leibniz Universität Hannover Institut für Informationsverarbeitung Overview
Coding Coding Complexity Tools Efficiency
Thorsten Laude 2 [email protected] History of Video Codecs
ISO/IEC/ITU-T 1980s 1990s 2000s 2010s
H.120 MPEG-1 AVC HEVC H.261 MPEG-2/H.262 (MPEG-4 Part 10/ (MPEG-H JEM H.263 H.264) Part 2/ MPEG-4 Part 2 H.265)
Contenders 1980s 1990s 2000s 2010s
VC-X TrueMotion S/RT/2 Dirac VP9 Real Video VP3-7 Daala AV1 VP8 Thor Real Video Real Video
Comparison of the latest video codecs (JEM/AV1) with HEVC
Thorsten Laude 3 [email protected] On the Difficulty of Comparing Video Codecs
AV1 is up to 43% better than HEVC HEVC is 30% better than AV1
Source: Feldmann, “Multi-Codec DASH Source: Grois et al., “Performance Dataset: An Evaluation of AV1, AVC, HEVC Comparison of AV1, JEM, VP9 and HEVC and VP9”, Bitmovin Blog, 2018 Encoders”, Proceedings of SPIE, 2017
“In terms of PSNR, the average BD-rate savings of AV1 relative to […] x264 high […] are […] 45.8% […]On the other hand, the encoding computational complexity […] was increased by factors of […] 5869.9x”
Source: Liu, “AV1 beats x264 and libvpx-vp9 in practical Source: Akyazi and Ebrahimi, “Comparison of use cases”, Facebook Blog, 2018 compression efficiency between HEVC/H.265 and VP9 based on subjective assessments”, QoMEX, 2018
Thorsten Laude 4 [email protected] On the Difficulty of Comparing Video Codecs
• Reference implementations Standard vs. Encoder (HM/JEM/aomenc) • Optimized encoders (x264/x265)
• Computing resources Codecs perform differently good Codec Codec • Applications: e.g. Sequences for different content Comparison Configurations Broadcasting, VoD, Social Media • Tuning (e.g. PSNR, visual
• Luma BD-rates, weighted BD-rates Metrics • Quality metrics: PSNR, SSIM, VMAF • Subjective Tests
Thorsten Laude 5 [email protected] Test Conditions for this Comparison
Class Sequence • Reference implementations Tango2 Standard vs. A1 Drums100 Encoder (HM/JEM/aomenc) (4K) Campfire ToddlerFountain2 CatRobot A2 TrafficFlow (4K) DaylightRoad2 Rollercoaster2 Kimono B ParkScene • HM/JEM: Common (1080p) Cactus BasketballDrive Codec Codec Test Conditions (CTC) Sequences BQTerrace Comparison Configurations • AV1 BasketballDrill --auto-alt-ref=1 --psnr C BQMall --tune=psnr --i420 -p 1 (WVGA) PartyScene -t 1 --fps= --bit- RaceHorses depth= --input-bit- BasketballPass depth= --cq- D BQSquare level= --kf-min- (WQVGA) BlowingBubbles dist= --kf-max- RaceHorses dist= -w -h E FourPeople (720p) Johnny Metrics • Luma BD-rates KristenAndSara • Quality metrics: PSNR F BasketballDrillText (Screen/ ChinaSpeed Mixed SlideEditing Content) SlideShow
Thorsten Laude 6 [email protected] Coding Tools
JEM AV1
Partitioning Partitioning • Quaternary and binary splits • Quaternary and binary splits • Bigger block size • Bigger block size
Inter coding Inter coding • Overlapped block motion compensation • Overlapped block motion compensation • Higher order motion model • Higher order motion models • Sub-CU MV prediction • Wedge mode partitioning • Compound intra-inter prediction
Intra coding Intra Coding • Additional directions • Directional, Paeth, Smooth prediction • Cross-component linear model • Intra block copy • Palette mode
Transform coding Transform coding • Adaptive multiple transforms • DCT, DST, Identity • Non-separable secondary transform • Independent horizontal/vertical • Signal-dependent transform transforms
Thorsten Laude 7 [email protected] Coding Efficiency
75%
55%
35%
15%
rate
-
Better BD -5%
-25%
-45%
-65% JEM vs. HM JEM vs. AV1 AV1 vs. HM JEM vs. HM JEM vs. AV1 AV1 vs.HM All-intra Random Access
Thorsten Laude 8 [email protected] Encoder Runtimes
60
e.g. 10 50 Total CPU time: ≈ 1 decade frames/day
40
30 Better
Complexity increase Complexity 20
10
0 JEM AV1 JEM AV1 All-intra Random Access Class A1 Class A2 Class B Class C Class D Class E Class F Overall HM
Relative factors to HM, i.e. HM=1
Thorsten Laude 9 [email protected] Decoder Runtimes
14
12
10
8
Better 6 Complexity increase Complexity 4
2
0 JEM AV1 JEM AV1 All-intra Random Access Class A1 Class A2 Class B Class C Class D Class E Class F Overall HM
Relative factors to HM, i.e. HM=1
Thorsten Laude 10 [email protected] Runtime-memory Complexity
Thorsten Laude 11 [email protected]
Trade-off Coding Efficiency vs. Complexity Better
Better
Thorsten Laude 12 [email protected] Summary
Coding Efficiency Runtimes
Comparison vs. HM Comparison vs. HM
All intra (AI) Encoder JEM: 20% gain JEM: 39×(AI)/10×(RA) slower AV1: 4% gain AV1: 9×(AI)/32×(RA) slower Random Access (RA) Decoder JEM: 28% gain JEM: 3×(AI)/7×(RA) slower AV1: 38% loss AV1: 2×faster (AI)/same (RA)
Closing remarks • Results are a snapshot of summer 2017 → AV1 finalization in March 2018 and JVET CfP evaluation in April 2018 • Since last summer, AV1 has gained additional 5% (based on 80 preliminary data points) • Complexity: Reference implementations vs. product implementations
Thorsten Laude 13 [email protected] Details: Laude, T., Adhisantoso, Y. G., Voges, J., Munderloh, M., & Ostermann, J. (2018). A Comparison of JEM and AV1 with HEVC: Coding Tools , Coding Efficiency and Complexity. In Picture Coding Symposium (PCS).
Dipl.-Ing. Thorsten Laude 14 [email protected]