Evaluation of Intra-Coding Based Image Compression

Evaluation of intra-coding based image compression

Steve Göring, Alexander Raake

Audiovisual Technology Group, Technische Universität Ilmenau, Germany; Email: [steve.goering, alexander.raake]@tu-ilmenau.de

code & demo: https://git.io/Je0ip

October 30, 2019 Motivation

1 I increase of uploaded/shared images , e.g. ﬂickr, instagram, . . .

I higher resolutions, more content of diﬀerent quality → image compression review

1for Flickr: average 1.68 million photos per day for 2016, see https://www.flickr.com/photos/franckmichel/6855169886/ 1 / 14 Image Compression – status

I popular/new lossy image codecs: ◦ JPEG, PNG, GIF, JPEG-2K, JPEG-XR ◦ video codec based: BPG2, HEIF [4]3, WebP4, AVIF5

I most evaluation, i.e. [5, 1, 2, 4, 3] ◦ small dataset (<100 images), small resolution (<1000p), mostly PSNR, SSIM

I intra-frame compression-quality vs. JPEG in case of high resolution images

→ large scale evaluation

2https://bellard.org/bpg/ 3hhttps://nokiatech.github.io/heif/ 4https://developers.google.com/speed/webp/ 5https://aomediacodec.github.io/av1-avif/ 2 / 14 Image Compression – status

I popular/new lossy image codecs: ◦ JPEG, PNG, GIF, JPEG-2K, JPEG-XR ◦ video codec based: BPG2, HEIF [4]3, WebP4, AVIF5

I most evaluation, i.e. [5, 1, 2, 4, 3] ◦ small dataset (<100 images), small resolution (<1000p), mostly PSNR, SSIM

I intra-frame compression-quality vs. JPEG in case of high resolution images

→ large scale evaluation

2https://bellard.org/bpg/ 3hhttps://nokiatech.github.io/heif/ 4https://developers.google.com/speed/webp/ 5https://aomediacodec.github.io/av1-avif/ 2 / 14 Image Compression – status

I popular/new lossy image codecs: ◦ JPEG, PNG, GIF, JPEG-2K, JPEG-XR ◦ video codec based: BPG2, HEIF [4]3, WebP4, AVIF5

I most evaluation, i.e. [5, 1, 2, 4, 3] ◦ small dataset (<100 images), small resolution (<1000p), mostly PSNR, SSIM

I intra-frame compression-quality vs. JPEG in case of high resolution images

→ large scale evaluation

2https://bellard.org/bpg/ 3hhttps://nokiatech.github.io/heif/ 4https://developers.google.com/speed/webp/ 5https://aomediacodec.github.io/av1-avif/ 2 / 14 Image Compression – status

I popular/new lossy image codecs: ◦ JPEG, PNG, GIF, JPEG-2K, JPEG-XR ◦ video codec based: BPG2, HEIF [4]3, WebP4, AVIF5

I most evaluation, i.e. [5, 1, 2, 4, 3] ◦ small dataset (<100 images), small resolution (<1000p), mostly PSNR, SSIM

I intra-frame compression-quality vs. JPEG in case of high resolution images

→ large scale evaluation

2https://bellard.org/bpg/ 3hhttps://nokiatech.github.io/heif/ 4https://developers.google.com/speed/webp/ 5https://aomediacodec.github.io/av1-avif/ 2 / 14 Image Compression – status

I popular/new lossy image codecs: ◦ JPEG, PNG, GIF, JPEG-2K, JPEG-XR ◦ video codec based: BPG2, HEIF [4]3, WebP4, AVIF5

I most evaluation, i.e. [5, 1, 2, 4, 3] ◦ small dataset (<100 images), small resolution (<1000p), mostly PSNR, SSIM

I intra-frame compression-quality vs. JPEG in case of high resolution images

→ large scale evaluation

2https://bellard.org/bpg/ 3hhttps://nokiatech.github.io/heif/ 4https://developers.google.com/speed/webp/ 5https://aomediacodec.github.io/av1-avif/ 2 / 14 Image Compression – status

I popular/new lossy image codecs: ◦ JPEG, PNG, GIF, JPEG-2K, JPEG-XR ◦ video codec based: BPG2, HEIF [4]3, WebP4, AVIF5

I most evaluation, i.e. [5, 1, 2, 4, 3] ◦ small dataset (<100 images), small resolution (<1000p), mostly PSNR, SSIM

I intra-frame compression-quality vs. JPEG in case of high resolution images

→ large scale evaluation

2https://bellard.org/bpg/ 3hhttps://nokiatech.github.io/heif/ 4https://developers.google.com/speed/webp/ 5https://aomediacodec.github.io/av1-avif/ 2 / 14 Image Compression – status

I popular/new lossy image codecs: ◦ JPEG, PNG, GIF, JPEG-2K, JPEG-XR ◦ video codec based: BPG2, HEIF [4]3, WebP4, AVIF5

I most evaluation, i.e. [5, 1, 2, 4, 3] ◦ small dataset (<100 images), small resolution (<1000p), mostly PSNR, SSIM

I intra-frame compression-quality vs. JPEG in case of high resolution images

→ large scale evaluation

2https://bellard.org/bpg/ 3hhttps://nokiatech.github.io/heif/ 4https://developers.google.com/speed/webp/ 5https://aomediacodec.github.io/av1-avif/ 2 / 14 Image Compression – status

I popular/new lossy image codecs: ◦ JPEG, PNG, GIF, JPEG-2K, JPEG-XR ◦ video codec based: BPG2, HEIF [4]3, WebP4, AVIF5

I most evaluation, i.e. [5, 1, 2, 4, 3] ◦ small dataset (<100 images), small resolution (<1000p), mostly PSNR, SSIM

I intra-frame compression-quality vs. JPEG in case of high resolution images

→ large scale evaluation

2https://bellard.org/bpg/ 3hhttps://nokiatech.github.io/heif/ 4https://developers.google.com/speed/webp/ 5https://aomediacodec.github.io/av1-avif/ 2 / 14 Image Compression – status

I popular/new lossy image codecs: ◦ JPEG, PNG, GIF, JPEG-2K, JPEG-XR ◦ video codec based: BPG2, HEIF [4]3, WebP4, AVIF5

I most evaluation, i.e. [5, 1, 2, 4, 3] ◦ small dataset (<100 images), small resolution (<1000p), mostly PSNR, SSIM

I intra-frame compression-quality vs. JPEG in case of high resolution images

→ large scale evaluation

2https://bellard.org/bpg/ 3hhttps://nokiatech.github.io/heif/ 4https://developers.google.com/speed/webp/ 5https://aomediacodec.github.io/av1-avif/ 2 / 14 Our Approach

raw png encode for all encoded image quality metrics images av1 (crf=0..63) vmaf, ssim, psnr, vif vp9 (crf=0..63)

H.265 (crf=0..51)

H.264 (crf=0..51) analysis JPEG (q=1..100)

I raw images: wesaturate.com; all raw images of 5 year 2018 I remove duplicates, unify to PNG: 1133 images I encode: AV1, VP9, H.264, H.265; JPEG: ◦ all possible settings CRF settings: ≈ 380k encoded imgs ◦ one pass, preset: veryslow (H.26X); cpu-count=1 (VP9/AV1)

◦ uniﬁed quality level: ql = 1 − crf /ncodec or ql = (JPEGq − 1)/99

I quality metrics: VMAF, SSIM, PSNR, VIF

3 / 14 Our Approach

raw png encode for all encoded image quality metrics images av1 (crf=0..63) vmaf, ssim, psnr, vif vp9 (crf=0..63)

H.265 (crf=0..51)

H.264 (crf=0..51) analysis JPEG (q=1..100)

◦ uniﬁed quality level: ql = 1 − crf /ncodec or ql = (JPEGq − 1)/99

I quality metrics: VMAF, SSIM, PSNR, VIF

3 / 14 Our Approach

raw png encode for all encoded image quality metrics images av1 (crf=0..63) vmaf, ssim, psnr, vif vp9 (crf=0..63)

H.265 (crf=0..51)

H.264 (crf=0..51) analysis JPEG (q=1..100)

◦ uniﬁed quality level: ql = 1 − crf /ncodec or ql = (JPEGq − 1)/99

I quality metrics: VMAF, SSIM, PSNR, VIF

3 / 14 Our Approach

raw png encode for all encoded image quality metrics images av1 (crf=0..63) vmaf, ssim, psnr, vif vp9 (crf=0..63)

H.265 (crf=0..51)

H.264 (crf=0..51) analysis JPEG (q=1..100)

◦ uniﬁed quality level: ql = 1 − crf /ncodec or ql = (JPEGq − 1)/99

I quality metrics: VMAF, SSIM, PSNR, VIF

3 / 14 Our Approach

raw png encode for all encoded image quality metrics images av1 (crf=0..63) vmaf, ssim, psnr, vif vp9 (crf=0..63)

H.265 (crf=0..51)

H.264 (crf=0..51) analysis JPEG (q=1..100)

◦ uniﬁed quality level: ql = 1 − crf /ncodec or ql = (JPEGq − 1)/99

I quality metrics: VMAF, SSIM, PSNR, VIF

3 / 14 Our Approach

raw png encode for all encoded image quality metrics images av1 (crf=0..63) vmaf, ssim, psnr, vif vp9 (crf=0..63)

H.265 (crf=0..51)

H.264 (crf=0..51) analysis JPEG (q=1..100)

◦ uniﬁed quality level: ql = 1 − crf /ncodec or ql = (JPEGq − 1)/99

I quality metrics: VMAF, SSIM, PSNR, VIF

3 / 14 Our Approach

raw png encode for all encoded image quality metrics images av1 (crf=0..63) vmaf, ssim, psnr, vif vp9 (crf=0..63)

H.265 (crf=0..51)

H.264 (crf=0..51) analysis JPEG (q=1..100)

◦ uniﬁed quality level: ql = 1 − crf /ncodec or ql = (JPEGq − 1)/99

I quality metrics: VMAF, SSIM, PSNR, VIF

3 / 14 Evaluation – Dataset (sample)

I CC0 licenced images; 36GB; download: https://zenodo.org/record/3459357#.XbdXVd-YWvZ I mean height/width 3980 to 4375 pixel 4 / 14 Evaluation – Visual Comparison (1)

I left: av1: crf=63, right: jpeg quality=1; ql = 0

5 / 14 Evaluation – Visual Comparison (2)

(a) Source (b) VP9 (c) AV1 (d) H.264 (e) H.265 (f) JPEG (83 MB) (15 KB) (11 KB) (19 KB) (17 KB) (110 KB) PSNR=45.36 PSNR=45.87 PSNR=39.74 PSNR=42.72 PSNR=32.26

I 360p center crop with ql = 0

6 / 14 Evaluation – Quality-level vs. Quality (VMAF)

100

80 I AV1 ≈ VP9,

vmaf codec I H.265 > H.264 vp9 JPEG < AV1,VP9 70 h264 I jpg av1 60 h265

0.0 0.2 0.4 0.6 0.8 1.0 quality_level 7 / 14 Evaluation – Quality-level vs. Quality (SSIM)

1.00

0.98

0.96 I AV1 ≈ VP9 > H.265,

ssim 0.94 codec I H.265 > H.264 > vp9 JPEG h264 0.92 jpg av1 0.90 h265

0.0 0.2 0.4 0.6 0.8 1.0 quality_level 8 / 14 Evaluation – Quality-level vs. Quality (PSNR)

60 codec vp9 55 h264 jpg 50 av1 h265 45 psnr I JPEG worst 40

0.0 0.2 0.4 0.6 0.8 1.0

quality_level 9 / 14 Evaluation – Quality-level vs. Compression

100

I cr = FS(I)/FS(R), 90 I I lossy compressed,

vmaf codec I R lossless vp9 85 I FS: ﬁlesize h264 AV1 JPEG jpg I = 80 av1 h265

0.0 0.2 0.4 0.6 0.8 1.0 ratio = compressed / uncompressed 10 / 14 Conclusion, Summary and Future Work

I evaluated diﬀerent intra-coding based image compression methods

◦ quality & compression: AV1|VP9 > H.265 > H.264 > JPEG

I large raw image dataset

◦ 1133 images; high resolution; user content

I open and next steps:

◦ evaluate image resolution as parameter

◦ include other image codecs; subjective test

11 / 14 Conclusion, Summary and Future Work

I evaluated diﬀerent intra-coding based image compression methods

◦ quality & compression: AV1|VP9 > H.265 > H.264 > JPEG

I large raw image dataset

◦ 1133 images; high resolution; user content

I open and next steps:

◦ evaluate image resolution as parameter

◦ include other image codecs; subjective test

11 / 14 Conclusion, Summary and Future Work

I evaluated diﬀerent intra-coding based image compression methods

◦ quality & compression: AV1|VP9 > H.265 > H.264 > JPEG

I large raw image dataset

◦ 1133 images; high resolution; user content

I open and next steps:

◦ evaluate image resolution as parameter

◦ include other image codecs; subjective test

11 / 14 Conclusion, Summary and Future Work

I evaluated diﬀerent intra-coding based image compression methods

◦ quality & compression: AV1|VP9 > H.265 > H.264 > JPEG

I large raw image dataset

◦ 1133 images; high resolution; user content

I open and next steps:

◦ evaluate image resolution as parameter

◦ include other image codecs; subjective test

11 / 14 Conclusion, Summary and Future Work

I evaluated diﬀerent intra-coding based image compression methods

◦ quality & compression: AV1|VP9 > H.265 > H.264 > JPEG

I large raw image dataset

◦ 1133 images; high resolution; user content

I open and next steps:

◦ evaluate image resolution as parameter

◦ include other image codecs; subjective test

11 / 14 Conclusion, Summary and Future Work

I evaluated diﬀerent intra-coding based image compression methods

◦ quality & compression: AV1|VP9 > H.265 > H.264 > JPEG

I large raw image dataset

◦ 1133 images; high resolution; user content

I open and next steps:

◦ evaluate image resolution as parameter

◦ include other image codecs; subjective test

11 / 14 Conclusion, Summary and Future Work

I evaluated diﬀerent intra-coding based image compression methods

◦ quality & compression: AV1|VP9 > H.265 > H.264 > JPEG

I large raw image dataset

◦ 1133 images; high resolution; user content

I open and next steps:

◦ evaluate image resolution as parameter

◦ include other image codecs; subjective test

11 / 14 Thank you for your attention

...... are there any questions?

12 / 14 ReferencesI

[1] Umar Albalawi, Saraju P Mohanty, and Elias Kougianos. “A hardware architecture for better portable graphics (BPG) compression encoder”. In: 2015 IEEE International Symposium on Nanoelectronic and Information Systems. IEEE. 2015, pp. 291–296. [2] Abhilash Antony and G Sreelekha. “HEVC-based lossless intra coding for efficient still image compression”. In: Multimedia Tools and Applications 76.2 (2017), pp. 1639–1658. [3] Nathan E Egge et al. “Using Daala intra frames for still picture coding”. In: Proceedings of Picture Coding Symposium. 2015. [4] Jani Lainema et al. “HEVC still image coding and high efficiency image file format”. In: 2016 IEEE International Conference on Image Processing (ICIP). IEEE. 2016, pp. 71–75.

13 / 14 ReferencesII [5] Maurizio Pintus et al. “Objective evaluation of webp image compression eﬃciency”. In: International Conference on Mobile Multimedia Communications. Springer. 2011, pp. 252–265.

14 / 14