Wavelet Coding & JPEG 2000

Home , Daubechies wavelet, WAV, Wavelet transform

INF5081 Multimedia Coding and Applications Vårsemester 2008, Ifi, UiO

Wolfgang Leister

Contributions by Hans-Jakob Rivertz Svetlana Boudko

Norsk Regnesentral Norwegian Computing Center

JPEG revisited

• JPEG ... – Uses DCT on 8x8 blocks – Visible blocks for high compression rates – Is not good for compression rate of > 40:1 – Low frequencies are not considered – provides competitive performance for the medium bit-rate range

FDCT Quantiser Encoder 01011010..

Table Table Norsk Regnesentral Norwegian Computing Center Wavelet-Coding

• Good quality up to 60:1 • Linear degradation for higher compression rates • No visible blocks • JPEG 2000 Standard - ISO 15444 • JPEG-2000 mostly provides improved performance at for low and high bit-rates

WT Quantiser Encoder 01011010..

Norsk Regnesentral Norwegian Computing Center

Wavelets - Introduction

• Wavelets are used for – compression, – data modelling, – data analysis – signal processing • Wavelets are suitable for representation of digital data in many applications.

Norsk Regnesentral Norwegian Computing Center One-dimensional Case !

Wavelets

Model signals with: • Fourier row development (whole area): f x=a0∑ ai cos us xbi sin usx • DCT-based (8x8 blocks): f  x=a0∑ ai cosus x • Wavelet-based (hierarchical):

f  x=a0∑ ai ui  x • wavelets use piecewise constant base functions • Average of wavelet function is 0

Norsk Regnesentral Norwegian Computing Center

Definition areas and side effects ω FFT

t ω STFT DCT

t ω WT

t Norsk Regnesentral Norwegian Computing Center One-dimensional Case !

Wavelet Transform (1)

a1 Δ a a a /2 2 1 2 Δ 1 ⋮ ⋮ ⋯ ∑ ai Δ N an−1 an−1an/2

9 1 8 7 2 6 6,2,1, 1 3 −1 [ − ] 4 5

Norsk Regnesentral Norwegian Computing Center

One-dimensional Case !

Wavelet Transform (2)

9 1 8 7 2 6 6,2,1, 1 3 −1 [ − ] 4 5

Norsk Regnesentral Norwegian Computing Center One-dimensional Case !

Wavelet Transform (3)

[6, 2, 1, -1]

+1* +2* -1*

Haar Wavelet

Norsk Regnesentral Norwegian Computing Center

One-dimensional Case ! V²-Basis and Haar Wavelet

2 2 2 2 2 2 2 2 ℘x=c00xc1 1 xc22xc3 3 x 2 2 2 2 [c0 ,c1 ,c2 ,c3]=[9,7,3,5]

Norsk Regnesentral Norwegian Computing Center One-dimensional Case !

Another Example

Norsk Regnesentral Norwegian Computing Center

One-dimensional Case !

Different Wavelets

1 -1 1 12 -2 -1

Haar Wavelet Daubechies Wavelet Lazy Wavelet = 9/7 Wavelet = 5/3 Wavelet Average of wavelet function is 0 Norsk Regnesentral Norwegian Computing Center One-dimensional Case !

Signal-Analysis / -Synthesis

H0 s0↓ Y0 s0↑ G0

H1 s1↓ Y1 s1↑ G1 X ...... + X’

HM-2 sM-2↓ YM-2 sM-2↑ GM-2

HM-1 sM-1↓ YM-1 sM-1↑ GM-1

1 ∑ =1 i si

Norsk Regnesentral Norwegian Computing Center

One-dimensional Case !

Analysis/Synthesis

H0 2↓ Y0 2↑ G0 X + X’ H1 2↓ Y1 2↑ G1

Norsk Regnesentral Norwegian Computing Center One-dimensional Case !

Analysis/Synthesis

H Y H X AS0 X’ L L Y1

H Y H A 0 S X’ X L L

H Y H AS1 L L Y2

Norsk Regnesentral Norwegian Computing Center

Two-dimensional Case !

Transformation of Images

H H YHH AV SV L L YHL

H H X AH SH X’ L L

H H YLH AV SV L L YLL

Norsk Regnesentral Norwegian Computing Center Two-dimensional Case !

Transformation of Images

BLL BHL B

BLH BHH

Norsk Regnesentral Norwegian Computing Center

Two-dimensional Case !

Construction of Haar Basis

Standard construction Non-Standard construction of Haar basis of Haar basis

Norsk Regnesentral Norwegian Computing Center Norsk Regnesentral Norwegian Computing Center

Norsk Regnesentral Norwegian Computing Center JPEG 2000

• ISO/IEC 15444 consists of 10 parts: – Part 1: Core coding system (JP2, code stream) – Part 2: Extensions (JPX, others) – Part 3: Motion JPEG 2000 – Part 4: Conformance Testing – Part 5: Reference software – Part 6: Compound image file format (JP2, JPX) – Part 7-10: JPSEC, JPIP, JP3D

Norsk Regnesentral Norwegian Computing Center

JPEG 2000

• Improved R-D performance at low and medium bitrates • Covers low bit-rate lossy coding up to lossless coding • Progressive Transmission Capability (resolution & SNR) • Region of interest (ROI) representation and editing • Error Resilience

• Approach: Wavelet Transformation + EBCOT + Adaptive Arithmetic Coding • Embedded Block Coding with Optimized Truncation

Source: Thomas Wiegand Norsk Regnesentral Norwegian Computing Center Part 1: Core Coding system

• overview & 11 annexes: – Annex A - Code stream Syntax – Annex B - Data ordering – Annex C - Arithmetic entropy coding (MQ coder) – Annex D - Coefficient bit modeling – Annex E - Quantization – Annex F - Discrete wavelet transform of tile components – Annex G - DC level shifting and component transforms – Annex H - Coding of images with Regions of Interest – Annex I - JP2 file format syntax – Annex J - Examples and guidelines – Annex K - Bibliography Norsk Regnesentral Norwegian Computing Center

JPEG 2000 Part 1

Norsk Regnesentral Norwegian Computing Center Encoding in JPEG 2000

• DC level component transform (G) • image decomposition (tiles, sub-bands) (B) • DFWT (F.3) • Quantization (E) • Coefficient bit modeling (D) • Arithmetic coding (C.1, C.2) (MQ coder) • Code stream Syntax (A) • Encoding to JP2 file format (I)

Norsk Regnesentral Norwegian Computing Center

Image Decomposition

• components • tiles • sub-bands • precincts • code blocks • layer • packet

Norsk Regnesentral Norwegian Computing Center DC level component transform

• components • unsigned → signed values • RCT: reversible component • tiles transform (used with 5-3 • sub-bands reversible WT) • precincts • ICT: irreversible component transform (used with 9-7 • code blocks irreversible WT) • layer • When first components are RGB: ICT is approximation to • packet YCbCr transform.

Norsk Regnesentral Norwegian Computing Center

Image Decomposition

• components Component • tiles • Two-dimensional array of samples (1D) • sub-bands • Color images consists • precincts of several • code blocks components • layer • packet

Norsk Regnesentral Norwegian Computing Center Reference Grid

• components • Component has • tiles reference grid • (0,0) upper left • sub-bands • precincts • code blocks • layer • packet

Norsk Regnesentral Norwegian Computing Center

Reference Grid

• components • Rectangular array of • tiles tiles • Row by row • sub-bands • precincts • code blocks • layer • packet

Norsk Regnesentral Norwegian Computing Center Sub-Bands

• components • tiles • sub-bands • precincts • code blocks • layer • packet

Norsk Regnesentral Norwegian Computing Center

Precinct Partition

• components • Tiles are divided into • tiles precincts of size PP PP • sub-bands 2 x ,2 y  • precincts • code blocks • layer • packet

Norsk Regnesentral Norwegian Computing Center • Each layer divided into code blocks Code Blocks • Code blocks equal size for entire tile • components • Coefficient bit modeling • tiles • sub-bands • precincts • code blocks • layer • packet

Norsk Regnesentral Norwegian Computing Center

Layers

• components • One or more layers in • tiles code stream • Each layer consists of • sub-bands consecutive bit-plane • precincts passes from each code • code blocks block in the tile • layer • Each layer successively • packet and monotonically improves image quality

Norsk Regnesentral Norwegian Computing Center Packet

• components • 8 bit packet boundaries • tiles • Contributions in raster order • For resolution r=0: • sub-bands – LL band only • precincts • For resolution r>0: • code blocks – HL, LH, HH bands only • layer • Progression order • packet

Norsk Regnesentral Norwegian Computing Center

FDWT Fast Discrete Wavelet Transform

Norsk Regnesentral Norwegian Computing Center Wavelet transforms in JPEG2000

5/3 wavelet transform

9/7 wavelet transform

Norsk Regnesentral Norwegian Computing Center

Extension of signal

Norsk Regnesentral Norwegian Computing Center Extension for Haar Wavelet

• Extend with mirrored pixel 17 21 9 5 7 5 • Perform wavelet transform • The 1 is redundant 19 7 6 -2 2 1

Reconstruction • Perform IWT 17 21 9 5 7 5 • The 1 is reconstructed from 6 and 5

Norsk Regnesentral Norwegian Computing Center

Decomposition into sub-bands

Norsk Regnesentral Norwegian Computing Center IDWT Inverse Discrete Wavelet Transform

Norsk Regnesentral Norwegian Computing Center

Quantization

• Divide by factor and use floor-function • Irreversible case: Scalar Quantization with dead zone • separate approach for every sub-band possible

Norsk Regnesentral Norwegian Computing Center Arithmetic entropy coding

• Compatible with ISO/IEC 14492 • MQ-Coder

Norsk Regnesentral Norwegian Computing Center

Codestream syntax

• Markers: 0xFF + Opcode (0x01-0xFE) • Marker segments: marker + parameter • Marker segments include a length-field • Headers: Collections of markers and marker segments • Codestream: SOC – SIZ - ... - EOC • Part: SOT - ... - SOD ... bitstream

Norsk Regnesentral Norwegian Computing Center Construction of codestream

Norsk Regnesentral Norwegian Computing Center

Coding into JP2 File Format

• Sequence of boxes • Simple boxes (containing payload) • Superboxes (containing other boxes) • optional/mandatory boxes

Norsk Regnesentral Norwegian Computing Center JPEG 2000 Process

Source: Thomas Wiegand Norsk Regnesentral Norwegian Computing Center

JPEG vs JPEG2000 (Original)

Norsk Regnesentral Norwegian Computing Center JPEG 2000 130 x compression

Norsk Regnesentral Norwegian Computing Center

JPEG 130 x compression

Norsk Regnesentral Norwegian Computing Center Literature - Wavelets

• Daubechies: Ten Lectures on Wavelets, Society for Industrial and Applied Mathematics • Fournier: Wavelets an their Applications in Computer Graphics, SIGGRAPH `94 Course Notes • Stollnitz, DeRose, Salesin: Wavelets for computer graphics: theory and applications, Morgan Kaufman, 1996

Norsk Regnesentral Norwegian Computing Center

End of Part

Thank you for your attention!

Norsk Regnesentral Norwegian Computing Center Sacalability

• Embedded codes permit decoding with reduced quality with parts of the complete code • Resolution scalability – Straight forward (DWT property) – Sort bits from lowest to highest sub-band • SNR scalability – Embedded quantization – Transmit sequence of quantizers Source: Thomas Wiegand Norsk Regnesentral Norwegian Computing Center

Embedded Quantization

Source: Thomas Wiegand Norsk Regnesentral Norwegian Computing Center