2011/2012 2011/2012 Multimedia Systems Multimedia ColorTransformationImage Compression/ Prof. Dr. Paul Müller Müller Dr.Paul Prof. UniversityKaiserslauternof DepartmentComputerof Science IntegratedCommunication ICSY Systems http://www.icsy.de

2

Sitemap Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

3 Color representation, transformation, and subsampling subsampling and transformation, representation, Color cones rods light-sensitivecellstheretina: rodsof and cones - approximately- 6,5million supply- color (chrominance) information three- typesdepending on thewavelength (red 600nm, green 540nm, blue 450nm) - approximately- 120 million supply- brightness (luminance) information verysensitive- lightto high- spatial resolution

- - - Eye

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. }

4 Color representation, transformation, and subsampling subsampling and transformation, representation, Color Anappropriate color representation should pay attention that. to Thehuman eyeismuch more sensitive changesto theof luminancethanchangesfor thewavelength! of

- - Colorresolution islower than brightness resolution

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. à

5 monitor system eachpixel consiststhreephosphor of typesof dots threeelectron guns notperceptually based Color representation, transformation, and subsampling subsampling and transformation, representation, Color

• • • • RGB Thetechnical evolution in theareacomputer of and monitor techniquesresulted indevelopment the variousof different schemescolorfor representation (physically and perceptually basedcolor models). Having in mind thethreecones,of types colormost models deal withthree basic colors, e.g.

- - Technicaldevelopment

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. }

6 Hue:sensory impression apureof spectral color (wave length) Saturation:mixturethecolor of tone with grey Luminance:overall brightness perceptuallybased Color representation, transformation, and subsampling subsampling and transformation, representation, Color

• • • • HLS

- Technicaldevelopment

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. }

7 or

HSL / HSV / HSL as HLS HSL alue. stands for v standing for I aturationand brightness HSV s are tworelated isoftenalso for ue, B , with h HSV HSV or HSB aturationand s and isoftenalso called , and ue, ightness,while HSL HSL intensity thethirdcoordinate; their definitionsare less standardized,and they occasionallysomereferto othersimilar model. than RGB, while thanRGB, remaining computationallysimple. standsfor l h HSI called representationspointsof in an colorRGB model attempt that describeto perceptual color relationshipsmore accurately

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. } }

8 Color representation, transformation, and subsampling subsampling and transformation, representation, Color Technicaldevelopment (cont.)

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. }

9 generatedwith theinvention thecolorof television maintainto basicelement thevideoof standards PALand NTSC "perceptuallybased” brightnessinformation isseparated Y colorfrom difference signals U compatibilitywith theb/wtelevision andV Color representation, transformation, and subsampling subsampling and transformation, representation, Color

• • • • YUV

- Technicaldevelopment (cont.)

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. }

10 can besubsampled in their color b C r YC

b C r deducedandfrom similaruse YUVto withto thedigital video standard Color representation, transformation, and subsampling subsampling and transformation, representation, Color

• BothYUVand YC componentsachieveto an optimal perceptionalfittingto needs.

- - Technicaldevelopment (cont.)

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. }

11 Color representation, transformation, and subsampling subsampling and transformation, representation, Color Technicaldevelopment (cont.)

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. }

12 Color representation, transformation, and subsampling subsampling and transformation, representation, Color PAL-Normcolor representation: YUVExample:→ RGB Y = 0,30 R + 0,59 G + 0,11+ B 0,59 0,30= R+ G Y 0,44B -0,100,29G+ 0,493= R- U= –Y) (B 0,62–0,100,8770,52 B = = R– G V (R–Y)

- - Colorspace transformation

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. }

13

subsampled Color representation, transformation, and subsampling subsampling and transformation, representation, Color Sincethehuman perception colorof ismuch less accurate In generalIn thecolor space transformation islossless thantheperception luminance,of chrominancethe data will often be

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. } }

14

, Uand are, V scanline downsampling only) , 2:1vertical , scanlines downsampling (= 4 Y samples 1Uand samplesfor 1V per scanline ) Y 4(= 2:1horizontal samplessamples 2Uor2V for per Y 4(= processedon alternating nolossinformation of 2:1horizontal downsampling, no vertical downsampling samples 2Uand samplesfor 2V per scanline ) Y 4(= 4:1horizontal downsampling, no vertical downsampling Encodingparameters digitalof television studios for Color representation, transformation, and subsampling subsampling and transformation, representation, Color

• a:b:c notationa:b:c samplingfor ratio accordingCCIR)(formerrecommendation ITU-R to 601-2:

- 4:2:0 4:4:4 4:2:2 4:1:1 Colorsampling ratioYUVfor

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. }

15 Color representation, transformation, and subsampling subsampling and transformation, representation, Color Colorsampling (cont.) ratioYUVfor

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. }

16 Hybrid Coding Steps for Images, Video, and Audio Audio and Video, Images, for Steps Coding Hybrid Basicencoding steps:

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. }

17

(baselinesystem)

Image Compression – JPEG JPEG –CompressionImage selectcolor space,color-subsampling, quantization,DCT, coding

- Image preparation and coding: and Imagepreparation

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. }

18

8-bitnumbers represent greyscales

– Thefollowing example showsan8x8 pixel image section

• Image Compression – JPEG JPEG –CompressionImage

r C selectcolor space color-subsampling DCT quantization coding b

1) 2) 3) 4) 5) greyscale RGB YC

– – – colorspace

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. •

19

r or C or b C Theeye can sense local Averagedvalues 2x2of Assumingagiven 16x16 blockwith differencesonly poorly sections

} } } Image Compression – JPEG JPEG –CompressionImage selectcolor space color-subsampling DCT quantization coding

1) 2) 3) 4) 5) Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

20 thus P=8; left shift of all of shift left thusP=8; sourcethisisthe X matrix 8x8block with 8-bit values itemscorresponds -128to

} } } blocks Image Compression – JPEG JPEG –CompressionImage NxN selectcolor space color-subsampling DCT quantization coding

normalizevalues enforce apply2-DDCT 1) 2) 3) 4) 5)

1) 2) 3) DCT DCT

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. •

21

T Applicationthe2-D DCTof 8x8block contains only valuesbetween -128 and 127 leadsthetransformedto A·X·A matrixD=

• •

Image Compression – JPEG JPEG –CompressionImage NxN selectcolor space color-subsampling DCT quantization coding

normalizevalues enforce apply2-DDCT blocks 1) 2) 3) 4) 5)

1) 2) 3) DCT DCT

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. •

22 according ij q quantifyaccording to selectaquantization table Q with items Q recommendation JPEG to statedformula and obtain table L with quantified DCT-coefficients

• • Image Compression – JPEG JPEG –CompressionImage selectcolor space color-subsampling DCT quantization coding

storeresultsin L selecttable Q quantifyDCT-

coefficients 1) 2) 3) 4) 5)

- - - Quantization

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. }

23 11) bitword assumingHuffman-Code for resultingsequence 0111DC: for here:3 3isin group 2and ison position group2is2-bit long 01) (e.g. 4there (thus

– – – – codingtable: 16 categories with codedifference precedingto DC-coefficient increasingnumber entriesof DC-coefficient:

• • • Image Compression – JPEG JPEG –CompressionImage selectcolor space color-subsampling DCT quantization coding

DC-coefficient AC-coefficients 1) 2) 3) 4) 5)

- - Coding

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. }

24 and coded absolutely 1 -> (0,1) -> 00 1(0,1)-> 1-> 1011 (0,4)-> -9-> 0110 01 11(0,2)-> 3-> 10100…0(0,0)-> -> (2ndposition incategory) 1st

• • • • Z: number precedingZ: of zeros categorycoefficientC: theAC of theexampleFor theresultsare:

– – – The (Z,C) table The(Z,C) contains AC-coefficientsare traversed in zick-zack Huffmancodes AC- for sequences

• • Image Compression – JPEG JPEG –CompressionImage selectcolor space color-subsampling DCT quantization coding

DC-coefficient AC-coefficients 1) 2) 3) 4) 5)

- - Coding

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. }

25 Image Compression – JPEG JPEG –CompressionImage thereconstructed block hasonly small errors [values in%] 8x8section coded with 23 bit ini.e. average 23/64 instead 8bitperof pixel 0111 0011 0110 1100 1111 010 0110 1100 1111 0111 0011

- - - Result

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. }

26 Overview Overview

Quantizer Tier2coding Tier1/ Preprocessing DWT

• • • • Comparison2000 JPEG with of JPEG JPEG 2000 JPEG

- - Waveletsin practice

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. }

27 JPEG 2000diagram JPEG block Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

28 Baboon original original Baboon Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

29 colorspace RGB RGB Baboon Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

30 colorspace

YCrCb Baboon Baboon Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

31 subsampled 2x subsampled luma Baboon Baboon Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

32 subsampled 4x subsampled luma Baboon Baboon Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

33 subsampled 8x subsampled luma Baboon Baboon Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

34 Baboon original original Baboon Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

35 subsampled 2x subsampled chroma Baboon Baboon Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

36 subsampled 4x subsampled chroma Baboon Baboon Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

37 subsampled 8x subsampled chroma Baboon Baboon Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

38 Baboon original original Baboon Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

39 JPEG 2000diagram JPEG block Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

40

lossy (9,7)floating point or(5,3)integer filter Transformation in JPEG 2000 JPEG in Transformation daubechies Multi-resolutionimage representation Full-frametransformation no-> blocking artifacts Uses Useintegerof bothfilters-> lossless and Twoimplementation approaches: filterbased lifting / DWT provides DWT frequency band decomposition the of scheme compression image

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. } } } } } }

41 1-D two-band DWT DWT 1-Dtwo-band Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

42 1-D two-band DWT DWT 1-Dtwo-band Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

43 2-D wavelet decomposition 2-Dwaveletdecomposition Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

44

Lena original Lena Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

45 Lena horizontal high-pass high-pass horizontal Lena Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

46 Lena horizontal low-pass low-pass horizontal Lena Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

47 Lena low + high-pass subsampled subsampled high-pass + low Lena Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

48 1-level 2-D wavelet decomposition 1-level2-Dwavelet decomposition Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

49 2-level 2-D wavelet decomposition 2-level2-Dwavelet decomposition Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

50 3-level 2-D wavelet decomposition 3-level2-Dwavelet decomposition Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

51 JPEG 2000diagram JPEG block Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

52

Dequantization Quantization and and Quantization Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

53 Quantization in JPEG 2000 JPEG in Quantization Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

54 Quantization by bit-plane coding coding bit-plane by Quantization Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

55 step size3step= Quantizer Bit-plane example Bit-plane value 0.5)*48(1+ = 72= value 0,5)*12(6+ = 78= value based on fullydecoded index value decodingafter 6BP‘s: value decodingafter 4BP‘s: index 270001101183/3 = = = Decodedindex 0001size48= Step1; = = Dequantized (270.5)*3+ 82, = Decodedindex 000110= 12Stepsize = 6; = Dequantized

- - - - - Dequantized Dequantized Dequantized Waveletcoefficient83; = Quantizer

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. } } } } }

56 JPEG 2000diagram JPEG block Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

57 Entropy coding in JPEG 2000 JPEG in coding Entropy Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

58 JPEG 2000 blocks coder JPEG Entropy Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

59

Bit-plane 1 Bit-plane Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

60 Bit-plane 1 enlarged by 8xby enlarged 1 Bit-plane Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

61

Bit-plane 2 Bit-plane Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

62 Bit-plane 2 enlarged by 8xby enlarged 2 Bit-plane Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

63

Bit-plane 3 Bit-plane Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

64

Bit-plane 4 Bit-plane Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

65

Bit-plane 5 Bit-plane Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

66

Bit-plane 6 Bit-plane Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

67

Bit-plane 7 Bit-plane Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

68

Bit-plane 8 Bit-plane Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

69

Bit-plane 9 Bit-plane Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

70

Bit-plane 10 Bit-plane Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

71

Bit-plane 11 Bit-plane Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

72 JPEG 2000diagram JPEG block Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

73

for inclusion for in the bitstreams for each for code-block Tier 2 role 2 role Tier bitstreams bitstream passes in an efficient manner and signals theordering resultingtheof bitplane One independent One independent

- Tier2multiplexes the Tier2enables SNR,resolution, andspatial, arbitraryROI Tier1generates acollection of codestream coded progressionand scalability

Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. } } }

74 Example of coded data coded Exampleof Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

75 Example of coded data coded Exampleof Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

76 Lowest resolution, highest quality quality highest resolution, Lowest Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

77 Highest resolution, highest quality quality highest resolution, Highest Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

78 Highest resolution, target SNR quality targetSNR quality resolution, Highest Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

79 Highest resolution, target visual quality quality target visual resolution, Highest Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

80 SNRprogressive example Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

81 example SNRprogressive Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

82 SNRprogressive example Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

83 Resolution progressive example Resolution Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

84 Resolution progressive example Resolution Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

85 Resolution progressive example Resolution Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

86 Resolution progressive example Resolution Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof.

87 JPEG (43:1) JPEG original JPEG vs. JPEG 2000 JPEG vs. JPEG Prof. Dr. Paul Müller, University Kaiserslauternof Paul Müller, Dr. Prof. JPEG 2000 JPEG (43:1)