Internet Engineering Dr. Marek Woda Multimedia and Computer Visualisation Part 4 JPEG compression Joint Photographic Expert Group - 1986 • ISO - International Standard Organisation • CCITT - Comité Consultatif International de Téléphonie et Télégraphie ISO standard - 1991 Application of the algorithm – compression of photorealistic image Assumptions: An image is an array: f = f ( x, y ) x = 0,1,2,..,N - 1; y = 0,1,2,..,M - 1 where f(x,y) - element of image (pixel), N, M - image width and height, Element f(x,y) can have different sense e. g. - gray level, f ( x, y ) Î {0,1,...,S } - color, f ( x, y ) = [r( x, y ) g( x, y ) b( x, y )] r , g,b Î{0,1,...,S} Phases of the JPEG algorithm 1. Conversion to luminance-chrominace color model (only for color image) 2. Division into blocks 3. Calculation of the Discrete Cosine Transform (DCT) 4. Quantisation of DCT coefficients 5. Conversion of DCT coefficients array to a vector 6. Entropy Coding 1. Conversion to luminance-chrominace color model (YUV, YCbCr) Source image representation - RGB model: R = [rij ], G = [gij ], B = [bij ] Conversion formula (from RGB to YUV color model): é yij ù é 0.229 0.587 0.114 ùé rij ù ê ú ê ú u = ê- 0.146 - 0.288 - 0.434ú g ê ij ú ê úê ij ú ê ú ê ú ëvij û ëê 0.617 - 0.517 0.100 ûúëbij û after conversion: Y = [yij ], U = [iij ], V = [qij ] where Y – luminance and U and V chrominance 2. Division into blocks Division of the image into matrices of 8 x 8 pixel blocks, where each block is an array : f ( x, y ) x = 0,1,...,7 y = 0,1,...,7 3. Calculation of the Discrete Cosine Transform (DCT) f(x,y) F(u,v) C( u )C( v ) 7 7 æ 2x + 1 ö æ 2 y + 1 ö F( u,v ) = å å f ( x, y )cos ç up ÷cos ç vp ÷ 4 x=0 y=0 è 16 ø è 16 ø F(u,v) f(x,y) 1 7 7 æ 2x + 1 ö æ 2 y + 1 ö f ( x, y ) = å åC( u )C( v )F( u,v )cos ç up ÷cos ç vp ÷ 4 u=0 v=0 è 16 ø è 16 ø ì1 / 2 dla u = 0 ì1 / 2 dla v = 0 where C( u) = í i C( v ) = í î1 dla u ¹ 0 î1 dla v ¹ 0 Image 1 „plane” block of the input image f(x,y) F(u,v) block (function) block (DCT transform) Image 2 „chessboard” block of the input image f(x,y) F(u,v) block (function) block (DCT transform) Image 3 „photorealistic image” block of the input image f(x,y) F(u,v)F(u,v) block (function) block (DCT transform) Function and DCT transform for the „photorealistic image” 186 198 199 190 182 177 182 197 179 184 183 176 173 172 175 184 188 182 180 178 174 172 171 166 f(x,y) = 132 130 139 146 151 169 191 201 131 134 137 140 139 139 139 138 153 157 161 172 177 145 89 49 190 178 192 196 120 43 39 47 176 184 187 112 41 39 43 44 1.2047 0.1372 -0.0212 -0.0364 0.0023 0.0088 0.0023 0.0002 0.2165 -0.1758 0.0319 0.0240 -0.0012 -0.0143 -0.0025 -0.0002 -0.0087 0.1324 0.0194 -0.0460 -0.0065 0.0029 0.0046 0.0001 0.0169 -0.0018 -0.0613 0.0242 0.0146 -0.0103 -0.0063 -0.0006 F(u,v) = * 1.0e+003 -0.0315 -0.0626 0.0572 -0.0192 -0.0225 0.0000 0.0069 -0.0004 0.0287 0.0069 -0.0122 -0.0150 0.0260 0.0086 -0.0065 0.0001 0.0123 0.0115 -0.0166 0.0300 -0.0216 -0.0075 0.0049 0.0004 -0.0005 0.0352 0.0060 -0.0166 0.0128 0.0052 -0.0039 -0.0005 4. Quantisation of DCT coefficients F(u,v) FQ(u,v) æ F( u,v )ö F Q ( u,v ) = Integer Round ç ÷ è Q( u,v )ø é16 11 10 16 24 40 51 61ù é17 18 24 47 24 40 51 61ù ê ú 12 12 14 19 26 58 60 56 ê18 21 26 66 26 58 60 56ú ê ú ê ú ê14 13 16 24 40 57 69 56ú ê24 26 56 99 99 99 99 99ú ê ú ê ú 14 17 22 29 51 87 80 62 47 99 99 99 99 99 99 99 Q( u,v ) = ê ú Q( u,v ) = ê ú ê18 22 37 56 68 109 103 77ú ê99 99 99 99 99 99 99 99ú ê ú ê ú ê24 35 55 64 81 104 113 92ú ê99 99 99 99 99 99 99 99ú ê49 64 78 87 103 121 120 101ú ê99 99 99 99 99 99 99 99ú ê ú ê ú ë72 92 95 98 112 100 103 99û ë99 99 99 99 99 99 99 99û for luminance Y for chrominance I and Q DCT transform coefficients after quantisation for the „photorealistic image” 1.2047 0.1372 -0.0212 -0.0364 0.0023 0.0088 0.0023 0.0002 0.2165 -0.1758 0.0319 0.0240 -0.0012 -0.0143 -0.0025 -0.0002 -0.0087 0.1324 0.0194 -0.0460 -0.0065 0.0029 0.0046 0.0001 0.0169 -0.0018 -0.0613 0.0242 0.0146 -0.0103 -0.0063 -0.0006 F(u,v) = * 1.0e+003 -0.0315 -0.0626 0.0572 -0.0192 -0.0225 0.0000 0.0069 -0.0004 0.0287 0.0069 -0.0122 -0.0150 0.0260 0.0086 -0.0065 0.0001 0.0123 0.0115 -0.0166 0.0300 -0.0216 -0.0075 0.0049 0.0004 -0.0005 0.0352 0.0060 -0.0166 0.0128 0.0052 -0.0039 -0.0005 75 12 -2 -2 0 0 0 0 18 15 2 1 0 0 0 0 -1 0 1 -2 0 0 0 0 FQ(u,v) = 1 0 -3 1 0 0 0 0 -2 -3 2 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Inverse operation (decoding) Integer Round (DCT-1( FQ(u,v)*Q(u,v) )) f *(x,y) 186 198 199 190 182 177 182 197 179 184 183 176 173 172 175 184 188 182 180 178 174 172 171 166 f(x,y) = 132 130 139 146 151 169 191 201 131 134 137 140 139 139 139 138 153 157 161 172 177 145 89 49 190 178 192 196 120 43 39 47 176 184 187 112 41 39 43 44 183 186 187 182 176 178 188 198 178 188 196 192 180 169 168 171 169 174 178 175 170 170 176 183 f*(x,y) = 147 140 133 135 148 168 186 197 131 126 126 135 149 153 146 136 150 160 173 178 163 127 82 51 176 190 195 172 125 75 44 31 181 185 168 114 50 19 32 58 Coding and Decoding (examples) • „photorealistic image” before compression f(x,y) after compression and decompression f*(x,y) • „chessboard” before compression f(x,y) after compression and decompression f*(x,y) 5. Conversion of DCT coefficients array to a vector Q F (u,v) [DC, AC1, AC2 ,..., AC63] zig-zag algorithm (A. G. Tescher 1978) 75 12 -2 -2 0 0 0 0 18 15 2 1 0 0 0 0 -1 10 1 -2 0 0 0 0 FQ(u,v) = 1 0 -3 1 0 0 0 0 -2 -3 2 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [DC, AC1,...,AC63] = [ 75, 12, 18, -1, 15, -2, -2, 2, 10, 1, -2, 0, 1, 1, 0, 0, 0, -2, -3, -3, 1, 0, 0,..., 0] 6. Entropy Coding vector - [DC, AC1, AC2 ,..., AC63] • coding of DC for blocks (array of the blocks) • coding AC1, AC2, ... , AC63 for each block The entropy coder compresses data by replacing each fixed-length input symbol by the corresponding variable-length prefix codeword. The length of each codeword is proportional to the negative logarithm of the codeword probability. 6.1. DC coding The image has been devided into blocks (8x8 pixels). DC0 DC1 DC2 block 0 block 1 block 2 ... DCi - DC value for block i, DCk DCk+1 i = 0, 1, ..., m block k block k+1 ... m – number of blocks DC2k block 2k ... ... Coding of the DC (DPCM algorithm) 1. Construction of the vector DC = [DC0, DC1, DC2,..., DCk, DCk+1,..., DCm]. 2. Calculation Δ = [Δ0, Δ1, ..., Δi , ..., Δm] where 0 Δ0 = DC i i-1 Δi = DC - DC i = 1,2, ...,m 3. Coding Δ = [Δ0, Δ1, ..., Δi , ..., Δm] using Huffman code table ( Table 1) Table 1. Huffman Δi value Size code for Additional bits Size 0 0 00 - -1, 1 1 010 0,1 -3,-2, 2, 3 2 011 00,01,10,11 -7,…,-4,4,…7 3 100 000,…,011,100,…,111 -15,…,-8,8,…,15 4 101 0000,…,0111,1000,…,1111 … … … … -2047,…-1024,1024,…,2047 11 1 1111 1110 000 0000 0000,…,111 1111 1111 Coding procedure: • For next Δi calculate Size using formula Size = Integer Round [log2( abs(Δi) ) +1] • For calculated Size read (Table 2) Huffman code and additional bits Example: For the next block the sequence of coefficients is: [DC, AC1,...,AC63] = [75, 12, 18, -1, 15,…] For the previous block DC value is 71.