Block Transform Coding
Section 8.2.8
Image Compression-II 1
Transform coding
Construct Forward Symbol n X n Quantizer subimages transform encoder N X N Compressed images image
Compressed Uncompressed Merge image Symbol Inverse image n X n decoder transform subimages
Blocking artifact: boundaries between subimages become visible
Image Compression-II 2 Transform Selection
DFT Discrete Cosine Transform (DCT) Wavelet transform Karhunen-Loeve Transform (KLT) …. n−1 n−1 T(u,v) = ∑∑ g(x, y)r(x, y,u,v) − − − −(8.2.10) x=0 y=0 N −1 N −1 g(x, y) = ∑∑T(u,v)s(x, y,u,v) − − − −8.2.11 u=0 v=0
Image Compression-II 3
Transform Kernels
r(x, y,u,v) = r (x,u)r ( y,v) − − − −(8.2.12) Separable if 1 2 E.g. DFT: r(x, y,u,v) = exp(− j2π(ux + vy) / n) E.g. Walsh-Hadamard transform (see page 568, text)
E.g. DCT
Image Compression-II 4 Approxima tions using DFT, Hadamard and DCT, and the scaled error images
Image Compression-II 5
Discrete Cosine Transform
Ahmed, Natarajan, and Rao, IEEE T-Computers, pp. 90-93, 1974.
Image Compression-II 6 1-D Case: Extended 2N Point Sequence
Consider 1- D first; Let x(n) be a N point sequence 0 ≤ n ≤ N -1. 2 - N point DFT 2 − N point N − point x(n) ↔ ↔ y(n) ↔ Y(u) C(u) x(n), 0 ≤ n ≤ N − 1 y(n) = x(n) + x(2N − 1− n) = x(2N − 1− n), N ≤ n ≤ 2N − 1 x(n) y(n)
0 1 2 3 4 0 1 2 3 4 5 6 7 8
Image Compression-II 7
DCT & DFT
2 N −1 2π Y(u) = ∑ y(n)exp − j un n=0 2N N −1 2π 2 N −1 2π = ∑ x(n)exp − j un + ∑ x(2N − 1− n)exp − j un n=0 2N n= N 2N N −1 2π N −1 2π = ∑ x(n)exp − j un + ∑ x(m)exp − j u(2N − 1− m) n=0 2N m=0 2N π N −1 π 2π = exp j u ∑ x(n)exp − j u − j un 2N n=0 2N 2N π N −1 π 2π + exp j u ∑ x(n)exp j u + j un 2N n=0 2N 2N π N −1 π = exp j u 2x(n)cos u(2n + 1) . 2N ∑ 2N n=0 Image Compression-II 8 DCT
The N-point DCTof x(n), C(u), is given by π exp − j u Y (u), 0 ≤ u ≤ N-1 C(u) = 2N 0 otherwise. The unitary DCT transformations are: N −1 π F(u) = α(u)∑ f (n)cos (2n + 1)u , 0 ≤ u ≤ N − 1, where n=0 2N 1 2 α(0) = , α(u) = for 1 ≤ k ≤ N − 1. N N The inverse transformation is N −1 π f (n) = ∑α(u)F(u)cos (2n + 1)u , 0 ≤ u ≤ N − 1. n=0 2N
Image Compression-II 9
Discrete Cosine Transform—in 2-D N −1 N −1 C(u,v) (u) (v) f (x, y)cos (2 x+1)uπ cos (2 y+1)vπ = α α ∑ ∑ 2 N 2 N x=0 y=0 for u, v = 0,1,2,…, N − 1, where 1 N for u = 0 α(u) = and 2 for u= 1,2,..,N-1. N N −1 N −1 f (x, y) (u) (v)C(u,v)cos (2 x+1)uπ cos (2 y+1)vπ = ∑ ∑α α 2 N 2 N u=0 v=0 for x, y 0,1,2, , N 1 = … −
Image Compression-II 10 DCT Summary
N −1 N −1 T(u,v) = ∑∑ f (x, y)r(x, y,u,v) − − − −8.2.10 x=0 y=0 r(x, y,u,v) = s(x, y,u,v) = α(u)α(v)cos (2x+1)uπ cos (2 y+1)vπ ...... 8.2.18 2 N 2 N
Image Compression-II 11
DCT Basis functions
Image Compression-II 12 Implicit Periodicity-DFT vs DCT (Fig 8.32)
Image Compression-II 13
Why DCT?
Blocking artifacts less pronounced in DCT than in DFT. Good approximation to the Karhunen-Loeve Transform (KLT) but with basis vectors fixed. DCT is used in JPEG image compression standard.
Image Compression-II 14 Sub-image size selection (fig 8.26)
Image Compression-II 15
Different sub-image sizes
Image Compression-II 16 Bit Allocation/Threshold Coding
• # of coefficients to keep Threshold coding • How to quantize them Zonal coding Threshold coding For each subimage i • Arrange the transform coefficients in decreasing order of magnitude • Keep only the top X% of the coefficients and discard rest. • Code the retained coefficient using variable length code.
Image Compression-II 17
Zonal Coding
Zonal coding: (1) Compute the variance of each of the transform coeff; use the subimages to compute this. th i coefficient (2) Keep X% of their coeff. which in each sub have maximum variance. image (3) Variable length coding (proportional to variance)
Bit allocation: In general, let the number of bits allocated be made proportional to the variance of the coefficients. Suppose the total number of bits per block is B. Let the number of retained coefficients be M. Let v(i) be variance of the i-th coefficient. Then M b(i) B 1 log v(i) 1 log v(i) = M + 2 2 − 2 M ∑ 2 i=1
Image Compression-II 18 Zonal Mask & bit allocation: an example
1 1 1 1 1 0 0 0 8 7 6 4 3 0 0 0
1 1 1 1 0 0 0 0 7 6 5 4 0 0 0 0
1 1 1 0 0 0 0 0 6 5 4 0 0 0 0 0
1 1 0 0 0 0 0 0 4 4 0 0 0 0 0 0
1 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Image Compression-II 19
Typical Masks (Fig 8.36)
Image Compression-II 20 Image Approximations
Image Compression-II 21
The JPEG standard
The following is mostly from Tekalp’s book, Digital Video Processing by M. Tekalp (Prentice Hall). For the new JPEG-2000 check out the web site www.jpeg.org. See Example 8.17 in the text
Image Compression-II 22 JPEG (contd.)
JPEG is a lossy compression standard using DCT. Activities started in 1986 and the ISO in 1992. Four modes of operation: Sequential (basline), hierarchical, progressive, and lossless. Arbitrary image sizes; DCT mode 8-12 bits/ sample. Luminance and chrominance channels are separately encoded. We will only discuss the baseline method.
Image Compression-II 23
JPEG-baseline.
DCT: The image is divided into 8x8 blocks. Each pixel is level shifted by 2n-1 where 2n is the maximum number of gray levels in the image. Thus for 8 bit images, you subtract 128. Then the 2-D DCT of each block is computed. For the baseline system, the input and output data precision is restricted to 8 bits and the DCT values are restricted to 11 bits. Quantization: the DCT coefficients are threshold coded using a quantization matrix, and then reordered using zig-zag scanning to form a 1-D sequence. The non-zero AC coefficients are Huffman coded. The DC coefficients of each block are DPCM coded relative to the DC coefficient of the previous block.
Image Compression-II 24 JPEG -color image
RGB to Y-Cr-Cb space – Y = 0.3R + 0.6G + 0.1B – Cr = 0.5 (B-Y) + 0.5 – Cb = (1/1.6) (R – Y) + 0.5 Chrominance samples are sub-sampled by 2 in both directions.
Y1 Y2 Y3 Y4 Cr1 Cr2 Cb1 Cb2 Y5 Y6 Y7 Y8 Cr3 Cr4 Cb3 Cb4
Y9 Y10 Y11 Y12 Non-Interleaved Y13 Y14 Y15 Y16 Scan 1:Y1,Y2,…,Y16 Scan 2: Cr1, Cr2, Cr3, Cr4 Scan 3: Cb1, Cb2, Cb3, Cb4 Interleaved: Y1, Y2, Y3, Y4, Cr1, Cb1,Y5,Y6,Y7,Y8,Cr2,Cb2,… Image Compression-II 25
JPEG – quantization matrices
Check out the matlab workspace (dctex.mat) Quantization table for the luminance channel. Quantization table for the chrominance channel. JPEG baseline method – Consider the 8x8 image (matlab: array s.) – Level shifted (s-128=sd). – 2d-DCT: dct2(sd)= dcts – After dividing by quantiization matrix qmat: dctshat. – Zigzag scan as in threshold coding. [20, 5, -3, -1, -2,-3, 1, 1, -1, -1, 0, 0, 1, 2, 3, -2, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, EOB].
Image Compression-II 26 An 8x8 sub-image (s)
s = (8x8block) sd =(level shifted) 183 160 94 153 194 163 132 165 55 32 -34 25 66 35 4 37 183 153 116 176 187 166 130 169 55 25 -12 48 59 38 2 41 179 168 171 182 179 170 131 167 51 40 43 54 51 42 3 39 177 177 179 177 179 165 131 167 49 49 51 49 51 37 3 39 178 178 179 176 182 164 130 171 50 50 51 48 54 36 2 43 179 180 180 179 183 164 130 171 51 52 52 51 55 36 2 43 179 179 180 182 183 170 129 173 51 51 52 54 55 42 1 45 180 179 181 179 181 170 130 169 52 51 53 51 53 42 2 41
Image Compression-II 27
2D DCT (dcts) and the quantization matrix (qmat) dcts= qmat= 312 56 -27 17 79 -60 26 -26 16 11 10 16 24 40 51 61 -38 -28 13 45 31 -1 -24 -10 12 12 14 19 26 58 60 55 -20 -18 10 33 21 -6 -16 -9 14 13 16 24 40 57 69 56 -11 -7 9 15 10 -11 -13 1 14 17 22 29 51 87 80 62 -6 1 6 5 -4 -7 -5 5 18 22 37 56 68 109 103 77 3 3 0 -2 -7 -4 1 2 24 35 55 64 81 104 113 92 3 5 0 -4 -8 -1 2 4 49 64 78 87 103 121 120 101 3 1 -1 -2 -3 -1 4 1 72 92 95 98 112 100 103 99
Image Compression-II 28 Division by qmat (dctshat)=dcts/qmat
dcthat= dcts=
20 5 -3 1 3 -2 1 0 312 56 -27 17 79 -60 26 -26 -3 -2 1 2 1 0 0 0 -38 -28 13 45 31 -1 -24 -10 -1 -1 1 1 1 0 0 0 -20 -18 10 33 21 -6 -16 -9 -1 0 0 1 0 0 0 0 -11 -7 9 15 10 -11 -13 1 0 0 0 0 0 0 0 0 -6 1 6 5 -4 -7 -5 5 0 0 0 0 0 0 0 0 3 3 0 -2 -7 -4 1 2 0 0 0 0 0 0 0 0 3 5 0 -4 -8 -1 2 4 0 0 0 0 0 0 0 0 3 1 -1 -2 -3 -1 4 1
Image Compression-II 29
Zig-zag scan of dcthat
dcthat= Zigzag scan as in threshold coding. 20 5 -3 1 3 -2 1 0 [20, 5, -3, -1, -2,-3, 1, 1, -1, -3 -2 1 2 1 0 0 0 -1, 0, 0, 1, 2, 3, -2, 1, 1, 0, -1 -1 1 1 1 0 0 0 0, 0, 0, 0, 0,1, 1, 0, 1, -1 0 0 1 0 0 0 0 EOB]. 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Image Compression-II 30 Threshold coding -revisited
Zig-zag scanning of the coefficients.
The coefficients along the zig-zag scan lines are mapped into [run,level] where the level is the value of non-zero coefficient, and run is the number of zero coeff. preceding it. The DC coefficients are usually coded separately from the rest.
Image Compression-II 31
JPEG – baseline method example
Zigzag scan as in threshold coding. [20, 5, -3, -1, -2,-3, 1, 1, -1, -1, 0, 0, 1, 2, 3, -2, 1, 1, 0, 0, 0, 0, 0, 0,1, 1, 0, 1, EOB].
The DC coefficient is DPCM coded (difference between the DC coefficient of the previous block and the current block.) The AC coef. are mapped to run-level pairs. (0,5), (0,-3), (0, -1), (0,-2),(0,-3),(0,1),(0,1),(0,-1),(0,-1), (2,1), (0,2), (0,3), (0,-2),(0,1),(0,1),(6,1),(0,1),(1,1),EOB. These are then Huffman coded (codes are specified in the JPEG scheme.) The decoder follows an inverse sequence of operations. The received coefficients are first multiplied by the same quantization matrix. (recddctshat=dctshat.*qmat.) Compute the inverse 2-D dct. (recdsd=idct2(recddctshat); add 128 back. (recds=recdsd+128.)
Image Compression-II 32 Decoder
Recddcthat=dcthat*qmat Recdsd= 320 55 -30 16 72 -80 51 0 67 12 -9 20 69 43 -8 42 -36 -24 14 38 26 0 0 0 58 25 15 30 65 40 -4 47 -14 -13 16 24 40 0 0 0 46 41 44 40 59 38 0 49 -14 0 0 29 0 0 0 0 41 52 59 43 57 42 3 42 0 0 0 0 0 0 0 0 44 54 58 40 58 47 3 33 0 0 0 0 0 0 0 0 49 52 53 40 61 47 1 33 0 0 0 0 0 0 0 0 53 50 53 46 63 41 0 45 0 0 0 0 0 0 0 0 55 50 56 53 64 34 -1 57
Image Compression-II 33
Received signal
s = (8x8block) Reconstructed S=
195 140 119 148 197 171 120 170 183 160 94 153 194 163 132 165 186 153 143 158 193 168 124 175 183 153 116 176 187 166 130 169 174 169 172 168 187 166 128 177 179 168 171 182 179 170 131 167 169 180 187 171 185 170 131 170 177 177 179 177 179 165 131 167 172 182 186 168 186 175 131 161 178 178 179 176 182 164 130 171 177 180 181 168 189 175 129 161 179 180 180 179 183 164 130 171 181 178 181 174 191 169 128 173 179 179 180 182 183 170 129 173 183 178 184 181 192 162 127 185 180 179 181 179 181 170 130 169
Image Compression-II 34 Example
Image Compression-II 35
Wavelet Compression
Image Compression-II 36 Image Compression-II 37
Image Compression-II 38 Image Compression: Summary
Data redundancy Self-information and Entropy Error-free compression – Huffman coding, Arithmetic coding, LZW coding, Run-length encoding – Predictive coding Lossy coding techniques – Predictive coing (Lossy) – Transform coding DCT, DFT, KLT, … JPEG image compression standard
Image Compression-II 39