Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Shujun LI (李树钧): INF-10845-20091 Multimedia Coding

Outline

z Review Lab Session 4 z Manual Exercises • Comparing coding performance of different : Shannon , Shannon-Fano code, Huffman code (and Tunstall code *) z MATLAB Exercises • Working with Huffman @ MATLAB May 18, 2009 • Compressing an image with the Huffman encoder • Writing your own Huffman codec

• Manipulating the colormap to add pseudo-color to UV channels • Testing Y4M support

1

Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Shujun LI (李树钧): INF-10845-20091 Multimedia Coding

Image and video encoding: A big picture Differential Coding and Compensation A/D Conversion Context-Based Coding Conversion … Pre-Filtering Predictive Partitioning Coding Review … Input Image/Video Post- Pre- Lossy Lossless Processing Processing Coding Coding (Post-filtering)

Quantization Entropy Coding Dictionary-Based Coding Model-Based Coding Run-Length Coding Encoded … … Image/Video

3

1 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Shujun LI (李树钧): INF-10845-20091 Multimedia Coding

The ingredients of entropy coding FLC, VLC and V2FLC

z A random source (X, P) z FLC = Fixed-length coding/code(s)/codeword(s) • Each symbol xi emitted from a random source (X, P) is encoded as an z A statistical model (X, P’) as an estimation n-bit codeword, where |X|≤2n. of the random source z VLC = Variable-length coding/code(s)/codeword(s) • Each symbol x emitted from a random source (X, P) is encoded as an z An algorithm to optimize the coding i ni-bit codeword.

performance (i.e., to minimize the average • FLC can be considered as a special case of VLC, where n1=…=n|X|. codeword length) z V2FLC = Variable-to-fixed length coding/code(s)/codeword(s) • A symbol or a string of symbols is encoded as an n-bit codeword. • V2FLC can also be considered as a special case of VLC. z At least one designer …

4 5

Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Shujun LI (李树钧): INF-10845-20091 Multimedia Coding

Static coding vs. Dynamic/Adaptive coding A coding

z Static coding = The statistical model P’ is static, z i.e., it does not change over time. z Shannon-Fano coding z z Dynamic/Adaptive coding = The statistical model z (Range coding) P’ is dynamically updated, i.e., it adapts itself to • Shannon-Fano-Elisa coding the context (i.e., changes over time). z Universal coding • Exp- (H.264/MPEG-4 AVC, Dirac) • Dynamic/Adaptive coding ⊂ Context-based coding • Elias coding family , Levenshtein coding , … z Hybrid coding = Static + Dynamic coding z Non-universal coding • Truncated binary coding, , … • A codebook is maintained at the encoder side, and the • Golomb coding ⊃ Rice coding encoder dynamically chooses a code for a number of z ⊂ V2FLC David Salomon, Variable-length Codes symbols and inform the decoder about the choice. z … for , Springer, 2007

6 7

2 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Shujun LI (李树钧): INF-10845-20091 Multimedia Coding

Shannon-Fano Code: An example Tunstall coding: An example *

z X={A,B,C,D,E}, z X={A,B,C}, P={0.7,0.2,0.1}, Y={0,1}, n=3 P={0.35,0.2,0.19,0.13,0.13}, Y={0,1} A Possible Code AAA 000 0.7 A Possible Code ABC AAB 001 0.35+0.2 0.19+0.13+0.13 0.2 0.1 A, B C, D, E A 00 AAC 010 0.49 B 01 ABC AB 011 0.13+0.13 0.14 0.07 0.35 0.2 0.19 D, E C 10 AC 100 A B C D 110 ABC B 101 0.343 0.098 0.049 0.13 0.13 E 111 C110 D E

8 9

Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Shujun LI (李树钧): INF-10845-20091 Multimedia Coding

Universal coding (code) Coding positive/non-negative integers

z A code is called universal if L≤C1(H+C2) for all possible z Naive binary coding values of H, where C , C ≥1. 1 2 • |X|=2k: k-bit binary representation of an integer. • You may see a different definition somewhere, but the basic idea remains the same – a universal code works like an optimal code, z Truncated binary coding except there is a bound defined by a constant C1. • |X|=2k+b: X={0,1,…,2k+b-1} z A universal code is called asymptotically optimal if C1→1 0 0 0 k k k 2 b 1 b0 bk 1 2 b b0 bk 2 + b 1 1 1 when H→∞. ⇒ ··· − − ⇒ //////////////////////··· − − ⇒ ··· − ⇒ ··· k k+1 z| {zUnary} code (Stone-age binary coding) | {z } • |X|=∞: X=Z+={1,2,…} f(x)=0 01or1 10 ··· ··· x 1 x 1 − −

10 | {z } | {z } 11

3 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Shujun LI (李树钧): INF-10845-20091 Multimedia Coding

Huffman’s rules of making optimal codes Huffman code: An example

z Source statistics: P=P0=[p1,…,pm], where p1≥…≥pm-1≥pm. z X={1,2,3,4,5}, P=[0.4,0.2,0.2,0.1,0.1]. z Rule 1: L1≤…≤Lm-1=Lm. p1+2+3+4+5=1 z Rule 2: If L1≤…≤Lm-2

12 13

Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Shujun LI (李树钧): INF-10845-20091 Multimedia Coding

Huffman code: An optimal code Huffman code: Small X problem

z Relation between Huffman code and Shannon code: z Problem H L L L

4 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Shujun LI (李树钧): INF-10845-20091 Multimedia Coding

Huffman code: Variance problem Modified Huffman code

z Problem z Problem • There are multiple choices of two smallest • If |X| is too large, the construction of the Huffman tree will be too long and the memory used for the tree will be too demanding. probabilities, if more than two nodes have the same probability during any step of the coding process. z Solution • Divide X into two set X ={s |p(s )>2-v}, X ={s |p(s )≤2-v}. • Huffman codes with a larger variance may cause 1 i i 2 i i • Perform Huffman coding for the new set X =X ∪{X }. trouble for data transmissions via a CBR (constant bit 3 1 2 • Append f(X ) as the prefix of naive binary representation of all rate) channel – a larger buffer is needed. 2 symbols in X2. z Solution • Shorter subtrees first. (A single node’s height is 0.)

16 17

Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Shujun LI (李树钧): INF-10845-20091 Multimedia Coding

Comparing different entropy codes

z X={A,B,C,D,E}, P=[0.38 0.18 0.15 0.15 0.14], Y={0,1} z Construct the following codes for this source: Manual Exercises Shannon code, Shannon-Fano code, Huffman code. z Compare the coding performance of the above three codes in term of average codeword length. z Extra exercise (*): Construct a Tunstall code (n=3) and see how the performance is.

19

5 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Shujun LI (李树钧): INF-10845-20091 Multimedia Coding

Huffman code for smaller X

z X={A,B}, P=[0.7,0.3], Y={0,1} z Construct a Huffman codes for the extended source (X2,P2) and (X3,P3). MATLAB Exercises z Compare the performance of them.

20

Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Shujun LI (李树钧): INF-10845-20091 Multimedia Coding

Working with Huffman codec Working with Huffman codec

z MATLAB Communications Toolbox includes an implementation of z MATLAB Communications Toolbox includes an implementation of Huffman codec: huffmandict, huffmanenco, huffmandeco Huffman codec: huffmandict, huffmanenco, huffmandeco

z Run the function huffmandict to verify the Huffman codes you got in z Given X={A,B,C,D,E}, P=[0.38 0.18 0.15 0.15 0.14], generate a the manual exercises. random message of size 100 which does NOT satisfy the probability z Given X={A,B,C,D,E}, P=[0.38 0.18 0.15 0.15 0.14], generate a distribution. random message of size 100 which satisfies the probability z Encode the message with huffmanenco, calculate the compression rate distribution. and compare it with the average codeword length of the Huffman z Encode the message with huffmanenco, calculate the compression rate code. (without counting the memory for storing the Huffman code) and z Estimate the real probability distribution from the message itself and compare it with the average codeword length of the Huffman code. redo the above experiments. z Decode the encoded message with huffmandeco, and check if the decoding result is correct (using isequal function).

22 23

6 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Shujun LI (李树钧): INF-10845-20091 Multimedia Coding

Compressing images with Huffman code My implementation of entropy

z Step 1: read a RGB image from a file. z Five main functions: HuffmanCode, ShannonFanoCode, ShannonCode, PrefixFreeCode; UnaryCodec, TruncatedBinaryCodec, GolombCodec, z Step 2: transform the RGB image to 4:2:0 YUV format. RiceCodec, ExpGolombCodec; EntropyEncoder, EntropyDecoder_Cell, z Step 3: estimate the statistics of YUV channels. EntropyDecoder_BS; HuffmanFileSave, HuffmanFileLoad. z Step 4: construct Huffman codes for YUV channels, z Some auxiliary functions: EntropyCodeShow, EntropyCodeTree, respectively, and compress the whole YUV image (as a 1- AverageCodewordLength, EntropyCodeTreeDepth, Z2Zplus, Zplus2Z. D signal). z Interesting features • Compatibility with huffmandict function. z Step 5: calculate the compression ratios of different • More entropy codes are supported. channels and of the image as a whole (count the storage of • Speed optimization – much faster than MATLAB’s implementation. the Huffman codes, too). • Three variance modes for Huffman code: ‘min’, ‘max’ and ‘rand’. Saving/Loading compressed data into/from binary files. z Tip: use reshape function to transform each image plane to • a 1-D array for encoding and transform back for decoding. • Display of the binary tree representation and statistics of entropy code. • … 24 25

Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Shujun LI (李树钧): INF-10845-20091 Multimedia Coding

My implementation of entropy codecs My implementation of entropy codecs

z Exemplar code z Exemplar code • x=imread('Images/lena_gray.bmp'); % Read a 512X512 grayscale image. • code=ShannonCode(X,P,1); % Generate a Shannon code X=[0:255]; P=imhist(x); % Set statistics of the random source. code=HuffmanCode(X,P); % Construct a Huffman code. • code=ShannonFanoCode(X,P,1); % Generate a Shannon-Fano code EntropyCodeShow(code); % Display information about the code. • code=PrefixFreeCode(X,L,1); % Generate a according to the EntropyCodeTree(code,1); % Display the tree representation of the code. codeword lengths (L) x_e=EntropyEncoder(x,code); % Encode x to get a 512X512 cell array. • x_e=ExpGolombCodec(x,7); % Exp-Golomb encoder x_d=EntropyDecoder_Cell(x_e,code); % Decode the encoded cell array. x_d=ExpGolombCodec(x_e,7,'Decoder'); % Exp-Golomb decoder imshow(x_d,[0 255]); % Show the decoded image. • x_e2=[x_e{:}]; % Generate a 1-D bitstream from the encoded cell array. • x_e=GolombCodec(x,100); % Golomb encoder x_d2=EntropyDecoder_BS(x_e2,code); % Decode the bitstream. x_d=GolombCodec(x_e,100,'Decoder'); % Golomb decoder imshow(reshape(x_d2,size(x)), [0 255]); % Show the decoded image. • … • HuffmanFileSave('Lena.hfm',x); % Compress x and save it in a file. x2=HuffmanFileLoad('Lena.hfm'); % Uncompress the data in a file. imshow(x2,[0 255]); % Show the decoded image.

26 27

7 Shujun LI (李树钧): INF-10845-20091 Multimedia Coding Shujun LI (李树钧): INF-10845-20091 Multimedia Coding

Your implementation of Huffman codec... Some more tips

z Do NOT refer to existing MATLAB implementations. z Exemplar MATLAB codes about generating a Huffman table. • P=[0.49 0.21 0.21 0.09]; z You have to depend on cell array a lot to store elements of HuffmanTable=cell(1,4); % HuffmanTable={[],[],[],[]}; different sizes. HuffmanTree={1,2,3,4}; HuffmanTable{3}=[0 HuffmanTable{3}], HuffmanTable{4}=[1 HuffmanTable{3}]; z You may assume X={0,…,m-1} to simplify your P=[0.49 0.21 0.3], HuffmanTree{3}=[3 4], HuffmanTree(4)=[]; implementation. HuffmanTable{2}=[0 HuffmanTable{2}], HuffmanTable{3}=[1 HuffmanTable{3}], z You may not consider removing zero probabilities from HuffmanTable{4}=[1 HuffmanTable{4}]; the distribution. P=[0.49 0.51], HuffmanTree{2}=[2 3 4], HuffmanTree(3)=[]; z You may not consider the variance problem, which helps HuffmanTable{1}=[0 HuffmanTable{1}], HuffmanTable{2}=[1 HuffmanTable{2}], simplify your implementation. HuffmanTable{3}=[1 HuffmanTable{3}], z You may not consider optimizing your code to get a faster HuffmanTable{4}=[1 HuffmanTable{4}]; encoding/decoding process. P=[1], HuffmanTree{1}=[1,2,3,4], HuffmanTree(2)=[];

28 29

8