Answers to Exercises A bird does not sing because he has an answer, he sings because he has a song. —Chinese Proverb Intro.1: abstemious, abstentious, adventitious, annelidous, arsenious, arterious, face- tious, sacrilegious. Intro.2: When a software house has a popular product they tend to come up with new versions. A user can update an old version to a new one, and the update usually comes as a compressed file on a floppy disk. Over time the updates get bigger and, at a certain point, an update may not fit on a single floppy. This is why good compression is important in the case of software updates. The time it takes to compress and decompress the update is unimportant since these operations are typically done just once. Recently, software makers have taken to providing updates over the Internet, but even in such cases it is important to have small files because of the download times involved. 1.1: (1) ask a question, (2) absolutely necessary, (3) advance warning, (4) boiling hot, (5) climb up, (6) close scrutiny, (7) exactly the same, (8) free gift, (9) hot water heater, (10) my personal opinion, (11) newborn baby, (12) postponed until later, (13) unexpected surprise, (14) unsolved mysteries. 1.2: A reasonable way to use them is to code the five most-common strings in the text. Because irreversible text compression is a special-purpose method, the user may know what strings are common in any particular text to be compressed. The user may specify five such strings to the encoder, and they should also be written at the start of the output stream, for the decoder’s use. 1.3: 6,8,0,1,3,1,4,1,3,1,4,1,3,1,4,1,3,1,2,2,2,2,6,1,1. The first two are the bitmap reso- lution (6×8). If each number occupies a byte on the output stream, then its size is 25 bytes, compared to a bitmap size of only 6 × 8 bits = 6 bytes. The method does not work for small images. 954 Answers to Exercises 1.4: RLE of images is based on the idea that adjacent pixels tend to be identical. The last pixel of a row, however, has no reason to be identical to the first pixel of the next row. 1.5: Each of the first four rows yields the eight runs 1,1,1,2,1,1,1,eol. Rows 6 and 8 yield the four runs 0,7,1,eol each. Rows 5 and 7 yield the two runs 8,eol each. The total number of runs (including the eol’s) is thus 44. When compressing by columns, columns 1, 3, and 6 yield the five runs 5,1,1,1,eol each. Columns 2, 4, 5, and 7 yield the six runs 0,5,1,1,1,eol each. Column 8 gives 4,4,eol, so the total number of runs is 42. This image is thus “balanced” with respect to rows and columns. 1.6: The result is five groups as follows: W1 to W2 :00000, 11111, W3 to W10 :00001, 00011, 00111, 01111, 11110, 11100, 11000, 10000, W11 to W22 :00010, 00100, 01000, 00110, 01100, 01110, 11101, 11011, 10111, 11001, 10011, 10001, W23 to W30 :01011, 10110, 01101, 11010, 10100, 01001, 10010, 00101, W31 to W32 :01010, 10101. 1.7: The seven codes are 0000, 1111, 0001, 1110, 0000, 0011, 1111, forming a string with six runs. Applying the rule of complementing yields the sequence 0000, 1111, 1110, 1110, 0000, 0011, 0000, with seven runs. The rule of complementing does not always reduce the number of runs. 1.8: As “11 22 90 00 00 33 44”. The 00 following the 90 indicates no run, and the following 00 is interpreted as a regular character. 1.9: The six characters “123ABC” have ASCII codes 31, 32, 33, 41, 42, and 43. Trans- lating these hexadecimal numbers to binary produces “00110001 00110010 00110011 01000001 01000010 01000011”. The next step is to divide this string of 48 bits into 6-bit blocks. They are 001100=12, 010011=19, 001000=8, 110011=51, 010000=16, 010100=20, 001001=9, and 000011=3. The character at position 12 in the BinHex table is “-” (position numbering starts at zero). The one at position 19 is “6”. The final result is the string “-6)c38*$”. 1.10: Exercise 2.1 shows that the binary code of the integer i is 1 + log2 i bits long. We add log2 i zeros, bringing the total size to 1 + 2 log2 i bits. Answers to Exercises 955 1.11: Table Ans.1 summarizes the results. In (a), the first string is encoded with k =1. In (b) it is encoded with k = 2. Columns (c) and (d) are the encodings of the second string with k = 1 and k = 2, respectively. The averages of the four columns are 3.4375, 3.25, 3.56, and 3.6875; very similar! The move-ahead-k method used with small values of k does not favor strings satisfying the concentration property. a abcdmnop 0 a abcdmnop 0 a abcdmnop 0 a abcdmnop 0 b abcdmnop 1 b abcdmnop 1 b abcdmnop 1 b abcdmnop 1 c bacdmnop 2 c bacdmnop 2 c bacdmnop 2 c bacdmnop 2 d bcadmnop 3 d cbadmnop 3 d bcadmnop 3 d cbadmnop 3 dbcdamnop2 d cdbamnop 1 mbcdamnop4 m cdbamnop 4 c bdcamnop 2 cdcbamnop1 nbcdmanop5 n cdmbanop 5 bbcdamnop0 b cdbamnop 2 o bcdmnaop 6 o cdmnbaop 6 abcdamnop3 abcdamnop3 p bcdmnoap 7 p cdmnobap 7 m bcadmnop 4 m bacdmnop 4 abcdmnopa7 a cdmnopba 7 n bcamdnop 5 n bamcdnop 5 b bcdmnoap 0 b cdmnoapb 7 o bcamndop 6 o bamncdop 6 c bcdmnoap 1 c cdmnobap 0 pbcamnodp7 pbamnocdp7 d cbdmnoap 2 d cdmnobap 1 pbcamnopd6 pbamnopcd5 m cdbmnoap 3 m dcmnobap 2 obcamnpod6 o bampnocd 5 n cdmbnoap 4 n mdcnobap 3 nbcamnopd4 n bamopncd 5 o cdmnboap 5 o mndcobap 4 mbcanmopd4 mbamnopcd2 p cdmnobap 7 p mnodcbap 7 bcamnopd mbanopcd cdmnobpa mnodcpba (a) (b) (c) (d) Table Ans.1: Encoding With Move-Ahead-k. 1.12: Table Ans.2 summarizes the decoding steps. Notice how similar it is to Ta- ble 1.16, indicating that move-to-front is a symmetric data compression method. Code input A (before adding) A (after adding) Word 0the () (the) the 1boy (the) (the, boy) boy 2on (boy, the) (boy, the, on) on 3my (on,boy,the) (on,boy,the,my) my 4right (my, on, boy, the) (my, on, boy, the, right) right 5is (right, my, on, boy, the) (right, my, on, boy, the, is) is 5 (is,right,my,on,boy,the) (is,right,my,on,boy,the) the 2 (the,is,right,my,on,boy) (the,is,right,my,on,boy) right 5 (right, the, is, my, on, boy) (right, the, is, my, on, boy) boy (boy,right,the,is,my,on) Table Ans.2: Decoding Multiple-Letter Words. 956 Answers to Exercises 2.1: It is 1 + log2 i as can be seen by simple experimenting. 2.2: The integer 2 is the smallest integer that can serve as the basis for a number system. ≈ 2.3: Replacing 10 by 3 we get x = k log2 3 1.58k. A trit is therefore worth about 1.58 bits. 2.4: We assume an alphabet with two symbols a1 and a2, with probabilities P1 and − − − P2, respectively. Since P1 + P2 = 1, the entropy of the alphabet is P1 log2 P1 (1 − P1)log2(1 P1). Table Ans.3 shows the entropies for certain values of the probabilities. When P1 = P2, at least 1 bit is required to encode each symbol, reflecting the fact that the entropy is at its maximum, the redundancy is zero, and the data cannot be compressed. However, when the probabilities are very different, the minimum number of bits required per symbol drops significantly. We may not be able to develop a com- pression method using 0.08 bits per symbol but we know that when P1 = 99%, this is the theoretical minimum. P1 P2 Entropy 99 1 0.08 90 10 0.47 80 20 0.72 70 30 0.88 60 40 0.97 50 50 1.00 Table Ans.3: Probabilities and Entropies of Two Symbols. An essential tool of this theory [information] is a quantity for measuring the amount of information conveyed by a message. Suppose a message is encoded into some long number. To quantify the information content of this message, Shannon proposed to count the number of its digits. According to this criterion, 3.14159, for example, conveys twice as much information as 3.14, and six times as much as 3. Struck by the similarity between this recipe and the famous equation on Boltzman’s tomb (entropy is the number of digits of probability), Shannon called his formula the “information entropy.” Hans Christian von Baeyer, Maxwell’s Demon (1998) 2.5: It is easy to see that the unary code satisfies the prefix property, so it definitely can be used as a variable-size code. Since its length L satisfies L = n we get 2−L =2−n,soit makes sense to use it in cases were the input data consists of integers n with probabilities P (n) ≈ 2−n. If the data lends itself to the use of the unary code, the entire Huffman algorithm can be skipped, and the codes of all the symbols can easily and quickly be constructed before compression or decompression starts.