Average-case Analysis and Lower Bounds by the Incompressibility Method
Tao Jiang Dept of Computer Science Univ. of California - Riverside
Joint work with Ming Li (Waterloo) and Paul Vitanyi (CWI)
1 Outline
1. Overview of Kolmogorov complexity
2. The incompressibility method
3. Trivial example: lower bound for sorting
4. Average complexity of Shellsort
5. Average complexity of Heapsort
6. Average complexity of sorting with networks of stacks and queues
7. Average complexity of boolean matrix multiplication
8. Average complexity of majority finding
9. Expected length of longest common subsequence
10. Expected size of Heilbronn’s triangles
11. String matching and one-way pointers (heads)
2 Kolmogorov Complexity
1. Introduced by R. Solomonoff (1960), A.N. Kolmogorov (1965), P. Martin-Lof (1966), and G. Chaitin (1969).
2. Also known as descriptional complexity, algorith- mic information complexity, Solomonoff-Kolmogorov-Chaitin complexity.
3. It is concerned with the a priori probability, infor- mation content and randomness of an individual object.
010101010101... 011001010001...
A good introduction is: M. Li and P. Vitanyi, An Introduction to Kolmogorov Complexity and Its Application, Springer-Verlag.
3
Kolmogorov Complexity of an Object
¡£¢ ¥¤ Def. The Kolmogorov complexity of object , denoted , is the length of the shortest program that prints .
What kind of programs? Pascal, C, Java, or even some pseudo-language?
Invariance Theorem. (Solomonoff’60) ¦ ©
For any two languages ¦¨§ and ,
¡ ¢ ¡ ¢
¤ ¥¤
for any , where is some constant depending only on ¦¨§ and ¦ © .
I.e., the language does not matter as long as it’s fixed.
Def. The conditional Kolmogorov complexity of relative to ,
¡£¢
¤
denoted , is the length of the shortest program that prints
when given as input.
Program description/encoding
4 ¡£¢ An Example — ¤
Program 1:
787:9<;
!#"%$'&)(+*-, ./..0,
1 243 5
6
9
(A@
> ? The length of description is = bits.
Program 2:
@ , =
For BDCFE to Print(“ ”);
9
(A@
= > ? The length is GIH¨J bits.
Are there shorter programs?
E KML N
Yes if, e.g., = for some integer .
9 (QP
In general, O is uncomputable.
5
Incompressibility of Objects
S R
Let be a finite set of objects and R .
¡£¢
¤UT VXWY
R Def. Object is -incompressible if R .
-incompressible objects are also said to be Kolmogorov ran- dom.
Incompressibility Lemma. There are at least
Z
`_
¢[Z Z
¤
R
\^]
-incompressible elements in R .
6 a VbWY
Proof. Let R . There are at most
]
dc c
§
\
e
]
dc
Z Z Z
R
\ \
a a a
\^] \^]
egfih programs of lengths less than 6j . Each program prints at most
one element of R . Q.E.D.
1. At least one element of R is -incompressible.
2. More than half are Z -incompressible. \
3. More than k ¢ Z 6 ¤ 6 VXWYn6 4. More than l are -incompressible. 6 The Incompressibility Method To prove a (lower or upper) bound: ¢ 6p¤ 1. Take an o -incompressible object to construct a “typi- cal” instance/input. 2. Establish the bound for this fixed instance . Show that if the bound does not hold for then we can compress by giving a clever, short encoding of . Remarks: (a) Such a typical instance q possesses all statistical properties; q makes a proof easy; q cannot be recursively constructed. (b) The result usually holds in the average/expected case since most objects are incompressible. 7 Success Stories of the Incompressibility Method 1. Solution of many longstanding open problems con- cerning the complexity of various models of com- putation. 2. Simpler proofs for many results in combinatorics, parallel computation, VLSI, formal language/automata theory, etc. 3. Average-case analysis of classical algorithms such as Heapsort, Shellsort, Boolean matrix multiplica- tion, etc. 8 Average-Case (Time) Complexity Analysis Assume all instances of size = occur with equal prob- ability. 926 Complexity 300 229 1 2 3 Size n Election 2000: The Florida recount 9 A Trivial Example: Lower Bound for Sorting Theorem. Any comparison based sorting algorithm requires r ¢ 6 6sVbWYt6p¤ comparisons to sort elements on the average. Proof. Let u be any comparison based sorting algorithm. Fix a Z ZAxzy{y{y{x 6}| w -incompressible permutation v of such that ¡£¢ x Z 6p¤UT VXWYn6-~ v u v v Suppose u sorts in comparisons. We can describe by listing L ¢Z q ¤ a description of this encoding scheme in bits, q the binary outcomes of the comparisons in bits. L L _ ¢Z ¡£¢ x ¤UT 6p¤ v u Since , L ¡£¢ x ¢[Z ¢Z r ¢ T 6p¤- ¤T VXWYt6-~ ¤a 6sVbWY 6¤ v u L Since more than half of the permutations are Z -incompressible, the average number of comparisons required by u is r ¢ r ¢ \ a 6DVXWYn6p¤ 6£VXWYn6p¤ l 10 Average Complexity of Shellsort Algorithm p-pass Shellsort with increments y{y{y Z 0 a §n ; x{y{y{y{x a § 1. Input: list ; Z a 2. For to 3. Divide list into equally spaced sublists 6 (or chains), each of length l ; 4. Perform Insertion Sort within each sublist; h1 = 6 h2 = 3 h3 = 2 h4 = 1 11 Previous Results on Shellsort Worst case: ¢ © 6 ¤ VXWYn6 1. time in passes (Shell’59) ¢ © 64 ¤ 2. time (Papernov-Stasevitch’65) ¢ © © 6DVbW 3. time in passes (Pratt’72) ¢ ¢ §p © 6 ¤ ¤ VbWY 6 4. time in l passes (Incerpi and Sedgewick’85 and Chazelle’??) r ¢ §p 6 ¤ 5. time for passes (Plaxton, Poonen and Suel’92) Average case: ¢ \ 6¢ ¤ £ 1. ¡ time for -pass (Knuth’73) 2. Complicated analysis for k -pass (Yao’80) ¢ © § 6 ¢¤ ` k 3. for -pass (Janson and Knuth’96) Open: 1. Average complexity of general -pass Shellsort (Plaxton, Poonen and Suel’92; Sedgewick’96,97) ¢ 6£VXWYn6p¤ 2. Can Shellsort achieve on the average? (Sedgewick’97) 12 An Average Lower Bound for Shellsort VXWYt6 Theorem. For any , the average-case running time of r ¢ §¤ § 6 ¤ a -pass Shellsort is under uniform distribution of input permutations. Z r ¢ © a 6 ¤ 1. : is tight for Insertion Sort. r ¢ ¢ ¦{§¨ \ © ¢ 64 ¤ a 6 :©¤ ¥ 2. : vs of Knuth. r ¢ ¢ © § ¢ a 6)ª ¤ 6 ¤ 4 £ k 3. : vs of Janson and Knuth. ® ¢ r ¢Q¢ ¢ \¬«z a VXWYn6 6p¤ 6sVXWYt6p¤ 6¤Q¤ l0o l¯o 4. : . r ¢ VXWYn6 6p¤ 5. : is a trivial lower bound. ¢ 6DVbW Hence, in order for a -pass Shellsort to achieve time ¢ a VbW 13 Proving the Average Lower Bound ¢ r ¢ x{y{y{y{x §+§ ¤ 6 ¤ u Fix any § Shellsort . We prove is a lower bound on the average number of inversions. ° Z Z 6 Def. For any and , consider the -chain ° containing element at the beginning of pass . Let be the ° ° ±³² number of elements that are to the left of and largerL than . 1 2 3 4 5 6 7 8 9 10 11 12 5 9 1 6 10 2 7 11 3 8 12 4 h3 e.g. m8,3 = 2 f ´ Fact. The Insertion Sort in pass makes precisely § _ L ±³² Z ± f ´ inversions (or § comparisons). L ±³² ± Let µ denote the total number of inversions: a µ ±³² L f f § § ± 14 Proving the Average Lower Bound – II ZAxzy{y{y{x 6}| VXWYn6 w Fix a -incompressible permutation of with ¡£¢ x 6 ¤jT VXWYt6-~ VXWYn6 u Fact. Given all the numbers ’s for the input permutation , L ±³² we can uniquely reconstruct . Proof. Reconstruct the initial permutation of each pass back- ward. Hence, ¡£¢ x{y{y{y{x x ¡£¢ x 6 ¤jT 6 ¤jT VXWYn6-~ VXWYn6 § § u u ² ² L L c § 6 ¶F· µ c Since there are possible divisions of into non- § ¸ negative integral summands ’s, L ±¹² _ Z 6 _ xzy{y{y{x ¡£¢ x µ VbWY VbWY º T 6 ¤jT VXWYn6-~¼½VXWYt6 § § µ u Z » 6 ² ² L L ¢ VbW Noting µ , a careful calculation shows r ¢ § a 6 ¤ ¾ µ The average number of inversions required is thus: Z Z 6 _ r ¢ r ¢ §¤ §¤ 6 ¤ a 6 ¤ ¾ ¾ 6 ¿ 6 ¿ 15 The Average Complexity of Heapsort @ÃÂF Á =¨Ä Algorithm Heapsort(var À ); 1. Heapify À ; @ = 2. For BÅCFE to ÂF Á¹BÄÆC%E À ÁÇB =ÈÄ À DELETEMIN( ); 9 ( = Fact. Heapify requires ? time. 9 ( = GIH¨J = Fact. Heapsort runs in ? time in the worst case. 16 Two Implementations of DELETEMIN ZAyÊy 6)Ë̤ Williams’ DELETEMIN( uDÉ ; Z a 6)Ë a uDÉ 1. ; ; _ ¢ x Z \ \ Ë Ë̤ Í Î8Ï uDÉ 2. While uDÉ _ ¢ x Z \ \ Ë a Ë Ë̤ Í Î8Ï uDÉ uDÉ 3. uDÉÐ ; //shift up// _ Z \ \ a a 4. or ; //move down// Ë a 5. uDÉÐ ; ZAyÊy 6<Ë̤ Floyd’s DELETEMIN( uDÉ : 1. Find the path leading to a leaf while shifting all elements up; 2. Climb up the path and insert at its correct location. a1 a2 a2 a3 DELETEMIN a l-1 al al x x Fact. Williams requires \Ñ comparisons and Floyd \ Ñ requires VbW 17 Avg Analysis of Williams and Floyd Heapsorts \ Theorem. Both Williams and Floyd Heapsorts require 6DVbWY 6 comparisons in the worst case. Theorem. (Schaffer and Sedgewick’92) ¢ \ 6DVbW On the average, Williams Heapsort requires _ ¢ 6¤ 6£VXWYn6 comparisons and Floyd Heapsort requires com- parisons. The following is a simple incompressibility proof due to Ian Munro. ZAx{y{y{y{x VXWYn6 6}| w Fix a -incompressible input permutation of with ¡£¢ 6¤T VbWY 6+~ VXWYt6 Denote the resulting heap of Heapify as Ò . ¡£¢ x ¢ 6 ¤na 6p¤ Ò Fact. x ¢ ¡£¢ ¡£¢ ¡£¢ 6 ¤jT VXWYt6-~ 6p¤ 6p¤- 6p¤jT Ò Hence, Ò . 18 Insertion Depths are the Key! 1 DELETEMIN 2 DELETEMIN 3 DELETEMIN ...... n-1 DELETEMIN x1 x2 x1 x2 x3 n n H1 = H H2 H3 Hn-1 Hn d(x1 )=l1 d(x 2 )=l2 d(xn-2 )=ln-2 ln-1 =0 dc \ÓÑ § f ´ Claim. Williams needs § comparisons. ± ± Claim. The number of Floyd’s comparisons is dc dc § § ° ¢ Z ¢ \ \ Ñ Ñ a VXWY 6 ¤~ VXWY 6 ¤ ± ± f f § § ± ± dc ¡£¢ Ñ § 6p¤ T f Ò ´ Claim. § ± ± Ò Ò Proof. Given the location of in § , we can recover from ± ± ± Ò § . The location can be described by the path from the root ± Ñ Ò of +§ to , in bits. ± ± ± 19 The Final Calculation Therefore, Williams Heapsort requires at least ¡£¢ ¢ ¢ \ \ \ 6p¤T VbWY 6+~ 6p¤nÔ 6DVXWYt6 6p¤ Ò comparisons and Floyd Heapsort requires at most _ _ ¢ ¢ Z ¡Å¢ ¢ \ 6¤ VXWY 6 ¤Õ~ 6p¤tÔ 6DVbWY 6 6p¤ VXWYn6-~ Ò VXWYt6 comparisons on -incompressible permutation . On the average, Williams and Floyd Heapsorts need Z Z 6 _ ¢ ¢ ¢ \ \ 6sVXWYn6 6p¤#¤ 6 a 6sVXWYt6 6p¤ 6 ¿ 6 ¿ Z Z 6 _ _ _ ¢ ¢ ¢ ¢ \ 6DVXWYt6 6p¤#¤ 6sVbW 6 ¿ 6 ¿ comparisons, respectively. _ Theorem. Both Williams and Floyd Heapsorts require 6£VXWYn6 ¢ 6p¤ data moves on the average. 20 Sorting with Networks of Stacks and Queues Knuth and Tarjan studied the problem in the early 70s. The main question is: assuming the stacks or queues are ar- ranged sequentially or in parallel, how many stacks or queues are needed to sort 6 elements with comparisons only? Here, the input sequence is scanned from left to right and the elements follow the arrows to go to the next stack or queue or output. Sorted output Input Permutation A sequence of stacks or queues Sorted Input Permutation output stacks/queues 21 Sorting with Sequential Stacks § VXWYn6 Fact. In the worst case, VXWYn6 stacks suffice, and stacks © are necessary (Tarjan’72). § Theorem. On the average, at least VXWYn6 stacks are needed © for sequential stack sort. VbWYt6 Proof. Fix a -incompressible permutation . Assume 6 stacks are sufficient to sort . Since exactly elements pass through each stack, we can encode the sequence of pushes and Z 6 pops on a stack uniquely as a binary string of 6 ’s and ’s. \ 6 Hence, the input permutation is described by bits. There- fore, ¢ \ 6 T VXWYn6-~ VXWYn6 a 6DVbW § T VbW and approximately . © 22 Sorting with Parallel Stacks x x x{y{y{y{x x[Z Z \ 6 6 m Fact. The permutation k requires parallel stacks. We show that on the average, the number of parallel stacks ¢ Ö 6 6p¤ needed to sort elements is ¡ . The bound is implied by the connection between sorting with parallel stacks and longest increasing subsequences (Tarjan’72) and the bounds on the length of longest increasing subsequences of random permutations given in Kingman’73, Logan and Shepp’77, and Kerov and Versik’77), using deep results from probability theory (such as Kingman’s ergodic theorem). 23 An Average-Case Upper Bound ¢ Ö 6p¤ Theorem. On the average, parallel stacks are needed to sort 6 elements. x{y{y{y{x a VbWY 6 Proof. Fix a -incompressible permutation § . x x{y{y{y h Rק The stacks are named R . We use the following triv- ial algorithm (Tarjan’72) to sort . Here, the stacks are named x{y{y{y x h § R R . Algorithm Parallel-Stack-Sort ° Z 6 1. For a to do Scan the stacks from left to right, and push on the the ± first stack RÙØ whose top element is larger than . If ± such a stack doesn’t exist, put on the first empty stack. ± 2. Pop the stacks in the ascending order of their top elements. 24 An Average-Case Upper Bound ¢ÚÖ 6p¤ We need show that algorithm Parallel-Stack-Sort uses stacks on the permutation . Claim. If the algorithm uses stacks on then we can identify L an increasing subsequence of of length (Tarjan’72). L Claim. The permutation has no increasing subsequence of Ö ¦ ¦ length longer than 6 , where is the natural constant. Proof. Suppose that Û is a longest increasing subsequence of a and Û . We can encode by: L ¢Z ¤ 1. a description of this encoding scheme in bits; 2. the number in VXWY bits; L L VXWY ¶ Û 3. as a combination in Ü bits; ¸ VXWY Û ¶ 4. the locations of the elements of in in at most Ü ¸ bits; and ¢ VXWY 6 Û 5. the remaining with the elements of deleted in ¤Õ~ bits. L 25 An Average-Case Upper Bound This description takes a total of 6+~ _ _ _ _ ¢ ¢Z \ \ VbWY 6n ¤Õ~ VbWY VXWY ¤ VbWYÝVbWY ¢ ~ 6 ¤Õ~ L L L L L Ö ¢ a Þ 6¤ bits. Using Stirling approximation and the fact 6 , we upper bound the above expression as: L ¢ ¦ 6 ¤ _ l VXWYn6-~ VXWY ¢ ¢Q¢ ¦ ¦ dc Ü Ü ¤ 6 ¤ ¤ © l l L L _ ¢ VXWYn6p¤ 6 6 _ _ ¢ 6 ¤}VXWY Ô VXWYn6-~ VXWY 6 © L L _ _ L L ¢ ¦ VbWY VXWYn6p¤ L 6 _ _ _ ¢ ¦ \ Ô VXWYn6-~ VXWY VXWY VbW © L L L VbWYß6 Since this must exceed VXWYn6-~ , we obtain (approximately) ¢ÚÖ Ö ¦ 6 a 6¤ . L 26 An Average-Case Lower Bound r ¢ Ö Theorem. On the average, 6¤ parallel stacks are required to sort a permutation. Proof. Let u be a sorting algorithm using parallel stacks. Again, VbW u ¥ fix a -incompressible permutation . Suppose uses parallel stacks to sort . We encode the sequence of moves in the sorting process as a sequence of the following terms: ° q push to stack , q pop stack à , where the element to be pushed is the next unprocessed element from the input sequence and the popped element is written as the next output element. VbWY Each of these terms requires ¥ bits, and totally we use pre- \ 6 cisely terms. Thus we have a (unique) description of in ¢ \ 6£VXWY 6£VXWYn6 VbWYt6p¤ T ¥ bits, which must exceed . So, ¥ r ¢ Ö Ö a 6p¤ 6 . 27 Sorting with Parallel Queues Sorting cannot be done with a sequence of queues. x ZAx{y{y{y{x[Z 6 6 Fact. The permutation 6 requires queues to sort (Tarjan’72). ¢ÚÖ 6p¤ Theorem. On the average, ¡ parallel queues are needed to sort 6 elements. The bound is implied by the connection between sorting with par- allel queues and longest decreasing subsequences (Tarjan’72) and the bounds in Kingman’73, Logan and Shepp’77, and Kerov and Versik’77, with sophisticated proofs. We can prove the above result using simple incompressibility arguments similar to the case of parallel stacks. 28 Average Complexity of Boolean Matrix Multiplication ¢ ¨ 6 á 6 a ¤ u Problem: Multiply two boolean matrices Ø and ±³² ¢ã â a ¤ Ø . ±³² Previous results: ¢ ©zä 6 å4æ[¤ 1. Best worst-case time complexity due to Copper- smith and Winograd, 1987. 2. In 1973, O’Neil and O’Neil gave a simple algorithm de- ¢ 6¤ scribed in next slide that runs in time in the worst ¢ © 6 ¤ case but achieves an average time complexity of . Here we analyze the average-case complexity of the algorithm using a simple incompressibility argument. 29 The Algorithm of O’Neil and O’Neil x â Algorithm QuickMultiply( u ) ¡ ¢ â a ¤ u 1. Let Ø denote the result of multiplying and . ±³² ° Z a 6 2. For to do Z ¨ a Ü à¯§nç ç à 3. Let be the indices with Øéè , ¿{¿{¿ ±³² Z . L Z 6 a 4. For à to do ã x{y{y{y{xã Z Ø Øëê Ø 5. Search the list Ø sequentially for a bit . ² ² Z Z a a Ø 6. Set Ø if a bit is found, or otherwise. ±³² ±³² â Theorem. Suppose that the elements of u and are drawn uni- ¢ © 6 ¤ formly and independently. Algorithm QuickMultiply runs in time on the average. 30 The Average Complexity of QuickMultiply Let 6 be a sufficiently large integer. The average time of Quick- ¢ ¢ © 6 ¤ 6d¤ Multiply is trivially bounded between and . Since ¢ Z \ \ © © 6 6 ¤ 6 á 6 l of the pairs of boolean matrices are VbWY 6 VbWY 6 -incompressible, it suffices to consider -incompressible boolean matrices. \ © 6 Take a VXWYn6 -incompressible binary string of length , and â 6 á 6 form two boolean matrices u and straightforwardly. We ¢ â © 6 ¤ u show that QuickMultiply spends time on and . ° °Ã Z Consider an arbitrary , where 6 . It suffices to show that the 6 sequential searches done in Steps 4 – 6 of QuickMultiply ¢ 6¤ take a total of time. 31 The Average Complexity of QuickMultiply By the statistical properties of incompressible strings, we know that at least Ö Z ¢ \ 6 6sVXWYt6p¤ l 1. of the searches find a in the first step, Ö ¢ Z 6 6DVbWY 6פ 2. l[m searches find a in two steps, Ö ¢ Z 6 6DVbW so on. VXWYn6 Claim. Each of the searches takes at most m steps. Hence, the 6 searches take at most a total of ì ¢ ¢Qí ¢ \ 6DVXWYn6p¤#¤ ¤ 6 l ¿ f § _ ¢ ¢ ¢ í VXWYt6p¤ 6sVXWYt6p¤ VbW m ¿ ¿ ì _ ¢ ¢ í \ © 6 VXWY 6 6DVbWY 6פ ç l f § _ ¢ ¢ í © a 6p¤ 6DVXWYn6p¤ VbW ¢ a 6p¤ steps. 32 The Average Complexity of QuickMultiply VXWYn6 Claim. Each of the searches takes at most m steps. Z ã 6 a a à à Ø Proof. Suppose that for some , , Ø ¿{¿{¿ ² ã a ðòñëó Ø Øéî%ï . Then we can encode as ² 1. A description of the above discussion. 2. The value of ° . 3. The value of à . ã x{y{y{y{xã Ø Ø Ø Ø ðòñëó 4. All bits of except the bits î%ï . ² ² This encoding takes at most _ _ _ ¢[Z ¢ \ \ © ¤ VXWYn6 6 VbW m \ © 6 VXWYn6 ç bits for sufficiently large 6 , which contradicts the assumption that VXWYt6 is -incompressible. 33 Average Complexity of Finding the Majority a Let § be a binary string. The majority of is the bit ¿{¿{¿ Z \Aõ 6 l ( or ) that appears more than ô times in . Previous results: 1. Saks,Werman’91; Alonso,Reingold,Schott’93: ¢ ö 6¤ 6 comparisons are necessary and sufficient in the worst case. 2. Alonso,Reingold,Schott’97: On the average, finding the majority requires at most _ ¢ \ í 6 VbW l0÷A l©k comparisons and at least _ ¢[Z \ í 6 6 ¤ l0÷A ¡ l©k comparisons. Here we prove an average-case upper bound tight up to the first major term, using a simple incompressibility argument. 34 Average-case Upper Bound a Algorithm Tournament( § ) ¿{¿{¿ Z 6 a 1. If then return § as the majority. \ a 2. Elseif 6 then a © § 3. If § then return as the majority. 4. Else return “no majority”. 6 a 5. Elseif k then a © § 6. If § then return as the majority. 7. Else return as the majority. a 8. Let . ° Z \^õ a 6 ô l 9. For to do a c © § © © 10. If then append the bit to . ± ± ± \Aõ 6 6 l 11. If is odd and ô is even then append the bit to . 12. Call Tournament( ). Theorem. On the average, algorithm Tournament requires at Ö _ ¢ \ 6 6DVXWYn6p¤ most l¯k comparisons. 35 Average Complexity of Algorithm Tournament VXWYn6 a Fix a -incompressible string § : ¿{¿{¿ ¡£¢ xø 6 W×ù¥ú ¤jT 6 VbW ÏAû¬Í ü^Ïý ¢ ¤ 6 For any , Û denotes the complexity of Tournament on L L any VXWYn6 -incompressible string of length . L Lemma. (Li,Vitanyi’93) a Ü þ þ þ Let § be a -incompressible binary string. Among the ¿{¿{¿ ¢ ¢ Ö x x{y{y{y{xQ¢ x \ ¤ ¤ ¤ c Ü Ü l þ ldm ¢ þ þ þ § © ©ÿ § ©Õÿ ©¡ ©¡ pairs , L L areL complementary. So, the new string obtained in Tournament is at most Ö _ ¢ 6 6£VXWYn6p¤ ldm bits long. VXWYn6 Lemma. is also -incompressible. Hence, we have recurrence relation: _ _ ¢ ¢ ¢ í \^õ ¤ VXWYn6p¤#¤ Û ldm Û ô l L L L L Ö _ ¢ ¢ \ 6p¤ 6DVXWYn6p¤ 6 Û A trivial expansion gives l¯k . The average complexity of Tournament is thus: Z Z 6 _ _ _ ¢ ¢ ¢ í í \ \ 6 6£VXWYn6¤#¤ 6 a 6 6£VXWYn6p¤ l¯k l¯k 6 6 36 Expected Length of Longest Common Subsequence y{y{y y#y{y § § § a a Ü þ þ þ § Given two sequences (i.e. strings) § and , ° ° y{y{y § § a Ü § ç ç þ þ is a subsequence of if for some , Ø . ±¤£ § A longest common subsequence (LCS) of þ and is a longest § þ possible sequence ¥ that is a subsequence of both and . For example, 0011 is an LCS of 010101 and 000111. x[Z a | w For simplicity, assume the alphabet ¦ . Consider two random sequences drawn independently from the uniformly distributed space of all binary strings of length 6 . Tight bounds on the expected LCS length for such two random se- quences is a well-known open question in string combinatorics. y¨§ © yg \ 6 6 The best bounds are and k . 37 An Upper Bound on Expected Length of LCS _ yò © § ¢ Þ 6p¤ Theorem. The expected LCS length is at most 6 . Proof. Let 6 be a sufficiently large integer. The expected length of an LCS of two random sequences of length 6 is trivially bounded \ 6 6 between l and . By the Incompressibility Lemma, again it suffices to consider VXWYn6 -incompressible sequences. \ VbWY 6 6 Take a -incompressible string of length , and let þ and § be the first and second halves of respectively. Suppose that § a Ü ¥×§ ¥¥© ¥ þ þ string ¥ is an LCS of and . We re-encode ¿{¿{¿ § and with respect to ¥ as follows. Write a Ü Ü þ § ¥×§ ©¥¥© ¥ þ ¿{¿{¿ § § a Ü Ü ¥ ¥ ¥ § § © © ¿{¿{¿ Encode e e e ® ® ® Z Z Z ¢ ê ¤na þ þ ¥ ¿{¿{¿ e e e ® ® ® ¢ Z Z Z § § ê ¤ a ¥ ¿{¿{¿ 38 An Upper Bound on Expected Length of LCS Hence, the string can be described by the following information in the self-delimiting form: 1. A description of the above discussion. 2. The LCS ¥ . ¢ ¢ § § ¤ ¤ ¥ ¥ þ 3. The new encodings þ and of and . _ ¢Z ¢ ¤ ¤ þ ¥ Items 1 and 2 take bits. Since contains at least L Z ’s, By simple counting and Stirling approximation L 6 _ ¢Z ¡£¢ ¢ ¤ ¤Q¤ VXWY ° þ ¥ f Ü ± 6 6 _ ¢[Z VXWY ¤ \ L 6 _ _ ¢Z VXWYt6 VXWY ¤ L _ \ VXWYn6 6£VXWYn6 VXWY L L _ ¢ ¢ ¢Z 6 ¤iVbW L L The second step in the above derivation follows from the trivial \ T 6 fact that l . L 39 An Upper Bound on Expected Length of LCS Similarly, we have _ ¡Å¢ ¢ § \ ¤#¤ VbWYt6 6DVbWY 6 VXWY ¥ L L _ ¢[Z ¢ ¢ ¤ 6 ¤}VXWY 6 ¤ L L Hence, the above description requires a total size of _ _ ¢ ¢ ¢ y \ \ \ VXWYn6p¤ 6DVbW L L L L L ¡£¢ \ ¥¤UT 6 VXWYn6 a 6 l Let . Since , we have L _ _ ¢ \ \ \ 6 VXWYn6 VbWYt6p¤ 6sVXWYn6 VXWY L L L ¢ ¢ \ 6 ¤ VXWY 6 ¤ L L _ ¢ \ a VbWYt6p¤ 6 6 VXWY ¢Z ¢[Z \ 6 ¤}VXWY ¤ Dividing both sides by 6 , we obtain _ ¢[Z ¢Z ¢Z \ \ \ Þ ¤ VXWY ¤}VXWY ¤ yg © § ¢Z Þ ¤ Solving the inequality numerically, we get . Note: Baeza-Yates and Navarro improved the analysis and ob- ©¥ tained a slightly better upper of yg . 40 A Lower Bound on Expected Length of LCS Next we prove a lower bound on the expected length of an LCS of two random sequences of length 6 . The proof uses the follow- ing greedy algorithm for computing common subsequences (not necessarily the longest ones). x § § § a a þ þ©§ þ Algorithm Zero-Major( § ) ¿{¿{¿ ¿{¿{¿ a 1. Let ¥ be the empty string. ° Z Z a a à 2. Set and ; ° 6 6 3. Repeat steps 4–6 until or à : § a þ þ ¥ 4. If Ø then begin append bit to string ; ° ± °<_ _ ± Z Z a a à à ; end _ Z a a à à 5. Elseif þ then ± ° ° _ Z a 6. Else 7. Return string ¥ . 41 A Lower Bound on Expected Length of LCS § 6 Theorem. Given two random sequences þ and of length , the above algorithm Zero-Major produces a common subsequence Ö y © © © © © ¢ 6 6£VXWYn6p¤ ¥ of length at least . VXWYn6 Proof. Let 6 be a sufficiently large integer, and take a - § \ 6 incompressible string of length . Let þ and be the first and second halves of . § We encode þ and using information from the computation of § Zero-Major on strings þ and . Consider the comparisons made by Zero-Major in the order that they were made, and create a ¢ x § ¤ þ pair of strings and as follows. For every comparison Ø , ± Z § þ 1. If and Ø are complementary, we append a to . ± 2. Otherwise, append a bit to . Furthermore, if the preced- ¢ x § ¤ Ø ing comparison þ involves complementary bits, we ±¨! ! ° ° Z Z a append a bit to the string if or a bit if Z a à à . § When one string ( þ or ) is exhausted, we append the remaining part (say " ) of the other string to . 42 An Example of Zero-Major Encoding § a a Consider strings þ 1001101 and 0110100. Algorithm a Zero-Major produces a common subsequence ¥ 0010, by the following comparisons: s = 10 01101 comparisons *|**||*|* t = 01101 0 0 a a The above encoding scheme yields 101100101 and a 01100. Here, " 0. 43 Some Properties of the Coding Strings § þ It is easy to see that the strings and uniquely encode and _ ¢ ¢ Ñ Ñ \ ¤ ¤ a 6 and, . Since ¡Å¢ ¡£¢ ¢[Z \ \ ¤UT ¥¤ VXWYt6 T 6 VXWYt6 ¤ # k _ ¢Z ¡£¢ ¢ Ñ ¤ ¤ ¤ and , we have ¡Å¢ ¢ ¢[Z Ñ ¤UT ¤+ VbW k Similarly, we can obtain ¡£¢ ¢ ¢Z Ñ ¤T ¤- VXWYn6 ¤ k and ¡£¢ ¢ ¢Z Ñ ¤T ¤- VXWYn6 ¤ " " k where " is the string appended to at the end of the above encoding. 44 Proving the Lower Bound Let us estimate the length of the common subsequence ¥ . Let ¢ ¢ $ ¦% ¦ $ ¦% ¦ § Þ ¤ Þ ¤ þ þ þ and be the number of ’s contained in ¢ x ¢ § $ ¦&% ¦ $ ¦&% ¦ § Þ ¤ Þ ¤Q| Í Î8Ï ¥ w þ þ þ þ and . Clearly, contains ’s. § VXWYt6 Since both þ and are -incompressible, we know _ ¢#í ¢ ¢#í $ ¦% ¦ \ \ Þ ¤ 6 6 6sVbWYt6p¤ 6£VXWYn6p¤ þ l þ l _ ¢ ¢ ¢ í í $ ¦&% ¦ § \ \ 6 6DVXWYn6p¤ Þ ¤ 6 6DVXWYn6p¤ l þ l Ö ¢ 6DVbW Hence, " has at most ’s. Combining with the fact ¡£¢ ¢ ¢Z Ñ ¤T ¤+ VXWYn6 ¤ " k that " , we claim y ¢ ¢ í Ñ 6DVXWYn6p¤ ¤ " Ö ¢ ¢ ¢ Ñ Ñ 6DVbW " " Hence, has at most ’s. Since ¢ ¢ Ñ Ñ ¤ ¤ ¥ ¥ , we have a lower bound on : y ¢ ¢ ¢ í Ñ Ñ 6DVbW ¥ 45 Proving the Lower Bound On the other hand, since every bit in the string corresponds to a unique bit in the common subsequence ¥ , we have ¢ ¢ $ ¦% ¦ Ñ ¤T Þ ¤ ¥ þ ¡£¢ ¢ ¢Z Ñ \ ¤T ¤- VXWYn6 ¤ Since , y ¢ ¢ ¢ ¢Qí $ ¦% ¦ Ñ Ñ \ 6sVXWYt6p¤ ¤T Þ ¤T ¤ ¥ þ l Hence, _ ¢ ¢ ¢ y ¢ ¢ í í Ñ Ñ \ Ñ ¤T ¤ 6£VXWYn6p¤jT 6 6DVbW k ¥ That is, y'© © © © © ¢ ¢ ¢ í í Ñ \ 6sVXWYt6p¤tÔ 6 6DVXWYn6¤ ¤T 6 ¥ l©k 46 Heilbronn’s Triangles P Â/ P Z)( ( 6 Consider = points in the unit square in the 9 (QP ÂÂ/ P Z+( ( 6 plane. Denote by * the smallest area of any triangle formed by three points. Define 9 (QP Â/Â/ P Z)( ( 6 xÊy³yÊyÐx * E , -/. * 10 32 Question: (Heilbronn’1950) How large is * ? 47 History of Heilbronn’s Triangles ¢[Z © 6 ¤ l 1. Heilbronn’50: ? r ¢[Z © 6 ¤ 2. Erdos’50:˝ l Ö ¢Z ¢ Ñ 6 VbWY Þ546 ¤ l 3. Roth’51: Ö ¢[Z ¢ 6 VXWYt6p¤ l 4. Schmidt’72: h ¢Z §äX§ ¢ 6 ¤ l 5. Roth’72: ¢Z §äX§Ú§ 6 婤 l 6. Roth’72: ¢Z §äX§ © ª 6 ¤ l 7. Komlos´ ,Pintz,Szemeredi’81:´ r ¢ © VXWYn6 6 ¤ 8. Komlos´ ,Pintz,Szemeredi’82:´ l 9. Bertram-Kretzberg,Hofmeister,Lefmann’97: r ¢ © VXWYn6 6 ¤ l Theorem. (Jiang,Li,Vitanyi’98) x{y{y{y{x | § Assuming uniform distribution of points w , ¢ ¢ x{y{y{y{x ¢[Z 6 798 7 7=< 6 a > ¤#¤a 6 ¤ ];: § ¡ l 48 Proving the Lower Bound á 6 ? ? @ Consider a ? grid on the unit square, where , and distribute points at grid intersections. 1 2 3 K 1 2 3 K ¢ x ¤ Each point is fully described by a pair of coordinates , Z x ? where . 6 ¶BA There are ways to put points. ¸ x{y{y{y{x | § w Fix a C -incompressible distribution . © ¡£¢ x{y{y{y{x x ? 6 ¤T VXWY º § ? C » 6 49 Warm-up Lemma. No three points can be collinear, and thus the smallest r ¢[Z ¢ ¢ Z \ © ¤ ¤#¤ ? triangle area is l . Proof. Suppose points , Ø and are on a straight line. Ob- ± VXWY ? serve that given and Ø , we can describe in just bits. ± x{y{y{y{x So we can encode § by specifying ¢Z q ¤ this encoding scheme in bits; x x q VXWY D ? the locations of Ø in bits; and ± q 6 VXWY A k ¶ the distribution of the other points in dc bits. ¸ Altogether this description takes © _ _ ¢Z ? VbW º D ? » 6 k Z © _ ¢ ¢ ¢ ¦ ? 6 ¤ VXWY VXWY 6 ¤ a 6 ¤}VXWY k k k \ 6 k _ _ ¢Z VbWY ¤ D ? © _ _ ¢Z ? ¤ a VbW º k ? » 6 © ? VbW C ç » 6 x{y{y{y{x C Contradiction since § is -incompressible. 50 Warm-up Lemma. No two pebbles can be on the same (horizontal or ver- tical) grid line. Proof. Otherwise, we can form a description of the arrangement using © _ _ ¢Z ? VbWY ¤ VbW k ? » \ 6 Z © _ ¢ ¢ ¢ ? ¦ \ \ \ a 6 ¤}VXWY 6 ¤ VXWY VXWY 6 ¤ \ \ 6 _ _ ¢Z VbWY ¤ k ? © _ _ ¢Z ? \ a VbW ? » 6 © ? VbW C ç » 6 _ ¢Z \ 6 VbWY 6 T VXWY ¤ ? E F ? C With fixed and , we obtain , which is a contradiction. 51 r ¢[Z 6 ©¤ Proving Lower Bound l ¢ x{y{y{y{x r ¢[Z 6 ¤a 6 ¤ l Lemma. § . ¢ x x Z ¢ ¢Z 6 ¤ a 6p¤ a Þ 6¤ l¯o l Proof sketch. Suppose Ø . ¢ x x ± ¤ Ø Let Ø be the longest edge. Given , we can describe ¢ ¢ ± ± © VXWY 6¤Q¤ ? l¯o in bits. xi xj xk x{y{y{y#x Hence we can describe § by specifying ¢Z q ¤ this encoding scheme in bits; x x © ¢ q VXWY VXWY 6¤ ? o the locations of Ø in bits; ± q 6 VXWY A k ¶ the distribution of the other points in dc bits. ¸ VXWY A ¶ C This description is shorter than as in the previous ¸ ¢ 6p¤ 6 @ lemma since o . Q.E.D. Since most distributions are C -incompressible, r ¢Z 6 ¤ r ¢[Z l 6 798 7 7=< a a 6 ¤ ];: l \ 52 ¢Z 6d¤ l Proving Upper Bound ¢ ¢ ¢ \ 6p¤ ¤}VbW ç o Let o be a function. Fix an -incompressible x{y{y{y{x | § distribution w . © ¡£¢ x{y{y{y{x x ¢ ? 6 ¤T VXWY º 6¤ § ? o » 6 ¢ x{y{y{y{x ¢ ¢ 6 ¤a 6p¤ 6¤ § o l Lemma. . Divide the unit square into upper and lower regions, each con- \ \ 6 ¢ taining l points. ¢ \ \ 6 6 6 ¤ l Connect all l points to form upper lines. Claim. Every horizontal grid line in the lower region intersects r ¢ © ¤ 6 upper lines. 53 ¢Z 6 ¤ l Proving Upper Bound We can in fact strengthen the claim. Claim. Every horizontal grid line in the lower region intersects r ¢ © ¤ 6 upper lines which are sufficiently far from each other. ¢ ¢ x{y{yzy{x 6 4 6p¤ a ¤ Now let § . Then each intersection elimi- ¢ \ 4 6¤ nates ? grid intersections that can be used to place the \ 6 l points in the lower region. 2g(n) no points in \ 6 This allows us to give a compact description of the lower l points. 54 ¢Z 6d¤ l Proving Upper Bound x{y{y{y{x We describe the points § by specifying their - and - coordinates separately: © ¢Z ? ? G HJILK VbW º º ? ¢ » » 6 6 ¢Z q ¤ a description of this encoding scheme in bits; q VXWY A ¶ a description of the -coordinates in bits; ¸ q a description of the -coordinates of the points in the upper VXWY region, each using ? bits, in ascending order of their -coordinates; and q a description of the -coordinates in the lower region, each ¢Z r ¢ ¢ © VXWY 6 4 6p¤Q¤#¤ using ? bits, in ascending order of their -coordinates. Altogether this is at most _ _ _ ¢ ¢[Z r ¢ ¢ ¢Z ? \ © VXWY º 6£VXWY 6 ¤}VXWY 6 4 6p¤#¤Q¤ ¤ ? l » 6 ¢ 6p¤ VbWY A o ¶ Since this must be at least , we obtain ¸ ¢ ¢ ¢ 4 6¤ a 6p¤ 6 ¤ o l 55 ¢Z 6d¤ l Proving Upper Bound Finally, the expected upper bound is computed as: §äNM Z Z ¢ Z ì 6p¤ _ ¢ ¢ o 6 798 7 7=< ¤ a ¤ ];: \ ® «z 6 6 6 §äNM § äX§ © ª f ® «z § Z ¢ x a ¤ 6 ZAyëZ \ 9 (A@PO m E ? = since * for all distributions. Combining both lemmas, 9 (A@PO k ¦ ¦ § ¦RQ * E S = Note. The above constructions are only rough sketches. For more detailed (and accurate) constructions, see our paper in RSA. 56 k One-way Heads Cannot Do String Matching (A lower bound or impossibility result by the incompressibility method.) Joint work with M. Li. 57 String Matching (or Pattern Matching): P P T Given pattern and text T , decide if occurs in . Sometimes we also want to locate the occurrences of P in T . 1. One of the most important problems in computer science. 2. Hundreds of papers written. 3. Many efficient algorithms found. E.g., KMP, BM, KR. 58 Main features of the algorithms: 1. Linear time. 2. Constant space (i.e., multihead finite automaton). A two-way six-head finite automaton can do string matching in linear time (Galil and Seiferas, 1981) Note: KMP and BM are not constant-space. 3. No need to back up pointer in the text (e.g., KMP). 59 Question 1: Can we do string matching without back- ing up the pointers at all? Note that, the question makes sense only in comparison- based model. Question 2: (Galil and Seiferas, 1981) Can a one-way U -head finite automaton do string match- ing, for any U ? Previous results. (Gereb-Gr´ aus, Li, Yesha) (WV E K Negative for U . Our result. Negative for any U . 60 Model of Computation Fix a deterministic finite automaton X , @\[ ( (^] (`_a(cb/ \(cd e Y Z U E with one-way read- only heads ( U -DFA). $ x # y $ Q Phg X P Tif T accepts an input f if appears in . Assume the heads are non-sensing, i.e., X cannot detect if two heads meet. 61 A Bit of Kolmogorov Complexity 9 P (QP The Kolmogorov complexity (KC) of , j , is the size (in bits) of the smallest description of P . A binary string P is random if 9 k l lnm l lÊ9 (QP P ( P j ? G HJ l 9 P (QP j T KC of conditional to T , , is the size of the P smallest description of , given T . P is random relative to T if l 9 k l lom l l89 (QP P ( P j T ? G HJ 62 Fact 1. Most strings of a fixed length are random. P pJqsr Fact 2. If string E is random, then l 9 k l lam l lÊ9 ( ( P j q ptr q ? GIH¨J Fact 3. (Symmetry of Information) Up to a logarithmic additive term, 9um l 9 9vm l 9 (QP (QP ( ( P j j T E j T j T 9 l 9 9 l 9 (QP ( P ( (QP j > j T E j T > j T 63 Some Useful Lemmas P X P ( Def. T are two segments of the input. matches P and T if, at some time a head moves a step in while another head is in T and vice versa. x y Matching Lemma. (Li and Yesha) X Pxg E f Tyf Let accept w . Suppose l m 9 k l lam l l (QP P P j T GIH¨J w P Then X must match the pattern with some occur- P rence of in T . Otherwise X will accept a wrong input with all occur- P rences of in T being replaced by some non-random string. 64 Z \ f{T T f Def. Let z be a set of heads, and an input. X Z Suppose the leading head is at the last bit of T . \ T is conscious of z if there exists a that can make some head in z move before any head reaches the right endmarker $. $ y1 y2 $ ...... G X Otherwise, is unconscious of z . That is, some heads will blindly move to the right. 65 , e Moving Lemma. Let | be any constant and Pxg Tyf T f an input, where is a sufficiently long random l l T string. If some heads move T steps in while the others remain completely stationary, these heads will move unconsciously of the others until some reaches the right endmarker $. $ x # y x $ I.e., a random text is like a desert: if some heads move too much in it, they will get lost and move blindly to the right. 66 Theorem. X cannot do string matching. l l Pxg P P Z ( Z Z e \ T T f T T Input: f , where are random and l l P k . \ l l P P Either all heads shift out of in U steps or Moving @}O V E Lemma will apply (with | )! The former case is prohibited by Matching Lemma. Thus, some heads will move blindly to the right. The head reaches f first becomes dead. $ x # y1 x y2 $ What if some moving heads are near the copy of P ? 67 A More Sophisticated Construction. Choose a large = and Z \ m 9 ~ @ ( O E > B E U U > K K ° Z a l l P ../. P Z Ñ E T T E = Fix a random string , where , ° ° l l m ° k k k T E = = . P X Pxg P ./.. Z \ Ñ T E T T f Tyf Let T . Simulate on input m @ P until a head dies. At most U copies of are near some head. 7 P P ( Find an relatively random to T and, replace the m 7 7 @ P P X T U copies of with to get . Make sure ends 7 Pxg T f up in the same configuration on f . m @ Repeat above until U heads have died. 68 Proof of Moving Lemma Want: If some heads move steps in consciously while the others remain still, we can compress by at least bits, for µ depending only on and . ¢ x % ¤ Def. An þ -grouping is a partition of heads into groups such that 1. heads in each group are within % bits, 2. groups are separated by at least þ bits. r s 69 Local Compression Lemma. Z \ For every , there is validating next claim: 9 9 ( ( ÂÂ/ Z Z ( ( ( \ 4 z E z Claim. Suppose z is a - ° p grouping of some heads at time . Let be the sub- ° \ z string of length to the right of . If the heads not ° z in z remain stationary till a head in some finishes ° p , then ../. Z 4 p p m 4 3 @ for some of size at most K , depending only X on the configuration of at time . γ 2 γ 2 |Q| |Q| u1 ... u g ... G 1 G g 4 Roughly, we can predict p#`p and compress it by m ' W m i 4 } bits. l l+ l l ] If z for all , and . 70 Great Compression Lemma. (#( 4 z z Consider a grouping z . Suppose k the groups are at least bits apart, for some z z . If the heads not in remain stationary in the next steps, we can compress the portions of input scanned by the heads in z during the steps by at L least bits, given initial positions of the heads. 2S 2S ...... G 1 G 2 ¡L The proof is by induction on , where L£¢ ¤ ¤ ¥ ¥ ¥ ¥¦¨§ª© ©¬« ¥¦¨§© ©¬«x® © © ¡¯ ¥ Note, decreases if some decreases in size or splits into more smaller groups. 71 c± ² Claim. The lemma holds when ° . 2S 2S 2S ...... G 1 G 2 G g ± ³ Proof by induction on ³ . ± µ Case 1: During the next steps, some head ´ pauses for ¶c steps. #¶^ · Consider the moving heads for steps. Case 2: No head pause for #¶^ steps. ª¹ #¶^P¼ ¸ ¸} ³»º ³ ³ Do local compression (with ) ³»º times to disjoint sections. #¶^ ³ (once every ³'º steps) 72 Inductive Step. ± ¶c½ Case 1: Some pauses for steps during the next steps. ± Case 2: Some splits, i.e., at a time two adjacent ± ¾ #¶^½ heads in are bits apart. Case 3: Neither Case 1 nor Case 2. 73 Case 3: Every group moves at least once in every #¶^½ steps and no group split. ± The heads in are always within distance ± Ài #¶^½ ¿ ¶c½ ³ ³ Dynamically partition input into sections of at most #¶^½ bits, and compress each -tuple of sections by at least bit. S 1/3 ... G 1 G g #¶^½ We can compress at least -tuples. 74 Two Subcases of A g-tuple ± Subcase 1: Some split by a large constant. Record head positions and compress recursively for steps. S 1 d 2 S 2 ... G 1 G 2 How to compensate for the overhead ' '¾ à '¾ ÂÁ Á }Ä ± Subcase 2: Heads in each are within a constant distance. 75 Open problem: What if the heads are sensing? The current Moving Lemma doesn’t hold since with Å heads one can easily make some heads moving con- sciously for a long time. 76 Concluding Remarks 1. We have demonstrated the usefulness and sim- plicity of the incompressibility method. 2. It seems that the method has advantage over other methods (such as the probabilistic method) espe- cially when dealing with algorithmic problems. 3. We expect more applications to come. ÈÇ '¾ Ç 4. Can Shellsort achieve Æ on the aver- age? 5. Give a simple analysis of Quicksort by incompress- ibility. 77 References 1. M. Li and P. Vitanyi. An Introduction to Kolmogorov Com- plexity and its Applications. Springer-Verlag, New York, 2nd Edition, 1997. 2. T. Jiang and M. Li. k one-way heads cannot do string- matching. Journal of Computer and System Sciences, 1996. 3. T. Jiang, J. Seiferas and P. Vitanyi. Two heads are better than two tapes. Journal of the ACM, 1997. 4. T. Jiang, M. Li, and P. Vitanyi. New applications of the in- compressibility method. Computer Journal, 1999. 5. T. Jiang, M. Li and P. Vitanyi. A lower bound on the average- case complexity of Shellsort. Journal of the ACM, 2000. 6. H. Buhrman, T. Jiang, M. Li, and P. Vitanyi. New applica- tions of the incompressibility method: Part II. Theoretical Computer Science, 2000. 7. T. Jiang, M. Li and P. Vitanyi. Average-case analysis of algorithms using Kolmogorov complexity. Journal of Com- puter Science and Technology, 2000. 8. T. Jiang, M. Li and P. Vitanyi. The average-case area of Heilbronn-type triangles. Random Structures and Algorithms, 2002. 78