Collected Works of Philippe Flajolet

COLLECTED WORKS OF PHILIPPE FLAJOLET

Editorial Committee: HSIEN-KUEI HWANG BRUNO SALVY Institute of Statistical Science Algorithms Project Academia Sinica INRIA Rocquencourt Taipei 115 F-78153 Le Chesnay Taiwan France

ROBERT SEDGEWICK MICHÈLE SORIA Department of Computer Science Laboratoire d’Informatique Princeton University Université Pierre et Marie Curie Princeton, NJ 08540 F-75252 Paris cedex 05 USA France

WOJCIECH SZPANKOWSKI BRIGITTE VALLÉE Department of Computer Science Département d’Informatique Purdue University Université de Caen West Lafayette, Indiana 47907 F-14032 Caen Cedex USA France

MARK DANIEL WARD (General Editor) Department of Statistics Purdue University West Lafayette, Indiana 47907 USA

ISBN TBA ©Cambridge University Press 2012+ (print version) ©TBA 2012+ (e-version) COLLECTED WORKS OF PHILIPPE FLAJOLET

There will be several types of introductions, including an introduction to the entire series of books (written by Donald E. Knuth), and also introductions to each speciﬁc volume (written the editors of that volume). Contents

Chapter I. STRING ALGORITHMS 1

Introduction 1. TEXT ANALYSIS 3

Paper 2. PAPER74 9

Paper 3. PAPER 76 11

Paper 4. PAPER 191 13

Chapter II. INFORMATION THEORY 15

ANALYTIC INFORMATION THEORY 17 Analytic Information Theory 17 Preliminary Discussion 18 Minimax Redundancy for a Class of Sources 20 Minimax Redundancy for Memoryless Sources 21 Minimax Redundancy for Renewal Sources 23

Paper 5. PAPER 158 27

Paper 6. PAPER 173 29

Paper 7. PAPER SEMINAR 31

Paper 8. PAPER SEMINAR 33

Chapter III. DIGITAL TREES 35

THE DIGITAL TREE PROCESS 37 1. A central role in computer science 37 2. Digital trees in Philippe Flajolet’s works 39 3. Conclusion 43

Paper 9. PAPER 34 45

Chapter IV. MELLIN TRANSFORM 47

DR FLAJOLET’SELIXIROR MELLINTRANSFORMANDASYMPTOTICS 49 Mellin transform and fundamental strip 49 Symbolic analysis 50

iii iv CONTENTS

Fundamental result 51 Harmonic sums 52 Zigzag method 52 Average-case analysis of algorithms and harmonic sums 53 Exponentials in harmonic sums 54 Technical point 55 Oscillations 56 Related topics 57

Paper 10. PAPER 58 59

Chapter V. DIVIDE AND CONQUER 61

DIVIDE-AND-CONQUER RECURRENCESANDTHE MELLIN-PERRON FORMULA 63 1. Introduction 63 2. The basic technique 65 3. Concluding Remarks 69

Paper 11. PAPER 115 71

Chapter VI. COMMUNICATION PROTOCOLS 73

FLAJOLET’S WORKON TELECOMMUNICATION PROTOCOLSAND COLLISION RESOLUTION AGORITHMS 75 1. Introduction 75 2. Telecommunication Protocols, Aloha protocol 76 3. The tree collision resolution algorithm 77 4. The free access tree algorithm 83 5. Q-ary free access tree algorithm 87

BIBLIOGRAPHY 89

Paper 12. PAPER 49 91

BIBLIOGRAPHY 93

INDEX 103 Chapter I

STRING ALGORITHMS

INTRODUCTION 1 Text Analysis

Pierre Nicodème

List of articles. – (#74)[66] Deviations from Uniformity in Random Strings (1988), P. Flajolet, P. Kirschenhofer and R.F. Tichy – (#76)[67] Discrepancy of Sequences in Discrete Spaces (1989), P. Flajolet, P. Kirschenhofer and R.F. Tichy – (#151,#174)[166, 167] Motif statistics (1999)-(2002), P. Nicodème, B. Salvy and P. Flajolet – (#164)[63] Hidden Pattern Statistics (2001), P. Flajolet, Y. Guivarc’h, W. Sz- pankowski and B. Vallée. – (#191)[121] Hidden Word Statistics, (2006), P. Flajolet, W. Szpankowski and B. Vallée. Since the computing capability of computers developed in the sixties and the seventies, text analysis has been a field of subject either for searching tools that find positions of matches with a motif in a specific text or for counting occurrences of motifs in random texts by combinatorial or probabilistic methods. Counting methods and statistics often provide limit laws, under various probability source models for the texts, which allows the detection of exceptional behaviours. As a typical object of computer science, finite automata have been used both for searching and for statistical analysis. Many statistical questions about word statistics have been solved by three different methods, combinatorial analysis, automata, and probability analysis. Such statistical researches imply the consideration of two objects, the source under which the text is generated, and the type of motif considered; the latter may be a single word or a finite set of words, reduced if no word is factor of another word of the set, or not reduced in the contrary and more difficult case, or an infinite set as defined by a regular expression with stars, or a hidden word.

Deviations from Uniformity in Random Strings. The important contributions of Philippe Flajolet in text analysis have to be situated historically with respect to the previously mentionned developments; his work however goes also upon searching intrinsic properties of texts. In the article [66] “Deviations from Uniformity in Random Strings” (1988) 1, coauthored with P. Kirschenhofer and R.F. Tichy, he goes from (intrinsic) properties of normality of inﬁnite strings, a problem set up by E. Borel in 1908

1. The 1989 published article “Discrepancy of Sequences in Discrete Spaces”, although published later of the 1988 article, is obviously a preliminary and unaccomplished version of the 1988 article; we will therefore not discuss it.

3 4 1. TEXT ANALYSIS during his researches on measure theory, to the speed of convergence to uniformity for large sequences, a computer science problem. Normal numbers (E. Borel, 1908) are numbers such that any block of bits of a given size occurs with its natural probability (1/2k for blocks of length k) in their infinite binary representation. P. Flajolet and his coauthors [66] cope with the asymptotic number of occurrences of blocks when a random binary sequence built upon a uniform Bernoulli source is large, but not infinite, a totally unexplored subject by the time. They build first a de Bruijn graph [25] counting simultaneously the occurrences of all words and deduce from it a universal Markov chain; the latter posseses strong convergence properties when the size k of the blocks remains fixed while the size n of the sequences tends to infinity, but these properties do not allow to conclude when k tends to infinity and approaches log2(n). Next comes an analysis based on words counting where Philippe Flajolet’s influence is clear; the proofs are based on combinatorics of words “à la Guibas-Odlyzko”, very delicate asymptotic manipulations and a saddle-point like integral. Guibas-Odlyzko [130, 131] (1981) introduced the autocorrelation polynomial of a word, the correlation polynomial of two words, and the language parsing of a sequence with respect to occurrences of a pattern. A key lemma of the proof of the “Deviations from Uniformity” article [66] extends a result of Guibas and Odlyzko [129] (1978) and indicates that the relevant counting generating function has no poles inside a circle of integration z = 1 + for a suitable small | | . It is worth noting that the results of this article are optimal, proving that all words of size (1 ) log (n) occurs with probability one in a binary random sequence of − 2 length n. Considering Proposition IV.4 p. 274. in Flajolet and Sedgewick book [114] and using bootstrapping as in Fayolle [40] (2004) should open the way to a general- ization of the result to alphabets of any size. As a consequence, the fill-up level of a suffix-tree built from an unbiased source upon a sequence of length n is likely to be (1 ) log (n) for an alphabet of cardinality α. Future work could study more − α general sources; in particular the study of a general notion of discrepancy for biased sources should be compared with Knessl and Szpankowski study [148] (2004) of the fill-up level in tries generated by an biased binary source, the analysis of the fill-up level of a suffix-tree remaining also an open problem.

Motif Statistics. The (1999-2002) articles [166, 167] entitled “Motif statistics”, coauthored with P. Nicodème and B. Salvy, build upon important previous developments of theoretical computer science. It is worth recalling some corner stones of automata theory, a major topic in this article coping on regular expressions. Kleene [147] (1956) and Rabin and Scott [171] (1959) provided constructions of DFA for regular expressions. Aho and Corasick devised an efficient algorithm [2] (1975) to construct an automata for searching finite set of words (1975) while Knuth, Morris and Pratt [149] (1977) gave a fast algorithm that is also realized by an automaton and searches for occurrences of a single word. In a fundamental article about context-free languages, Chomsky and Schützenberger [17] (1963) gave an algorithm computing the generating function of words recognized by a Deterministic Finite Automaton on a finite alphabet. This generating function is always solution of a system of linear equations, the homogeneous part of which having coefficients that are monomials of degree one 1. TEXT ANALYSIS 5 with respect to the alphabet; it follows that the resulting generating functions (those of regular languages by the classical automata constructions [147, 171] previously mentioned) are rational. Note that P. Flajolet and coauthors results have a wide generality, providing an algorithmic construction for regular patterns, a class which contains all finite patterns. From the resulting bivariate generating function follows computation of the moments and access to the normal limit law. The automata constructions however hide the structural properties of finite patterns. These are mostly provided by language analysis and once again we have to mention the pioneering work of Guibas and Odlyzko [130, 131] (1981); they followed the idea of parsing a text with respect to the occurrences of the pattern, and defined the languages Right of words finishing with the first occurrence of a word of the pattern, Minimal of words separating two occurrences and Ultimate of words following the last occurrence. Guibas and Odlyzko provided the generating functions of these languages by recurrence; later, Régnier and Szpankowski [175, 176] (1997,1998) and Régnier [172] (2000) provided a set of formal equations for these languages and proved for single word patterns Gaussian or Poisson limits, depending on number of occurrences being Θ(n) or O(1). Régnier and Denise [173, 174] (2003, 2004) obtained large deviation results for occurrences of one word conditioned by the number of observed occurrences of another word. Quite contemporary with Guibas-Odlyzko seminal work, Goulden and Jackson [127, 128] (1979,1983) devised a very powerful analytic inclusion-exclusion method that provides multivariate counting for a reduced set of words. This was later generalized to general sets of words by Noonan and Zeilberger [168] and the corresponding complete proofs have been done by Bassino et al. [7] (to appear). Although the work of Goulden and Jackson [127, 128] deals with multivariate counting of occurrences of words, it seems that no multivariate counting with respect to the Chomsky-Schützenberger algorithm has been used before the “Motif Statis- tics” article. There, P. Flajolet and his coauthors use the trick of enlarging the alphabet by a counting letter of size zero, and replacing each transition going to a final state by a “compound transition” that include this letter. This counting letter records the number of positions where a match with the regular expression is found. Then, using the Chomsky-Schützenberger algorithm provides a bivariate generating function for the lengths of the texts and the number of matching positions. Later, Nicodème [165] (2003) replaced the compound transitions by marked states, where a marked state corresponds to a matching position and there can be several types of marked states for multivariate counting of several regular expressions; each marked state emits next the corresponding counting variable during the processing of the Chomsky-Schützenberger algorithm. These marked states have the same algorithmic properties as final states; in particular, it is easy to generalize the determinization and minimization algorithms for automata with marked states or to apply to them the construction providing a Markov automaton. An important result of “Motif Statistics” is the asymptotic normal law for the number of matching positions of a regular expression in a random text. P. Flajolet and his coauthors provided an algorithm to cope with a Markovian source of any order, and therefore “Motif Statistics” generalizes previous work of Régnier and Sz- pankowski [175, 176] who proved a Gaussian limit law in Markovian models for one 6 1. TEXT ANALYSIS word. It does not subsume however Bender and Kochman work [9] since they considered simultaneous counts of matches with patterns that are finite set of words and obtained multivariate Gaussian limit laws. Since Deterministic Finite Automata are representable by positive matrices, P. Flajolet and his coauthors use the Perron-Frobenius theorem asserting that positive matrices have a unique dominant real positive eigenvalue; this eigenvalue is a function of the formal variable used for the counting of the matching positions, variable that is assumed to be positive. Then follows a uniform separation property of the poles of the rational bivariate generating function counting the text lengths and the matches and an application of the Cauchy theorem provides a formula suitable for application of Hwang’s Quasi-Power-Theorem [135, 136] (1996, 1998); note that Heuberger extension of this theorem [133] (2007) to two dimensions should apply to simultaneous statistics of two regular motifs while no theory is presently available for statistics of higher dimensions. The variability condition of Hwang’s theorem is asserted valid by a delicate property of log-convexity of the dominant eigenvalue. Another algorithmic trick of the “Motif Statistics” article reduces the computation of asymptotic moments to the solution of a few linear systems with constants entries instead of one linear system with polynomials entries. Bourdon and Vallée [11] (2006) generalized the analysis of motif statistics to the case of dynamical sources. We quote here that a purely probabilistic approach to word counting has been devised by several authors. We refer for bibliographic entries to Chapter 6 “Statistics on Words with Applications to Biological Sequences” of Lothaire book [157] (2005) “Applied Combinatorics on Words” that is devoted to this approach. This probabilistic method uses in particular the Chen-Stein method [182, 14] (1972,1975) and Poisson and compound Poisson approximations [15] (2004). The algorithms present in the “Motif Statistics” article have been programmed in the Maple package regexpcount available within the algolib library 2 and applied to biological applications; indeed, the protein motifs of the database PROSITE 3 correspond to regular expressions generating finite languages; these languages however can have enormous size while the automatons recognizing them have moderate sizes that make them physically computable. As an experimental conclusion, P. Flajo- let and his coauthors observe that the theoretical predictions of number of occurrences of protein motifs tend to underestimate systematically the observations.

Hidden Pattern Statistics. This (2001) article coauthored with Y. Guivarc’h, W. Szpankowski and B. Vallée considers a model typically different of the one of “Motif Statistics” where the number of matching positions in a sequence of length n is upper bounded by n. By instance, the hidden pattern a#∞a occurs n(n 1)/2 times in − the sequence an; all pairs of positions with an occurrence of the letter a are counted. This pattern is unconstrained in opposition to the pattern a#2a where all the pairs of positions with an a are counted, under the constraint that the gap separating these positions is less than 2 symbols.

2. This library is available at http://algo.inria.fr/libraries/ 3. See http:prosite.expasy.org 1. TEXT ANALYSIS 7

P. Flajolet and his coauthors were motivated by giving some statistical threshold for detection of network intrusion, where a hidden pattern occurs repetitively with a variable number of intervening events between each occurrence. Hidden patterns can also serve as a model for the exon-intron systems in molecular biology, or help for searches in data mining. Last but not least, hidden patterns statistical analysis can put a bit of intelligence in the “mysteries” of “secret codes” and “secret messages”; following links from P. Flajolet Web page, one ﬁnds that the book Moby Dick predicted in full details the accidental death of Lady Di... More seriously, the line of proof of this article is crystal clear. The dominant effect in the analysis come from the unbounded constraints #∞ between two letters, as seen previously in the toy example a#∞a. When considering a generic random 1 text, each of these “unbounded” constraints provides a factor 1−z in the probability generating functions of the ﬁrst moment of the random variable counting the number of occurrences. There are also two unbounded constraints which do not belong intrinsically to the pattern, namely corresponding to the set of words preceding the occurrence of the pattern and following this occurrence; these two constraints provide 1 a factor (1−z)2 . This implies that a hidden pattern with b 1 unbounded constraints − b #∞ will lead to an expectation of number of occurrences that is Θ n . The analysis of the second moment implies considering two simultaneous occurrences and possible intersections of blocks of positions, where a block is a set of positions that corresponds to a maximal subpattern without unbounded constraint of the hidden pattern. One occurrence of the pattern a#2b#∞aa#∞b will lead to three blocks of positions and two simultaneous occurrences will lead to (i) three hitting blocks, (ii) two hitting blocks and two non-hitting blocks, (iii) one hitting block and four non-hitting blocks and (iv) 1 six non-hitting blocks, each contributing again by a factor 1−z to the generating function of the second moment. Then, and the proof is as simple as beautiful, the less the number of intersecting blocks of two simultaneous occurrences, the highest the number of blocks and the highest asymptotic contribution to the second moment, namely Θ(n2b−1) for a hidden pattern with b 1 unbounded constraints. The second moment − is explicitely provided as a function of a generalized correlation of the pattern that extends the usual notion of words correlation. The k-th moment is also computed by considering k simultaneous occurrences, the minimum number of intersecting blocks corresponding once more to the maximum freedom degree and to asymptotically dominant terms; using some bijections and properties of involutions provide the authors with a proof by convergence of moments to an asymptotic Gaussian law.

Hidden Word Statistics. This (2006) article coauthored with W. Szpankowski and B. Vallée gives important extensions to the conference article “Hidden Pattern Statistics”. A delicate dynamical programming procedure computes the variance of number of occurrences in polynomial time with respect to the size of the pattern speciﬁcation. The limit distribution for fully constrained patterns is analysed from a different point of view which leads to results that cannot be reached by convergence of moments. A de Bruijn graph [25] (1946) is used to embed the count of occurrences in the spirit of transfer matrices; then, Perron-Frobenius dominant properties and the Quasi-Power- Theorem of Hwang [134, 135] (1994, 1996) are used in a vein similar to their use in 8 1. TEXT ANALYSIS the article “Motif Statistics”. P. Flajolet and his coauthors push further the analysis of the dominant real eigenvalue of the system by considering the cycles of the de Bruijn graph to obtain a Central Limit Law, Large Deviations bounds, and, for primitive patterns, a Local Limit Theorem. P. Flajolet loved emphasizing his “engineering” skill, and the “Hidden Word Sta- tistics” article presents an unexpected counting wheel machine that is a fruit of this skill. For the same reason, he wanted to apply his theoretical results; in the present case, he and his coauthors performed a comparison of observed counts of a hidden word or sentence in Shakespeare’s Hamlet with the expected corresponding counts computed under the same distribution of letters. The concluding section of the article provides an estimator for detection of true alarms, which was the original motivation of the article. Bourdon and Vallée [10] (2002) provide an analysis of hidden pattern statistics under a dynamical source model, compute the two ﬁrst moments of the count and prove concentration in distribution to the expectation; they leave however pending the proof of a limiting Gaussian law. PAPER 2 Paper74

PAPER 3 Paper 76

PAPER 4 Paper 191

Chapter II

INFORMATION THEORY

Analytic Information Theory

Wojciech Szpankowski

Philippe Flajolet had long term research interests in algorithms — that are at the heart of virtually all computing technologies – and combinatorics — that provide in- dispensable tools for ﬁnding patterns and structures. However, we shall argue that from early days Flajolet was fascinated with information, that permeats every corner of our lives and shapes our universe. In fact, he was the “midwife” and active participant of analytic information theory that combines analytic combinatorics and information theory [39, 163, 37, 81, 119, 145, 185, 146]. In this chapter we concentrate on information theory of data compression and present Flajolet’s work in this area [119, 145, 185, 146], while in other chapters of this volume we touch upon other information-theoretic work of Flajolet [39, 163, 37, 81].

Analytic Information Theory Jacob Ziv in his 1997 Shannon Lecture presented compelling arguments for “back- ing off” from first-order asymptotics in order to predict the behavior of real systems with finite length description. To overcome these difficulties, the so called non- asymptotic analysis, in which lower and upper bounds are established with control- lable error terms, becomes quite popular. However, we argue that developing full asymptotic expansions and more precise analysis may be even more desirable. Fur- thermore, following Hadamard’s precept 1, Flajolet and others proposed to study information theory problems using techniques of complex analysis 2 such as generating functions, combinatorial calculus, Rice’s formula, Mellin transform, Fourier series, sequences distributed modulo 1, saddle point methods, analytic poissonization and depoissonization, and singularity analysis [188]. This program, which applies complex-analytic tools to information theory, constitutes analytic information theory. Analytic information theory can claim some successes in the last decade. We mention a few: proving in the negative the Wyner-Ziv conjecture regarding the longest match [183, 184]; establishing Ziv’s conjecture regarding the distribution of the number of phrases in the LZ’78 compression scheme [139]; showing the right order of

1. The shortest path between two truths on the real line passes through the complex plane. 2. Andrew Odlyzko wrote: “Analytic methods are extremely powerful and when they apply, they often yield estimates of unparalleled precision.”

17 18 ANALYTIC INFORMATION THEORY the LZ’78 redundancy [178, 158]; disproving the Steinberg-Gutman conjecture regarding lossy pattern matching compression schemes [159, 196, 153]; establishing precise redundancy of Huffman’s code [187] and redundancy of a fixed-to-variable no prefix free code [189]; deriving precise asymptotics of minimax redundancy for memoryless sources [195, 186, 190], Markov sources [177, 141] and renewal sources [120, 33]; precise analysis of variable-to-fixed codes such as Tunstall and Khodak codes [32]; designing and analyzing error resilient Lemple-Ziv’77 data compression scheme [155], and finally establishing entropy of hidden Markov processes [137] and the noisy constrained capacity [132, 142]. In this chapter, we only present Flajolet’s work on data compression, in particular, on the minimax redundancy for renewal sources [120]. We start with some preliminary discussion.

Preliminary Discussion Let us start with some definitions and preliminary results. A source code is a bijective mapping C : ∗ 0, 1 ∗ A → { } from the set of all sequences over an alphabet to the set 0, 1 ∗ of binary se- A { } quences. We write x ∗ for a sequence of unspecified length, and xj = x . . . x ∈ A i i j ∈ j−i+1 for a sequence of length j i + 1. We denote by P the probability of A − the source, and write L(C, x) (or simply L(x)) for the code length of the source sequence x over the code C. Finally, the source entropy is defined as usual by

H(P ) = ∗ P (x) log P (x) and the entropy rate is denoted by h. We often − x∈A present our results for the binary alphabet = 0, 1 . P A { } Our interest lies in the so called prefix codes for which no codeword is a prefix of another codeword. For such codes there is a mapping between a prefix code and a path in a tree from the root to a terminal (external) node (e.g., for a binary prefix code move to the left in the tree represents 0 and move to the right represents 1). We also point out that a prefix code and the corresponding path in a tree defines a lattice path in the first quadrant [32]. If some additional constraints are imposed on the prefix codes, this translates into certain restrictions on the lattice path [32]. The prefix condition imposes some restrictions on the code length. This fact is knows as Kraft’s inequality discussed next. Theorem 4.1 (Kraft’s Inequality). Let = 2. For any prefix code the codeword |A| lengths `1, `2, . . . , `N satisfy the inequality N (1) 2−ì 1. ≤ i=1 X Conversely, if codeword lengths satisfy this inequality, then one can build a prefix code.

Proof. This is an easy exercise on trees. Let `max be the maximum codeword length. Observe that at level `max some nodes are codewords, some are descendants of codewords, and some are neither. Since the number of descendants at level `max of a PRELIMINARY DISCUSSION 19

`max−`i codeword located at level `i is 2 , we obtain

N 2`max−i 2`max , ≤ i=1 X which is the desired inequality. The converse part can also be proved, and is left for the reader.

Kraft’s inequality begs for an interesting combinatorial question. How many tu- N −`i ples (`11, . . . , `N ) are there such that the equality in (1) holds, that is, i=1 2 = 1. This question was answered in [81]. P Observe that the Kraft’s inequality implies the existence of at least one sequence x such that L(x) log P (x). e ≥ − Actually, a stronger statement is due to Barron [6] who proved the following result. e e Lemma 4.1 (Barron). Let L(X) be the length of a preﬁx code, where X is generated by a stationary ergodic source over a binary alphabet. For any sequence an of positive constants satisfying 2−an < the following holds n ∞ PPr(L(X) < log P (X) a ) 2−an , − − n ≤ and therefore L(X) log P (X) a (almost surely). ≥ − − n Proof: We argue as follows:

Pr(L(X) < log2 P (X) an) = P (x) − − −L(x)−a x:P (x)

Using Kraft’s inequality we can now prove the ﬁrst theorem of Shannon that bounds from below the average code length. Theorem 4.2. For any preﬁx code the average code length E[L(C,X)] cannot be smaller than the entropy of the source H(P ), that is,

E[L(C,X)] H(P ). ≥ where the expectation is taken with respect to the distribution P of the source sequence X. 20 ANALYTIC INFORMATION THEORY

Proof. Let K = 2−L(x) 1 for a binary alphabet, and L(x) := L(C, x). Then x ≤ E[L(C,XP)] H(P )] = P (x)L(x) + P (x) log P (x) − ∈A∗ ∈A∗ xX xX P (x) = P (x) log log K 0 2−L(x)/K − ≥ ∈A∗ xX since log x x 1 for 0 < x 1 or the divergence is nonnegative, while K 1 by ≤ − ≤ ≤ Kraft’s inequality. What is the best code length? We are now in a position to answer this question. As long as the expected code length is concerned, one needs to solve the following constrained optimization problem for, say a binary alphabet

min L(x)P (x) subject to 2−L(x) 1. L ≤ x x X X This optimization problem has an easy solution through Lagrangian multipliers, and one ﬁnds that the optimal code length is L(x) = log P (x) provided the integer − character of the length is ignored. In general, one needs to round the length to an integer, thereby incurring some cost. This cost is usually known under the name redundancy. For known distribution P , the pointwise redundancy R(x) for a code C, the average redundancy R¯, and maximal redundancy R∗aredefinedrespectivelyasto R(x)=L(C,x)+log P (x), R¯ = E[L(C,X)] H(P )] 0, ∗ − ≥ R = maxx L(C, x) + log P (x)].The pointwise redundancy can be negative, but the average redundancy cannot due to the Shannon theorem. From now on we assume that the source string x is of length n, and study the maximal redundancy – that we denote as R∗ – for a class of sources . Such redundancy si known as the minimax n S redundancy discussed next.

Minimax Redundancy for a Class of Sources Let us begin with a precise information-theoretic definition of the minimax redun- n dancy and its Shtarkov’s bounds. Throughout this section, we write L(Cn, x ) for the length of a fixed-to-variable code C ; n 0, 1 ∗ assigned to the source sequence n A → { } xn over the alphabet = 1, 2, . . . , m of size m that can be finite or not. In practice, A { } one can only hope to have some knowledge about a family of sources that generates S the data, such as the family of memoryless sources or Markov sources of M0 Mr order r > 0. Following Davisson [24] and Shtarkov [181], we define the minimax worst-case (maximal) redundancy R∗ ( ) for a family as n S S ∗ n n (2) Rn( ) = min sup maxn [L(Cn, x1 ) + log P (x1 )] , S Cn P ∈S x1 where Cn represents a set of prefix codes, and the source P generates the se- n ∈ S n quence x = x1 . . . xn. If we ignore the integer nature of the code length L(Cn, x ), n then we can approximate it by log 1/Pθ for some θ. Furthermore, log supP ∈S P (x ) = MINIMAX REDUNDANCY FOR MEMORYLESS SOURCES 21

ˆ log(1/Pθˆ), where θ is the ML estimator, so that

∗ Pθˆ (3) Rn( ) = inf maxn log + O(1). S θ x Pθ We derive now Shtarkov’s bound [181]. Deﬁne ﬁrst the maximum likelihood distribution n ∗ n supP ∈S P (x ) Q (x ) := n . yn∈An supP ∈S P (y ) Then observe [33] P ∗ n n Rn( ) = min sup maxn (L(Cn, x ) + log P (x )) S Cn P ∈S x

n n = min max L(Cn, x ) + sup log P (x ) C xn n P ∈S n ∗ n n = min maxn (L(Cn, x ) + log Q (x )) + log sup P (y ) Cn x ∈S n∈An P y X GS ∗ n n = Rn (Q ) + log sup P (y ) = log sup P (y ) + O(1) ∈S ∈S n∈An P n∈An P y X y X GS ∗ where 0 < Rn (Q ) 1 is the redundancy of the optimal generalized Shannon code ≤ GS ∗ (see [33]). Therefore, ignoring again the integer constraint (i.e., setting Rn (Q ) = 0) and using (3) rather than (2) we arrive at

∗ n Pθˆ Rn( ) = sup Pθ(x ) = inf max log . S θ∈Θ xn P xn θ∈Θ θ X From now on, we assume that R∗( ) = log D ( ) where S n,m S n (4) Dn,m( ) = sup P (x ). S ∈S n∈An P x X The O(1) term in (3) can be computed for ﬁnitely parameterized sources as in [33], but we will not elaborate on it here.

Minimax Redundancy for Memoryless Sources As a warm-up exercise, we ﬁrst consider the minimax redundancy for a class of memoryless sources over alphabet of size m. We follow here [186]. Observe that D := D ( ) deﬁned in (4) takes the form n,m n,m M0 n k k1 k km (5) D = 1 m , n,m k , . . . , k n ··· n ··· 1 m k1+ X+km=n where k is the number of times symbol i occurs in a string of length n. Indeed, i ∈ A observing that P (xn) = pk1 pkm where p are unknown parameters θ representing 1 ··· m i 22 ANALYTIC INFORMATION THEORY the probability for symbol i , we proceed as follows ∈ A n Dn( 0) = sup P (x1 ) M n xn P (x1 ) X1

n k1 km = sup p1 pm k , . . . , k p ,...,p ··· ··· 1 m 1 m k1+ X+km=n n k k1 k km = 1 m , k , . . . , k n ··· n ··· 1 m k1+ X+km=n where the last line follows from

k1 km n k1 km k1 km sup P (x1 ) = sup p1 pm = . n p ,...,p ··· n ··· n P (x1 ) 1 m We should point out that (5) has a form that re-appears in the redundancy analysis of other sources. Indeed, the summation is over tuples =˛(k1, . . . , km) representing a (memoryless) type and under the sum the first term n counts the number k1,...,km of sequences xn of the same type while the second term is the maximum likelihood distribution. It is argued in [186] that the asymptotics of such a sum can be analyzed through its so-called tree-like generating function defined as ∞ nn D (z) = D zn. m n! n,m n=0 X Here, we will follow the same methodology and employ the convolution formula for tree-like generating functions (cf. [188]). Observe that Dm(z) relates to another tree- like generating function defined as ∞ kk B(z) = zk. k! kX=0 This function, in turn, can be shown to be (cf. [188]) B(z) = (1 T (z))−1 for k−1 − z < e−1, where T (z) = ∞ k zk is the well-known tree function — that | | k=1 k! counts the number of rooted labeled trees on n vertices [114] — satisfying the implicit equation P (6) T (z) = zeT (z) with T (z) < 1. The convolution formula [188] applied to (5) yields | | (7) D (z) = [B(z)]m 1. m − n! n m n n Consequently, Dn,m = nn [z ][B(z)] where [z ]f(z) denotes the coefficient of z in f(z). Defining β(z) = B(z/e), z < 1, noticing that [zn]β(z) = e−n[zn]B(z), and | | applying Stirling’s formula, (7) yields −1 n m (8) Dn,m = √2πn 1 + O(n ) [z ][β(z)] . MINIMAX REDUNDANCY FOR RENEWAL SOURCES 23

Thus, it suffices to extract asymptotics of the coefficient at zn of [β(z)]m, for which a standard tool is Cauchy’s coefficient formula [114, 188], that is, 1 βm(z) (9) [zn][β(z)]m = dz 2πi zn+1 I where the integration is around a closed path containing z = 0 inside which βm(z) is analytic. However, asymptotic evaluation of the above depends whether m is finite or is a function of n [186, 190].

Minimax Redundancy for Renewal Sources Let us continue our analytic extravaganza and consider non-finitely parameterized sources. We study here the so called renewal sources first introduced in 1996 by Csiszár and Shields [23]. Such a source is defined as follows: – Let T1,T2 ... be a sequence of i.i.d. positive-valued random variables with distribution Q(j) = Pr T = j . { i } – The process T0,T0 + T1,T0 + T1 + T2,... is a renewal process. – In a binary renewal sequence the positions of the 1’s are at the renewal epochs T ,T + T ,... with runs of zeros of lengths T 1, T 1, . . . in between the 0 0 1 1 − 2 − 1’s. – The process starts with x0 = 1. We follow here the analysis presented in [120]. A sequence generated by such a source becomes xn = 10α1 10α2 1 10αn 1 0 0 0 ··· ··· k∗ where k is the number of i such that α = m. Then m i | {z } P (xn) = [Q(0)]k0 [Q(1)]k1 [Q(n 1)]kn−1 Pr T > k∗ . 1 ··· − { 1 } The last term introduces some difficulties in finding the maximum likelihood distribution, but it can be proved that the minimax redundancy R∗ ( ) = log D ( ) of the n R0 n R0 renewal source satisfies R0 n r 1 D ( ) r n+1 − ≤ n R0 ≤ m m=0 X n where rn = k=0 rn,k and

k0 k1 kn−1 P k k0 k1 kn−1 (10) rn,k = . k0 kn−1 k k ··· k IX(n,k) ··· Above (n, k) is is the integer partition of n into k terms, i.e., I n = 1k + 2k + + nk − , k = k + + k − . 0 1 ··· n 1 0 ··· n 1 n Since rn is too difﬁcult to analyze, we rather study sn = k=0 sn,k where

k0 kn−1 −k k k rn,kP k! sn,k = e , = k −k k0! ··· kn−1! sn,k k e PX(n,k) 24 ANALYTIC INFORMATION THEORY

Figure 4.1. Iluustration to the Saddle Point Method since ∞ k0+···+kn−1 k0 kn−1 k n 1k0+2k1+···+nkn−1 u k k i S(z, u) = sn,k(u/e) z = z = β(z u) e k ! ··· k − ! P 0 n 1 i=1 Xk,n Xn,k Y where β(z) = B(z/e) is deﬁned in the previous section. To compare sn to rn, we introduce the random variable Kn as follows sn,k Pr Kn = k = . { } sn Stirling’s formula yields n rn rn,k sn,k −Kn Kn = = E[(Kn)!Kn e ] sn sn,k sn k=0 X 1 − 2 = E[ 2πKn] + O(E[Kn ]). Thus p

rn = snE[ 2πKn](1 + o(1)) = sn 2πE[Kn](1 + o(1)).

To understand probabilisticp behavior of Kpn, we apply sophisticated tools of analytic combinatorics such as Mellin transform and the saddle point [114, 188]. In particular, we must evaluate [zn]S(z, 1) by the saddle point that leads to the following c 1 s = [zn]S(z, 1) = [zn] exp + a log n 1 z 1 z − − MINIMAX REDUNDANCY FOR RENEWAL SOURCES 25 which is illustrated in Figure 1. We prove in [120] the following. 2 Lemma 4.2. Let µn = E[Kn] and σn = V(Kn). Then 1 n n µ = log + o(√n), σ2 = O(n log n) = o(µ2 ), n 4 c c n n r where c = π2/6 1, d = log 2 3 log c 3 log π. − − − 8 − 4 This leads to our ﬁnal result proved in [120]. Theorem 4.3 (Flajolet and Szpankowski, 1998). We have the following asymptotics 7 s exp 2√cn log n + O(1) , n ∼ − 8 2 5 1 log r = √cn log n + log log n + O(1). n log 2 − 8 2 that yields 2 R∗ ( ) = √cn + O(log n). n R0 log 2 2 where c = π 1 0.645. 6 − ≈

PAPER 5 Paper 158

PAPER 6 Paper 173

PAPER 7 Paper Seminar

PAPER 8 Paper Seminar

Chapter III

DIGITAL TREES

The digital tree process

Julien Clément, Mark Daniel Ward

The analysis of trees was integrated in Philippe Flajolet’s writings and research throughout his life. In particular, he had a very keen interest in the digital tree process [116, 53], omnipresent in Computer Science, for which the trie data structure is the strongest and best known embodiment. During Philippe’s invited lecture at STACS’06, which was devoted to tries, he said that the treatment “cannot be but a brief guide to a rich subject whose proper development would require a book of full length” [53].

1. A central role in computer science The digital tree process is found practically anywhere that data is classified or sorted and has abundant applications. In a nutshell, this process relies on the principle of the thumb rule in dictionaries. As a data structure, the most pervasive kind of digital tree is the retrieval tree, introduced by de la Briandais [28] and Fredkin [126] 1, and usually shortened to “trie”. (Section 6.3 of Knuth [152] is a very helpful and fundamental discussion of the fun- damentals, which traces the principles of tries to Thue [191].) A partitioning of the data items—often using a sorting or classification by types—takes place at the root node. The tree is built recursively, according to subsequent bits or digits of the data. The children of the root are sorted further into subtrees and are thus partitioned more finely. The data items (also known as keys or strings) eventually require no more sorting and are ultimately stored in leaves of the trie. Due to their generality, tries are one of the most widely-known and greatly-studied data structures for representing a set of words. The trie data structure allows for all basic algorithmic operations one can expect from a dynamic dictionary-type data structure (inserting, deleting, searching, enumerating) and can also be used to sort a set of strings. More formally, given an alphabet = a , . . . , a of cardinality r, and for a A { 1 r} prefix-free 2 set of words Y with letters from , the trie T (Y ) associated to Y is A 1. As a side note, let us mention that initially Fredkin intended that tries should be pronounced “tree” as in the word “retrieval”. Alas the trie data structure escaped its creator and, nowadays, people mostly pronounced it as the word “try”, to distinguish verbally from “tree”. 2. The prefix-free property just means that none of the words is prefix of another.

37 38 THE DIGITAL TREE PROCESS 28 THE DIGITAL TREE PROCESS

a b d f h o s t w

n e i o r o e n † t u h h r

y c d e n a r r e e r e y e

† a † s e m t e † p v † n † c

u † † a h † † i † k

s † † v †

e e

† †

a b d f h o s t w ny† ecause† orth† ere† ne†

i o r † t u h h r d† ama† ep† rvive† y† eck† e n e s† e† † n †

the†

a b d f h o s t w

ny† ecause† rama† orth† ere† ne† † hen† hy† i o t u r d† ne† ep† rvive† eck† e s†

Figure 8.1.. From From top top to to bottom, bottom, a a fully fully developed developed digital digital tree, tree, a a trie trie and and a a digital search tree built upon the words from the last sentence of the novel Moby Dick by H. Melville (inserted in order – relevant only for the DST):): “The Drama’s Done. Why then here does any one step forth? - Because one did survive the wreck.”. InIn the the trie, trie, unary unary nodes nodes which which would would not not be be present present inin PATRICIA triestries are are double double circled. circled. A A terminal terminal symbol symbol ‘†’ ‘†’ is is added added to to each each word inserted. defined recursively thanks to the following rules ,, ifif Y = ;; ;∅ ;∅ T ((Y ):=) := σ,, ifif Y = σ ;; 8 { } > { } < ,T,T((Y a11)),T,T((Y a22)),, T ((Y att)) ,, otherwise. hh•• \ \ ······ \ ii where denotes an:> internal node and Y ↵α isis the the subset subset built built from from Y by considering •  \ words that start with the letter ↵α,, and and stripped stripped of of their their initial initial symbol symbol ↵α.. For For a a trie, trie, thethe recursion recursion stops stops as as soon soon as as Y contains less than two elements. Thus in order to build the trie for Y , one needs to consider only the minimal set of prefixes from Y by 2. DIGITAL TREES IN PHILIPPE FLAJOLET’S WORKS 39 build the trie for Y , one needs to consider only the minimal set of prefixes from Y by which all words are distinguished one from another. The prefix-freeness condition is merely technical and is easily forced by adding a terminal symbol (not belonging to the alphabet) to each string. (If two strings have the same finite length, these terminal symbols might need to be different to avoid collisions of two identical strings; e.g., see the two occurrences of the word one in the Moby Dick example.) The abstract data structure has given rise to many algorithmic variants. For instance PATRICIA tries 3 [164] are tries where only “useful internal nodes”, i.e., participating in the branching process, are considered. Thus in this setting unary nodes (with only one child) are removed. At the very end of the spectrum lies the digital search tree (DST): it uses the same partitioning process as tries but strings are stored inside internal nodes (like Binary Search Trees). Unlike the case of tries, a DST for a set of words Y depends on the order in which the strings are inserted. So loosely speaking, digital search trees are intermediary between tries and binary search trees. When coming to implementation of digital trees, even for “usual” tries, several options are possible depending on the decision structure chosen to guide descent in each node to subtrees (arrays of pointers, linked lists, or binary search trees for instance). Some variants of digital trees have been precisely analyzed by Philippe Flajolet [102, 19, 21]. With a more conceptual point of view, the structure of a trie can be used to model or analyze the behavior of both deterministic and stochastic algorithms in computer science. Tries are especially relevant to branching and sorting processes. So it is not surprising that the digital tree process has ramifications in the management of large databases (dynamic hashing [70], see also the introduction to FIXTHIS-the Chapter on Hashing; probabilistic counting [44], see FIXTHIS-Chapter on Approximate Count- ing), in communication protocols [64] (for instance, for leader election [143, 170]), data compression (Lempel-Ziv and its variants [197], suffix trees [41, 156]), pattern matching [8, 138], random generation (to analyse precise schemes [95] or used as auxiliaries in Buffon machines [77]) and finally, rather unexpectedly, in computational geometry (for exact comparison of rationals [122]). The digital tree process is elegant and simple. In its algorithmic form, it is intu- itive to implement and utilize. This helps explain why the algorithmic, analytic and probabilistic aspects of digital trees are fundamental in both theoretical and applied domains in computer science.

2. Digital trees in Philippe Flajolet’s works We can identify three main periods with respect to the study of digital trees by Philippe Flajolet. The first period corresponds roughly to the early 1980s and focuses on tries and their applications in computer science. One must remember that, at this time of precur- sors, the average case analysis of algorithms was not yet a well recognized field among computer scientists (despite the spreading of Knuth’s ideas). Worst-case analysis— focusing on pathological cases—were then the norm to study efficiency of algorithms. Tries (especially binary tries) and the underlying dyadic partition process were (and

3. For “Practical Algorithm to Retrieve Information Coded in Alphanumeric”. 40 THE DIGITAL TREE PROCESS still are) ideal tools for demonstrating the utility of average case analysis. Indeed, although the trie data structure is very efficient on average (and, indeed, can compete with the best known data structures in many applications), the worst case complexity is unbounded 4! It is worth noting that these first papers [116, 43, 91] introduce, in a pedagogical way, the case of binary tries built on a set of finite words of the same length ` (choosing n words amongst the 2` possible ones). This is a first combinatorial model easy to grasp. Of course it is not surprising that this model coincides—when ` tends to infinity—with the trie model for infinite binary strings, which has proved to be the natural framework for the analysis of tries; this framework is also rigorous, using a probability measure discussed in [169]. In view of the applications (for instance, dynamic hashing or sorting), a general and symbolic methodology for average case analysis of digital trees is presented, with a distinction made between additive parameters such as size or path length, and extremal (or multiplicative) parameters like the height. The second period, during the mid 1980’s to mid 1990’s, Philippe Flajolet’s work on digital trees is more oriented towards the refinement of methodological tools for studying variations of tries (that are numerous: multi-way trie, PATRICIA trie, quad- trie, k dimensional trie, LC-trie, ternary search tries, Digital search tree, etc). What was certainly appealing to Philippe Flajolet is that these analyses lead to challeng- ing mathematical problems. He sharpened and generalized several analytic tools by working with digital trees. Amongst these many techniques used by Philippe and his co-authors to analyze digital trees and their variants [43, 91, 93, 60], we emphasize his pioneering work to systematically apply generating functions, the symbolic method, singularity analysis, the saddle point method, the Mellin transform, and poissonization/depoissonization techniques. Using tries and variants, he made many fruitful in- cursions into domains such as polynomial factorization [117], algebraic methods [91], differential equations [93], and random number generation [95]. For roughly the last 12 years of his life (1999–2011), he paid particular attention to developing and analyzing general probabilistic frameworks and tools [21, 53, 192, 94], characterizing the stochastic generation of words and strings which are inserted in digital trees 5. This aspect is clearly related to information theory (see Chapter FIXTHIS-XYZ on Information Theory). Indeed one fundamental aspect of random digital trees is that the randomness essentially comes from the set of words they are built on; hence, there is a need to precisely model the source producing (infinite) words. This need was resolved by a very general framework introducing dynamical sources (mainly designed by Brigitte Vallée; see the corresponding chapter/volume- FIXTHIS-on-Dynamic-Sources for related works of Philippe Flajolet), which relies on transfer operators of dynamical systems theory. In this framework various parameters of tries are analysed [21] showing that the constants involved are related to intrinsic properties of the source like the entropy of the source. The Grail of such a study would be to consider for analyses a totally general framework where a (stochastic) source is induced from the family of fundamental probabilities p ∈A∗ , where p is the { w}w w 4. Intuitively two words in a trie can share a prefix of arbitrary length; fortunately, the probability of having any infinite-length overlaps is 0 in the naturally induced probability measure. 5. A study initiated by Devroye in [30] for binary tries. 2. DIGITAL TREES IN PHILIPPE FLAJOLET’S WORKS 41 probability that an infinite string produced by the source on the denumerable alphabet begins with w. P. Flajolet’s methodology was then to put stochastic properties of A the source into a correspondence with analytic properties of the Dirichlet series

s Λ(s) = pw, ∈A∗ wX for a complex parameter s; thus a connection is forged between a stochastic and analytic approach, which yields results for analysis of algorithms. This unifying view illuminates, for instance, ways that the fundamental characteristics of the source (like the entropy) appear in critical ways during the analyses of algorithms or data structures related to the digital trees, especially when computing constants in asymptotic expansions related to generating functions of tree parameters. This approach allows one to relate, for example, the study of the famous Quicksort algorithm for strings [192] (considering keys are strings produced by a source) to the one of ternary search tries (which mixes binary search trees inside tries, whereas Quicksort for strings mixes tries inside a binary search tree). A recurring theme in the analysis of tries is the appearance of minute oscillatory phenomena under some conditions. These oscillations are small but nonetheless are of great interest for the precise asymptotic mathematical analysis. In fact, this is inher- ent to the partitioning process, especially in the simplest stochastic model, in which words are drawn from an unbiased memoryless source. One key aspect for evaluating these oscillations together with error terms is to study precisely poles of the Mellin transform, relying on geometric and arithmetic conditions [94] of the source 6.

Methodology. We concentrate here more on tries. Some of the main parameters for tries are the size (the number of internal nodes) in relation with the memory space needed to store the data structure, the external path length (sum of the lengths of all paths from the root to all leaves) which relates to the construction cost of the data structure, and the height of trie (corresponding to the worst case for the number of symbol comparisons between any two strings in a set of strings). A random trie built over 500 uniformly generated strings, originally displayed in [53], is given in Figure 2.

As already mentioned, the studies differ whether we are interested in additive (size, path length) or multiplicative (height) parameters. However there are always two main steps: one is algebraic and provides exact expressions, the second one is analytic and aims at providing precise asymptotics. An additive parameter γ for a trie (Y ) built on the set of strings Y is decom- T posed thanks to a “toll” function τ in the following recursive way

γ( (Y )) = τ(Y ) + γ( (Y α)), T T \ ∈A αX

6. This leads to a surprising irruption of the Riemann hypothesis, to the great pleasure of P. Flajolet, when studying tries built upon words resulting from the continued fraction expansion of random numbers of the unit interval (see [21]). 42 THE DIGITAL TREE PROCESS

Figure 8.2. “A random trie of size n = 500 built over uniform data” [53] where τ(Y ) is a toll function associated to the root of the trie. For instance, the toll functions for size (number of internal nodes), external path length are respectively (using the Iverson notation, i.e., B is one if property B is true and zero otherwise) J K τ(Y ) = Card(Y ) 2 , τ(Y ) = Card(Y ) Card(Y ) 2 . J ≥ K × J ≥ K Concerning the algebraic step, when the number of strings in the trie is a Poisson random variable N with parameter z k −z z P(N = k) = e . k! (this is called a Poissonized model) instead of a fixed number n (in the traditional Bernoulli model), the calculations are greatly simplified in the resulting model. The Poisson distribution is well concentrated around the mean z, so that—by choosing the parameter z = n—the two models agree in many ways. Expressions in the Poisson model are also called Poissonised generating functions and are related to exponential generating functions. Then at this stage, for additive parameters of the trie, we usually obtain a functional equation for the Poissonised generating functions which can be iterated, yielding an infinite sum. The variable z is then interpreted as a complex-valued variable, and asymptotic methods are used. The expressions computed at the previous stage are either directly or very close to harmonic sums of the form

G(z) = λkg(µkz), ∈ kXK where families (λk) and (µk) are called amplitude and frequencies and g is the base function. Then the tool of choice to study asymptotics of harmonic sums is the Mellin 3. CONCLUSION 43 transform as it isolates the transform of the base function and a Dirichlet-type series involving only the λk’s and µk’s. We refer to the corresponding chapter [FIXTHIS- CITE-Dumas-chapter] for a more precise description. The next step is then to relate the asymptotic development to the poles of the Mellin transform using methods from analytic combinatorics. The ﬁnal, required step to perform depoissonisation, i.e., to interpret the degree to which the results from the Poissonized model are also valid in the (original) Bernoulli model, in which the number of strings is ﬁxed. We note that this is the most standard path to analyse tries (or related structures). This is sometimes referred as the Poisson-Mellin-Newton cycle. But there are other paths possible for instance using the Rice-Nörlund formula [192] (instead of Mellin transform) after an algebraic depoissonisation step. The schema does not apply to multiplicative parameters. The precise analysis of the distribution of the height relies on a saddle point estimate [44].

Tries and the Analytic Combinatorics book. Although digital trees have been a focus of interest throughout the whole Philippe Flajolet’s career, it is worth noting that digital trees are not present 7 in the book co-authored with R. Sedgewick [115] which is undoubtedly bound to be the reference in the field of analytic combinatorics. It may be useful to put this on perspective. Indeed there was discussion between the authors, P. Flajolet and R. Sedgewick, to decide the fate of digital trees together with Mellin transform with respect to the book. The Mellin transform, and the analysis of tries as an advanced application, were in fact originally planned to be included as a book chapter [108]. These topics were omitted however, in the final stages, according to R. Sedgewick because revising and incorporating the full discussion would caused too much delay for this long awaited book. One may also infer that digital trees do not completely fit into the “philosophy” of the book, since problems related to a trie are of a more stochastic than combinatorial nature. Indeed, for digital trees, one is often confronted with generating functions for parameters satisfying a functional equation, from which an explicit expression (under the form of an infinite sum) is deduced by iteration. But these generating functions are not generating functions for combinatorial structures.

3. Conclusion This whole chapter illustrates why the digital tree process is central in computer science, and how the related analyses make intervene deep mathematical tools.

A small lecture guide. The beautiful survey [53] made by Philippe Flajolet himself is surely a wonderful entry point into the subject. The reader interested in algebraic methods for tries should be delighted with [91]. Two articles are more concerned with methodology and in particular extend analyses to digital search trees [93, 102] (for digital search trees, functional equations now involve differentiation). Finally a general framework where strings are generated by very general sources—together

7. However we remark that tries are the subject of a chapter in the introductory book by the same authors Flajolet and Sedgewick [180] (with an analysis on the average relying only on elementary computations). 44 THE DIGITAL TREE PROCESS with analyses of related structures (like tries) and sorting & searching algorithms—is found in [21, 192, 94]. PAPER 9 Paper 34

Chapter IV

MELLIN TRANSFORM

Dr Flajolet’s elixir or Mellin transform and asymptotics

Philippe Dumas

Philippe Flajolet (abbreviated PF in the sequel) greatly developed the use of Mellin transform in the asymptotic evaluation of some combinatorial sums that appear in the average case analysis of algorithms. In fact, the Mellin transform runs throughout PF’s work from the beginning [74] to the very end [13].

Mellin transform and fundamental strip The Mellin transform is an integral transform, like the Laplace transform or the Fourier transform. It takes as input the original f(x), which is a function of a real variable defined on the real positive half-line. It produces the image f ∗(s), which is a function of a complex variable, defined by +∞ f ∗(s) = f(x)xs−1 dx. Z0 It is not clear that the formula actually defines anything, but the kernel xs−1 leads us to a comparison with the powers of x. It is readily seen that the assumption α f(x) =x→0 O(x ) guarantees the convergence of the lower part of the integral, say from 0 to 1, for complex numbers s whose real part is greater than α, that is for the − s which are on the right of α. We can make a similar assumption about the behavior − at infinity. In this way, the image f ∗(s) is defined within the intersection of a right half-plane and a left half-plane. This is a strip, called the fundamental strip. Certainly the most basic example of a Mellin transform is the gamma function +∞ Γ(s) = e−xxs−1 dx. Z0 It is the Mellin transform of the exponential e−x. In that case the original is O(1) at 0, so the left abscissa of the fundamental strip is 0. It decreases as x tends to infinity more rapidly than every power of x and the right abscissa is + . Hence the fundamental ∞ strip in this case is the positive right half-plane. But the gamma function extends to the whole complex plane as a meromorphic function, and the extension has poles at

49 50 DR FLAJOLET’S ELIXIR OR MELLIN TRANSFORM AND ASYMPTOTICS

Figure 9.1. Absolute value of the gamma function. all the nonpositive integers. The extension has poles at all the nonpositive integers, hence the peaks on the left-hand side of Figure 1.

Symbolic analysis Imagine a flat landscape, something like a flat sand desert. This country is the complex plane. A track straight through it. This is the real axis. But you are thinking of a meromorphic function and suddenly this changes the landscape. Some hills or even prodigious mountains appear and at the top of these mountains some placards are fixed on poles. Each vertex is located above a pole of the meromorphic function and on the placard you can read the singular part of the function at the pole. For example, if you are thinking of the gamma function, you see an infinity of chim- neys aligned on the negative part of the real axis, which disappear at the horizon in a haze of heat. On the placard at abscissa k, you read ( 1)k/(x + k). Your fan- − − tasy knows no limits and if you thought only for a moment about Stirling’s formula a placard appears on the other side, at the end of the real positive axis, with its wording x Γ(x) → ∞ (x/e) 2π/x. The more you direct your gaze toward a part of the ∼x + landscape, the more details spring up. p There is no doubt that PF had such a mental image of analytic functions, certainly in a more subtle way, refined by more than thirty years of practice. Undisputable evidences are the introduction to the saddle-point method of [115, Chap. VIII], the discussion about coalescing saddle-points from [4], or the picture of [55] illustrat- ing the application of the saddle-point method to a generalized exponential integral. FUNDAMENTAL RESULT 51

+∞ X 1 De Bruijn, 1948 ln 1 − e−xrk k=0

X b −k2x √ De Bruijn, Knuth, k d(k)e x = 1/ n Rice, 1972 k≥1 X j −x/2j j Knuth, 1973 2 (e − 1 + x/2 ) x = n j≥1

X −k2x √ Sedgewick, 1978 ∇∆F (k)e x = 1/ j k≥1

X a −16k2x √ Kemp, 1979 k v2(2k)e x = 1/ n k≥1 TABLE 1. The ﬁrst uses of the Mellin transform approach related to the analysis of algorithms, with their authors and date, and the harmonics sums therein.

Similarly, the search for summatory formulas is reduced to a purely formal han- dling [98]; the asymptotic study of divide-and-conquer type sums is reduced to picking residues [58, 62]. Always he was deﬁning rules that provide an automatic treatment of issues and reduce mathematical analysis to algebra and rewriting systems [100, 51]. Here we try to mimic this attitude and concentrate on the ideas, neglecting the mathematical assumptions.

Fundamental result The fundamental result about Mellin transform is the following. There is a strong correspondence between the behavior of the original at 0 and the poles of the image in the left half-plane with respect to the fundamental strip. Similarly, the behavior at inﬁnity of f(x) is related to the poles of f ∗(s) in the half-plane to the right of the fundamental strip. The correspondence is explicit and given by the following formulas: a term xξ lnk x in the expansion at 0 corresponds exactly to a singular term ( 1)kk!/(s + ξ)k for a pole ξ at the left of the fundamental strip. The formula is − − the same for the expansion at + and the poles on the right-hand side of the strip, but ∞ with an opposite sign: to ( 1)kk!/(s + ξ)k corresponds xξ lnk x. Particularly, for − − simple poles (that is k = 0), it is very simple: on the left-hand side the coefﬁcients of the expansion at 0 are the residues of the poles, and similarly on the right-hand side. Clearly, the correspondence works very well in case of the gamma function, ∞ ∞ + ( 1)k 1 + ( 1)k (1) Γ(s) = − , e−x = − xk. Re(s)≤0 k! s + k x→0 k! kX=0 kX=0 Note that in (1) we do not claim that the series converges or even that the sum is the gamma function. This equation is only a formal writing, in line with symbolic analysis. 52 DR FLAJOLET’S ELIXIR OR MELLIN TRANSFORM AND ASYMPTOTICS

Harmonic sums The second component of the story is the notion of harmonic sum. We start with a base function and its Mellin transform. To the base function, we associate a harmonic sum

(2) F (x) = λkf(λkx). Xk This is a linear combination of some dilations of the base function. We merge both ideas and we obtain a very simple result about the Mellin transform F ∗(s) of the harmonic sum. It is the product of the Mellin transform f ∗(s) of the base function and some generalized Dirichlet series Λ(s), which depends only on the coefﬁcients involved in the harmonic sum,

∗ ∗ λk (3) F (s) = f (s) s . µk Xk

Zigzag method We understand the power of the previous results by playing with the zigzag method, going back and forth between originals and images. We start with our favorite −x example f1(x) = e and its Mellin transform, the gamma function. We consider an alternative function ∞ x + f (x) = = xe−kx, 2 ex 1 − kX=1 which is actually a harmonic sum. We compute its Mellin transform and we collect its poles

+∞ j ∗ 1 ( 1) ζ( j) 1 f2 (s) = Γ(s + 1)ζ(s + 1) = + − − . Re(s)≤0 s j! s + j + 1 j=0 X They are 0 and the negative integers. But we know that ∞ 1 + B f (x) = 1 x + 2k x2k 2 − 2 (2k)! n=1 X is the generating function of the Bernoulli numbers. Moreover these numbers are zero for odd integers starting at 3, hence the writing

+∞ ∗ 1 1 1 B2k 1 f2 (s) = + . Re(s)≤0 s − 2 s + 1 (2k)! s + 2k kX=1 As a consequence the zeta function vanishes for the negative even integers. We now bring into the game a new harmonic sum

−k2x2 f3(x) = e kX=1 AVERAGE-CASE ANALYSIS OF ALGORITHMS AND HARMONIC SUMS 53 and compute its Mellin transform 1 s f ∗(s) = Γ ζ(s). 3 2 2 The vanishing of zeta at all the even negative integers removes almost all the poles of the gamma function. There remain only two poles, at 0 and 1. With

∗ √π 1 1 1 f3 (s) = Re(s)≤1 2 s 1 − 2 s − (again, the writing is purely formal), we readily obtain the expansion of the function at 0 √π 1 1 +∞ f3(x) = + O(x ). x→0 2 x − 2 Once we have understood the trick, it is not difﬁcult to deal with other examples, like

−k2x2 f4(x) = d(k)e . kX=1 ∗ 2 Here, d is the divisor function and the Mellin transform is f4 (s) = Γ(s/2)ζ(s) /2. Again we obtain the expansion at 0 easily (the double pole at 1 makes a logarithm arise)

√π ln x √π 1 1 +∞ f4(x) = + (3γ ln 4) + + O(x ). x→0 2 x 4 − x 4

Average-case analysis of algorithms and harmonic sums The application of the previous ideas to the analysis of algorithms began in the seventies, with a 1972 article [27] of Nicolaas G. De Bruijn, Donald E. Knuth, and Stephen O. Rice about the height of rooted plane trees (Table 1). Knuth [150, p. 132-124] refers to the method of the gamma function and credits De Bruijn with first having this idea. De Bruijn had used it in a 1948 paper [26] about the asymptotic evaluation of the binary partitions number. Next we encounter the study of radix-exchange sorting by Donald Knuth [150], of the odd-even merging by Robert Sedgewick [179], and of the register allocation for binary trees by Rainer Kemp [144]. In every case, a harmonic sum comes out (in the expressions of Table 1, d is the divisor function, ∇ and ∆ are the backward and forward difference operators, v2 is the dyadic valuation function). The first sum is a generating function, while the others ones are combinatorial sums, but they are all amenable to the same treatment. According to [71], PF learned the Mellin transform from Rainer Kemp around 1979. In a 1977 work about register allocation [86, 87], PF and his coauthors follow an elementary way à la Delange, but in a 1978/1979 talk [42] at the Séminaire Delange-Pisot-Poitou PF gives an explanation about the “Mellin-Fourier transform”, with only words but in a totally clear way. PF has systematized the idea starting in the eighties and completely defined the method in the early nineties. This led him to write first [88], a very illuminating presentation of the Mellin transform in the context of the analysis of algorithms, and next [60], a more comprehensive version. He returned to the topic in [108], in which the reader can find not only examples but even exercises. 54 DR FLAJOLET’S ELIXIR OR MELLIN TRANSFORM AND ASYMPTOTICS

X −nu PF, Odlyzko, 1981 average height of σr(n)e [73, 74] simple trees n≥1 k−1 X X j(k−s) PF, Puech, 1983 retrieval of multi- ε` 2 × [82, 84] dimensional data ` j≥0 −xαi,j −xαj,` (1 − e − βj,`e ) X −µ Fayolle, PF, Hofri, multi-access e ( (1 + Kµ) × 1986 [36, 37] broadcast chan- σ∈H (e−rx − 1 − rx) nel + Krx(e−rx − 1) ) +∞ k b−1 X k (1 + 2 t) PF, Richmond, generalized 2 k b , Q(2 t) +∞ 1992 [92, 93] digital trees k=0 Y u Q(u) = 1 + 2j j=0 +∞ X −z/Bk Mahmoud, PF, bucket selection z (1 − e ) and relatives Jacquet, Régnier, and sorting k=0 2000 [160, 161] +∞ X −x/2k −2x/2k −xu/2k PF, Fusy, cardinality (e − e )e Gandouet, estimation k=1 Meunier, 2007 [54] −ht X r e Broutin, PF, 2010 height in h (1 − e−ht)2 [13] non-plane h≥1 binary trees TABLE 2. Some of PF’s contributions with their authors, date, and refer- ences, next the topics under consideration, and ﬁnally the harmonic sums that appear in these papers.

Table 2 provides a small sample of PF’s contributions. The third example [36, 37] is impressive: the sum is over the affine transforms σ(z) = µ + rz in a semi-group H = σ , σ ∗ generated by two affine transforms σ (z) = λ + pz, σ (z) = λ + qz { 1 2} 1 2 with p + q = 1. The scope of application is quite broad and we refer to [60, p. 5] for a list of relevant fields.

Exponentials in harmonic sums It is remarkable how often the exponential function appears in the harmonic sums. The reason is the following. In the process of analyzing an algorithm, we are faced with combinatorial sums, which generally are not harmonic sums. But there may be a suitable approximation which is a harmonic sum. There are essentially two rules. The ﬁrst is the approximation of a large power by an exponential,

(1 a)n = e−na 1 + O(na2) with na = nε, 0 < ε < 1/2 − n→+∞ n→+∞ TECHNICAL POINT 55

Here is an example related to the analysis of the radix-exchange sort algorithm [150, p. 131] +∞ n +∞ 1 1 k 1 1 = F (n) + O ,F (x) = (1 e−x/2 ). − − 2k n→+∞ √n − kX=0 kX=0 Others examples can be found in [70, p.188], [84, p. 230], [76, p. 388], [13, p.131], or [125, p. 71]. The second rule is the approximation of the binomial distribution by a Gaussian distribution 2n n k 2 1 − = e−w 1 + O with k = w√n, k = o(n3/4). 2n n→+∞ n n→+∞ n It appears in the study of the expected height of plane (Catalan) trees [27, p. 20] 2n n +∞ n k 1 2 2 d(k) − = G + o(1) G(x) = d(k)e−k x . 2n n→+∞ √n k=1 k=1 X n X or in the study of odd-even merging [179], [85, p. 153] resumed in [46, p. 286] and [194, p. 478].

Technical point For the benefit of the reader who wants to apply the Mellin transform to his/her own problem, we leave for a while the formal style and enter into analysis. Frequently, the original f(x) is not only defined on the real positive axis, but on a sector arg x < | | ω of the complex plane. This constraints the image strongly. In this case, it satisfies (4) f ∗(s) = e−ω| Im(s)|. | | s→±i∞ Such an inequality (not necessarily of exponential type) is the key point allowing to use the inverse formula 1 (5) f(x) = f ∗(s)x−s ds 2πi Z(c) (see the proof of Theorem 4 in [60]). In this formula, (c) is the vertical line at abscissa c taken in the fundamental strip. It is noteworthy that x can be a complex variable and not only a positive real variable, contrary to what we started with ([31] or [140] for a brief account). This is of practical importance since the application of the Mellin transform to a generating function is frequently the first step of an analytical process. It provides the local behaviour of the function at a distinguished point, and can be followed by the use of the Cauchy formula, for example with the saddle-point method [26, 35], or followed by singularity analysis [48, p. 397], [76, p. 238], [13, p. 24]. 56 DR FLAJOLET’S ELIXIR OR MELLIN TRANSFORM AND ASYMPTOTICS

Figure 9.2. The Mellin transform captures oscillating behavior of a very small amplitude.

Oscillations The study of the number of registers necessary to evaluate an expression repre- sented by a binary tree is perhaps the most classical example which provides a harmonic sum. PF and Helmut Prodinger dealt with a variant of the problem in a 1986 paper [80]. They begin by revisiting the case of a binary tree. They need to know the local behaviour of the function 1 u2 E(z) = − v (k)uk u 2 ≥ kX1 in the neighborhood of 1/4. For this, they perform some changes of variables u 1 r z = , u = − , r = √1 4z, u = e−t (1 + u)2 1 + r − (z = 1/4 corresponds to u = 1 and t = 0) and a harmonic sum V (t) comes out. They then compute its Mellin transform, ζ(s) V (t) = v (k)e−kt,V ∗(s) = Γ(s). 2 2s 1 × ≥ kX1 − They collect the coefﬁcients of the asymptotic expansion of V (t) at 0 by a process which seems now to be routine. Because of the denominator 2s 1, there is a line of − poles χk = 2kπi/ ln 2 on the imaginary axis. These poles are regularly spaced and RELATED TOPICS 57 contribute a trigonometric series C + P (log2 t) with respect to log2 t,

1 γ ln(2π) Γ(χk)ζ(χk) C = + − ,P (log t) = e−2kπi log2(t). 4 2 ln 2 2 ln 2 6 kX=0 In this way they obtain the expansion they are looking for 1 1 V (t) = + ln(t) + C + P (log2 t) + O(t), t→0 t 2 ln 2 2 E(z) = 2 + 2r log2 r + (2C + 1)r + 4rP (log2 r) + O(r ), r = √1 4z. z→1/4 − The key point is the occurrence of a function which is 1-periodic with respect to log2 t. The gamma function decreases very rapidly on the imaginary axis and this periodic function therefore has a very small amplitude. (Figure 2 displays the graph of P (log t)). More precisely, the magnitudes are C 0.66, Γ(χ ) 5.5 10−7, 2 ' − | 1 | ' Γ(χ ) 2.5 10−13, Γ(χ ) 1.4 10−19... One could say that this function is so | 2 | ' | 3 | ' small as to be of no importance. But it emphasizes the difﬁculty to obtain such an asymptotic expansion by elementary arguments. This point in particular delighted PF [163, Comment 5, p. 226], [?, p. 8], [70, p. 206].

Related topics In this small introduction to the Mellin transform, we have neglected many issues. Among them, the Rice formula permits to study high order differences of a sequence. Also, the Poisson-Mellin-Newton cycle relates the Poisson generating function of a sequence, its Mellin transform, and the Rice integral. A good reference about these topics is PF and Robert Sedgewick’s 1995 article [107]. Another topic omitted here is the Mellin-Perron formula. It is presented in the next chapter of this volume. Mellin transform is a pivotal ingredient in depoissonization [140, 54]. It is also related with the Lindelöf representation, which appears in [50, Formula (11)], [51, p. 565], itself connected with the magic duality. PF often spoke about this topic, but he has written very little about it. It is alluded to in [55] and developed brieﬂy in a note of Analytic combinatorics [115, p. 238]. The right reference is [154, Chap. V].

PAPER 10 Paper 58

Chapter V

DIVIDE AND CONQUER

Divide-and-Conquer Recurrences and the Mellin-Perron Formula

Yun Kuen Cheung, Mordecai Golin

Deriving the asymptotic behavior of functions described by divide-and-conquer recurrences is often quite easy. Deriving the exact behavior can be difﬁcult, though. Philippe Flajolet and his collaborators helped us see how the use of tools from analytic combinatorics, in particular the Mellin-Perron formula and its variations, permits mechanical derivation of the exact behavior. In particular, these tools make clear the periodic terms that often arise out of such recurrences. He also showed how many number-theoretic (digital sum) functions can be analyzed in the same way.

1. Introduction Divide-and-conquer recurrences model many algorithmic and mathematical problems. The most basic such recurrence is the binary one,

(1) n 2, f = 2f + e , ∀ ≥ n n/2 n where f1 and the en are given. Such a recurrence corresponds physically to solving a problem of size n by (1) splitting the problem into two equal size sub-problems of size n/2;

(2) solving the two subproblems recursively in time 2fn/2 and then;

(3) combining the two subsolutions in time en to derive a solution to the full problem. The canonical example of such a problem is bottom-up list-Mergesort in which a list of n elements is sorted by (1) ﬁrst splitting the list into two lists of size n/2; (2) then recursively sorting the two lists; (3) and ﬁnally merging the two sublists by using a maximum of e = n 1 comparisons. f denotes the worst case number of n − n comparisons needed to sort n items; its initial condition is f1 = 0.

63 64 DIVIDE-AND-CONQUER RECURRENCES AND THE MELLIN-PERRON FORMULA

It is well known, e.g., the Master Theorem [22, p. 73] that

o(n) f = Θ(n) ⇒ n if en =  Θ(n) f = Θ(n log n) .  n  ⇒  Θ(nk), k > 1 f = Θ(nk) ⇒ n  More discriminating versions of this theorem can replace (1) with more general re- n currences of the form fn = af b + en. Furthermore, in many cases these theorems provide exact ﬁrst order asymptotic terms. So, it might seem as if the analysis of such recurrences was completely understood. That they are not is because equations in the form (1) do not completely capture the underlying physical processes. More speciﬁcally, odd sets of size n cannot be split into two equally size subsets. For example, in Mergesort, the original set of size n is split into one subset of size n/2 and the other of size n/2 . Thus (1) must be b c d e replaced by

(2) fn = fbn/2c + fdn/2e + en. Even in the simple Mergesort case of e = n 1 and f = 0 the solution is now more n − 1 complicated, i.e.,

(3) fn = n lg n + nA(lg n) + 1 where 1 lg x log x and A(u) = 1 u 21−{u}, in which x denotes the ≡ 2 − { } − { } fractional part of x; e.g., 2.7 = 2. { } Observe that, by definition, A(x) = A( x ). In particular, this implies { } A(x + 1) = A( x + 1 ) = A( x ) = A(x) { } { } so A(x) is periodic with period 1. Noting that limx↑1 A(x) = 1 = A(0) shows that A(x) is continuous as well. This phenomenon of periodic terms appearing in the solution to divide-and-conquer recurrences is very common. These periodic terms often appear as coefficients of the second order term as in (3), but occasionally they are coefficients of the leading term, e.g., with fn = nA(lg n) + o(n). They are usually continuous but often are also non-differentiable, at least at a countably infinite number of points. As can be imagined, such terms cause difficulty when trying to derive solutions to recurrences. Papers [57, 58] associated with this chapter develop a method, based on the Mellin- Perron formula, for deriving solutions that capture these periodic terms. Because of the method used, these periodic terms are naturally derived in Fourier series form. Paper [62] uses a very similar technique to analyze digital sums. These are number theoretic functions that are summations of functions of the digital representation of integers. The canonical example is the sum-of-digits function. Let v1(n) be the number of “1”s in the binary representation of n, e.g., v1(13) = v1(11012) = 3 and

1. See [58] for a derivation, based on a derivation of a related function in [151, p. 400]. 2. THE BASIC TECHNIQUE 65 v1(24) = v1(110002) = 2. Set fn = i

2. The basic technique The basic technique follows from two observations. Some introductory deﬁni- tions are ﬁrst needed. Start by recalling

f = f f − and ∆g = g g , ∇ n n − n 1 n n+1 − n the backward and forward difference operators on sequences fn and gn. Set wn = ∆ f ∆( f ) to be the double-difference operator. Simple algebra yields ∇ n ≡ ∇ n n−1 (5) f = nf + (n k)∆ f . n 1 − ∇ k kX=1 As noted in [62, Theorem 2.1], the Mellin-Perron Formula 2 states that: THEOREM 1 (Mellin-Perron). Let (s) denote the real part of complex number s. If, < in the complex plane, there is some real c > 0 such that the Dirichlet series W (s) = ∞ wn s converges absolutely for (s) > c , then n=1 n < P n−1 c+i∞ n s ds (6) (n k)wk = W (s)n . − 2iπ c−i∞ s(s + 1) kX=1 Z

2. Theorem 2.1 of [62] actually states a more general version of the Mellin-Perron formula. The version stated here is the special case of m = 2 in [62]. 66 DIVIDE-AND-CONQUER RECURRENCES AND THE MELLIN-PERRON FORMULA

Problem fn deﬁnition fn solution

Worst case Mergesort f + f + n 1 n lg n + nA(lg n) + 1 bn/2c dn/2e −

Average Case Mergesort f + f + n γ n lg n + nB(lg n) + O (1) bn/2c dn/2e − n

Variance of Mergesort f + f + δ n C(lg n) + o(n) bn/2c dn/2e n ·

1 Sum of Digits Function i

v1(i) lg 3 Coefﬁcents in ﬁrst i

Figure 10.1. Representative divide-and-conquer recurrences [58] and digital sums [62] analyzed by papers in this chapter. γn and δn are deﬁned in [58]. v1(n) is number of “1”s in the binary representation of n, e.g., v1(13) = v1(11012) = 3 and v1(24) = v1(110002) = 2. v2(n) is the exponent of 2 in the prime decomposition of n, e.g., v2(13) = v2(11012) = 0 v2(24) = v2(110002) = 3. h(n) evaluates a base 2 number as a base 3 number, i.e., ei ei 2 h ( i 2 ) = i 3 . E.g., h(5) = h(1012) = 3 + 1 = 10 and h(13) = 3 2 h(11012) = 3 + 3 + 1 = 37. All of the A, B, C, D, E, F functions are periodicP with periodP 1 but many of them are not differentiable, at least at a dense set of points.

∞ ∆∇f n Combining (5) and (6) then gives that, if W (s) = n=1 ns converges absolutely for (s) > c, then < P n c+i∞ ds (7) f = nf + W (s)ns . n 1 2iπ s(s + 1) Zc−i∞

A-priori, this doesn’t look helpful; unknown function fn has now been rewritten ns in terms of a complicated integral whose kernel I(s) = W (s) s(s+1) contains a term W (s) which is itself a complicated function of the sequence fn. What makes (7) interesting, though, is the following observation: 2. THE BASIC TECHNIQUE 67

Observation 1: If fn is deﬁned by a divide-and-conquer recurrence or a digital sum then W (s) has a simple form.

More particularly, [57, 58] shows that if fn is deﬁned by the divide-and-conquer recurrence (1) with initial conditions e0 = f0 = e1 = 0 then ∞ Ξ(s) ∆ e (8) W (s) = where Ξ(s) = ∇ n . 1 2−s ns n=1 − X This means that W (s) is now expressed in terms of known sequence en instead of unknown sequence fn. Similarly, [62] shows that for many fn that are deﬁned by digital sums, W (s) can be written in terms of known functions. A canonical example (implicit in the proof of Theorem 3.1 in [62]) is that if fn = i

Note that this W (s) converges absolutely for, e.g., (s) > 3, so we may set c = 3. < Since f1 = 0, (7) implies that f 1 3+i∞ ns ds n = . n 2iπ 1 2−s s(s + 1) Z3−i∞ − By Observation 2, it is equal to the sum of residues at all singularities of ns 1 I(s) = 1 2−s s(s + 1) − in the left half plane (s) < 3, plus, possibly, an asymptotically negligible term. In < this particular case, [58, 57] further show that this error term is zero. We now note that, in that half plane, the singularities of I(s) are (1) A double pole at s = 0 with residue lg n + 1 1 . 2 − log 2 (2) A simple pole at s = 1 with residue 1 . − n (3) Simple poles at s = 2kiπ/ log 2, k Z 0 with residues a e2ikπ lg n: ∈ \{ } k 1 1 2ikπ ak = with χk = . log 2 χk(χk + 1) log 2 The sum of these residues is 1 lg n + A(lg n) + n where A(u) has the explicit Fourier expansion, 1 1 A(u) = a e2ikπu, with a = . k 0 2 − log 2 ∈Z kX Thus

fn = n log2 n + nA(lg n) + 1. It can be veriﬁed that this A(u) is exactly the Fourier series representation of A(u) = 1 u 21−{u} in (3). − { } − Note that the periodic term A(lg n) comes from adding the residues of I(s) at χk, 1 which are the singularities (simple poles) of the factor 1−2−s in I(s). Similarly, in the analysis of the sum of digits function, we have, from (9) that, for fn = i 2. Consider the corresponding < 2s 2 ζ(s) I(s) = − . 2s 1 s(s + 1) − ζ(s)’s only singularity is a simple pole at s = 1 which is cancelled in I(s) by the simple zero of 2s 2 at s = 1. Since ζ(s) has no zeros, Delange’s solution (4) can − now be simply rederived by noting that I(s) has a double pole at s = 0 and simple 3. CONCLUDING REMARKS 69 poles at s = 2kiπ/ log 2, k Z 0 and then summing all of their associated ∈ \{ } residues. Note that in both the examples described above the periodic term in the solution falls directly out of the analysis when adding up residues at the poles s = 2kiπ/ log 2, k Z 0 , because they can be interpreted as the terms of a Fourier series. This ∈ \{ } phenomenon appears in all of the problems addressed by the papers in this chapter.

3. Concluding Remarks This chapter introduced and quickly sketched the Mellin-Perron techniques developed by Philippe Flajolet and his collaborators in [16, 58, 57, 62] for analyzing divide-and-conquer recurrences and digital sums. These techniques are particularly appropriate for deriving the periodic terms that often arise in these problems. The techniques are “recipes”, in that Observations 1 and 2 above are frequently applicable. This chapter glossed over some of the difficulties and extensions that appear in those papers and we urge readers who want more details to read the papers. The most important are – Showing that Observation 2 is correct often requires more work. This usually involves proving that integrals over specific contours are asymptotically negligible. Such proofs are often problem-specific. – Proving convergence of the derived Fourier series can be tricky (and sometimes uniform convergence doesn’t occur). – This chapter only showcased the version of the Mellin-Perron formula that applies to double-summation. There are other versions that can be used for single-summation, triple-summation, etc..

PAPER 11 Paper 115

Chapter VI

COMMUNICATION PROTOCOLS

Flajolet’s Work on Telecommunication Protocols and Collision Resolution Agorithms

Philippe Jacquet

1. Introduction When we consider the contributions of Philippe Flajolet in the telecommunication domain, the first item that come to memory is the wonderful "approximate counting" algorithm [5]. As discussed in a separate chapter, approximate counting alllows to estimate the number n of distinct elements in a multiset with arbitrary order with only a memory of logarithmic size. The simplicity and elegance of this algorithm makes it the ideal tool to prevent cyber-attacks in the Internet, either in web server or in edge routers. Indeed the packet address headers of an internet flow can be seen like a multiset. Too many distinct client at the same time toward the same server is the symptom of a cyber-attack, The algorithm can also be used to distinguish between heavy ses- sions and short session, bringing a very efficient tool to traffic shaping. Anyhow the approximate counting algorithm was not originally invented for telecommunication and this is later on that it was found pertinent to the internet. However Flajolet has worked on many algorithms originally designed for telecommunication during the 90’s (more precisely in the year 1985, annus mirabilis). His favorite field of experimentation was the resolution of collision. Some of his algorithm are now in standard home internet access. Less obvious, some of the collision resolution algorithms analyzed by Flajolet have direct links with tree structure, and this may explain why he found so much interest in them. This chapter is devoted to the review of those algorithms. We first make a short review of what is a protocol, what is a collision in the universe of telecommunication. We will in passing describe the Aloha algorithm as the main originator of this technology. Second we will describe the tree collision resolution algorithm, and describe the main contribution of Flajolet in the analysis of the performance and optimization.

75 76FLAJOLET’S WORK ON TELECOMMUNICATION PROTOCOLS AND COLLISION RESOLUTION AGORITHMS

2. Telecommunication Protocols, Aloha protocol A protocol is a set of rules that a group of computer must follow in order to achieve a common objective. For instance the common objective is to enable an efﬁ- cient way of communication between the computers. The question of designing good telecommunication protocols arisen in the 60’s when the computers started to be or- ganized in networks. In the mid 60’s the university of Hawaii dispatched on several island proposed to use radio links to connect the computers and to send information in packets. The main issue in such networks is that when two computers broadcast their packets at the same time then the data collide. In short a collision makes the packet content to be unintelligible and the data to be lost. There were several proposals to design collision free protocols, but all turned to need to have a centralized control which is not practical in a distributed context. To be fully distributed the schemes needed to solve a chicken egg problems. In 1968 Abramson [1] had the luminous idea that blew away the vicious circle and made the foundation of the Aloha protocol: "instead of avoiding collision, why not just try to resolve them afterward". This idea has a corollary: "when you have packet to send, just transmit it when you want". The collision resolution algorithm of the Aloha protocol (literally "welcome" in Hawaiian) is the simplest of all: "computers randomly schedule the retransmissions of their collided packets". This algorithm, confounding in its simplicity, is indeed the foundation of all modern protocols, such as Ethernet, WiFi. Before going further let’s introduce the basic model of collision resolution algorithm. Time is slotted, time unit is the slot. Computers can transmit at most one packet per slot and transmissions occur at beginning of slot. All computers listen at feedback at end of slots (there are alternative models with delayed feedback, for example in cable TV networks or satellite uplinks [15]). At every slot: – if no contenders: empty slot; – if only one transmitter: successful slot; – if two or more contenders: collision slot. The detection of collision give rise to a great number of variation: it can either rely on an absence of acknowledgement from the intended receiver as with Aloha or WiFi, or by a violation of code or energy level as with Ethernet (IEEE 802.11 [14]).

2.1. Performance of Aloha collision resolution algorithm and the contribution of Philippe Flajolet. Let’s illustrate the above model with the Aloha algorithm. We assume that new packets that are ready for transmission are actually generated according to a Poisson process of rate λ per slot. The fact that it is a Poisson process is the usual way to model telecommunication traffics. Indeed such traffics likely result as the superposition of merely independent and mostly unpredictable processes. We assume that a computer that has a packet for retransmission, will reschedule a retransmission on the current slot is p. This is the simplest rescheduling process, being memoryless, in this case retransmission backoff times follow a geometric distribution. Assume that there are n packets waiting for retransmission then the probability that 3. THE TREE COLLISION RESOLUTION ALGORITHM 77 the current slot is an empty slot is P (empty) = e−λ(1 p)n . − The above identity reads the following way: e−λ is the probability that no new packet is generated for this slot and (1 p)n is the probability that no waiting packet are not − rescheduled on this slot. With the same way of reasoning we also get P (success) = λe−λ(1 p)n + e−λnp(1 p)n−1 , − − which reads exactly as follows: either the unique packet transmitted on this slot is a new packet (with probability λe−λ) and no waiting packet is rescheduled, or no new packet is generated and only one waiting packet is scheduled (probability np(1 − p)n−1). Given the above estimate it is clear that P (collision) 1 when n . Also → → ∞ P (success) 0. Therefore the output of the system decays to zero when the number → of backlogged packets tends to infinity. If N(t) denotes the number of backlogged packet at time t, then we have E(N(t + 1) N(t) N(t) = n) = λ P (success) − | − The average discrepancy lim sup (N(t+ 1) N(t)) > 0 as soon as λ > 0. From n→∞ − this property Fayolle et al. [2] proved via probabilistic methods that E(N(t)) → ∞ when t . In other words the collision resolution of the Aloha protocol as simple → ∞ it is, is unfortunately unstable. The news is not so bad because there is a simple way to stabilize the Aloha protocol. The value of p which maximizes the quantity P (success) is in fact 1 λ p = − N(t) −1 and in this case limn→∞ P (success) = e . With this condition the constrained Aloha protocol is stable as long as λ < e−1. Thus the maximum attainable throughput λmax, with constrained Aloha as a stable system satisfies: −1 λmax(constrained aloha) = e . This number looks like a magic number, the resolution algorithm with the extra cost of empty slots and collision slots, spares a proportion of e−1 slot for successful communication. The main difficulty is how to get a proper estimate of the number N(t) at each time slot in a distributed way. It turns out that it is not needed to have an exact estimate of the quantity N(t), but an approximation that converges to the actual value when n increases. Many solutions based on Bayesian methods have been introduced. Philippe Flajolet et al. [6] proposed a much simpler method based on leader election that allows to stabilize the Aloha algorithm for the magical number.

3. The tree collision resolution algorithm 3.1. Back to the origins. Stabilizing the Aloha protocol consists into introducing non trivial rules that somewhat alleviate the apparent original simplicity of the 78FLAJOLET’S WORK ON TELECOMMUNICATION PROTOCOLS AND COLLISION RESOLUTION AGORITHMS

{A,B,C,D}

{A,B} {C,D} €

€ ∅ {A,B} € {C} {D}

{A} {B} € € € €

€ €

Figure 11.1. A tree collision resolution of an initial collision of four nodes: A, B, C, D { } scheme. Therefore there were a room for defining a simple and stable collision resolution algorithm. This was done by Capetanakis [3] and Vvedenskaya, Tsybakov [4]. In 1979, Capetanakis proposed a scheme based on a recursive tree structure. After each collision a rooted binary tree is created. Every contender indepen- dently tosses a coin: – if ’head", then the contender enters the first subtree; – if "tail", then the contender waits for the resolution of the first subtree and enters the second subtree. Figure 1 illustrates the tree resolution of a subset of four contenders A, B, C, D . { } To find the succession of slots one must read the tree left depth first. In its first version, Capetanakis supposed that a central entity indicates to the contenders the current node in the tree. This looks again like an unacceptable complication for a distributed algorithm. In fact it turns out via the work of Vvedenskaya and Tsybakov that no such central entity is needed. Indeed the later authors introduced in 1980 [4] the stack algorithm described as follow. After each collision in the stack valgorithm, every contenders initializes a counter C(t) and tosses a coin: – if "head" then C(t) = 0; – if "tail" then C(t) = 1. The C(t) evolves the following way 3. THE TREE COLLISION RESOLUTION ALGORITHM 79

– If C(t) = 0 then the computer transmits on the next slot; – If C(t) > 0 and the next slot is a collision, then C(t + 1) = C(t) + 1; – If C(t) > 0 and the next slot is not a collision, then C(t + 1) = C(t) 1. − Figure 2 illustrates the collision resolution of the stack algorithm of the subset A, B, C, D . { } Notice that with a same sequence of coin tossing for each contender, the stack algorithm resolves collision with the same sequence of slots as with the tree algorithm. This analogy is not fortuitous: the stack algorithm is exactly a distributed implementation of the tree algorithm. Stack and tree resolution are the same algorithm.

t 0 1 2 3 4 5 slot feedback coll coll empty coll succ succ C(t) = 0 A, B, C, D A, B A, B A B { }{ } ∅ { }{ }{ } C(t) = 1 - C,D A, B C,D B C,D { }{ }{ }{ }{ } C(t) = 2 -- C,D - C,D - { } { } t 6 7 8 slot feedback coll succ succ C(t) = 0 C,D C D { }{ }{ } C(t) = 1 - D - { } C(t) = 2 ---

Figure 11.2. A stack resolution

3.2. A tree resolution is a trie. Philippe Flajolet made a spectacular break- through in the analysis of the performance of the tree algorithm for a very good reason: a tree resolution is exactly a trie. Assume that you identify each contender by its sequence of coin tossing. This sequence is a binary key. Take the contender A, its binary key is (0, 1, 0,...). It turns out that this binary key is exactly the access path to the leaf that contains the successful transmission of contender A. In other words the tree resolution is exactly the trie of the initial contenders identiﬁed by their coin tossing sequence. Assuming that no new packet is generated during the resolution the average number of slots needed in a collision resolution of n contenders, is exactly the size of a binary trie with n records. Let Ln be this average size, we have L0 = L1 = 1. Since Knuth we know that for n > 1:

k=n −n n L = 1 + 2 (L + L − ) . n k k n k kX=0 80FLAJOLET’S WORK ON TELECOMMUNICATION PROTOCOLS AND COLLISION RESOLUTION AGORITHMS

zn −z Introducing the Poisson generating function, L(z) = n Ln n! e Flajolet obtains the functional equation: P z L(z) = 1 2(1 + z)e−z + 2L( ) . − 2 ∞ Using the Mellin transform L∗(s) = L(x)xs−1dx, deﬁned for 2 < (s) < 1, 0 − < − Flajolet [?] gives a closed form expression R 2(1 + s)Γ(s) L∗(s) = . 1 2s − And it turns out via the singularity analysis of the inverse Mellin transform L(z) = 1 L∗(s)z−sds that occur for c = 1 (see [13] for the technical details): 2iπ <(s)=c − R 2z L(z) = 1 + s Γ(s 1)z−sk + o(z) . log 2 k k − ∈ ∗ ! kXZ 2ikπ with sk = log 2 . Noticing that the contribution of the sk leads to an periodic term (in log z) of amplitude not more than the order 10−6 we have z L(z) = (1 + O(10−6)) . 0.346573 Flajolet could offer the luxury to extend the analysis to the case where the coin toss is biaised by the probabilities (p, q), with p + q = 1: L(z) = 1 2(1 + z) + L(pz) + L(qz) , − and in this case we get 2z L(z) = (1 + r (log z)) p log p q log q 2 − − log p the remaining term either o(1) (in general) or, when log q is rational, function r2(x) periodic with amplitude within 10−6. It turns that the result can also be used to characterize the individual coefﬁcients Ln = L(n) + o(n) by a direct application of depoissonization theorems (unknown at this time).

3.3. Performance analysis of the tree algorithm with blocked access. The above estimates apply for the speciﬁc case when no new packets participate to the resolution of an initial collision. We call this version the "blocked access" tree algorithm. The time is divided in epoch. To detect that the collision is resolved and a that a new collision can start, one just need that each computer manage a counter K(t) as follows: – when the next slot is a collision then K(t + 1) = K(t) + 1; – otherwise K(t + 1) = max 0,K(t) 1 . { − } When K(t) = 0 then the computers transmit the new packets that have been generated during the last epoch. Let `(i) and N(i) respectively denote the length of ith epoch and the number of new packet generated during this epoch. The variable N(i) conditioned 3. THE TREE COLLISION RESOLUTION ALGORITHM 81 by `(i) is Poisson variable of mean λ`(i). In other words P (N(i) = n `(i) = `) = n (λ`) −λ` | n! e . Thus we have the exact identity: (λ`)n E(`(i + 1) `(i) = `) = L e−λ` = L(λ`) . | n n! n X From the asymptotic analysis it turns out that `(i + 1) L(λ`) lim E( `(i) = `) = lim > 1 `→∞ `(i) | `→∞ `

2 −6 when λ > log 2 (1 + O(10 )). In this case the quantity `(i) tends to diverge when i , therefore the tree algorithm is unstable. → ∞ Conversely when λ < 2 (1 O(10−6)). log 2 − `(i + 1) lim E( `(i) = `) < 1 `→∞ `(i) | and the quantity `(i) remains ﬁnite in average when i and the tree algorithm is → ∞ stable. In other words the tree algorithm with blocked access is stable and its maximum achievable throughput satisﬁes the inequality 2 2 (1 O(10−6)) < λ (binary tree) < (1 + O(10−6)) . log 2 − max log 2

There is no exact evaluation of the quantity λmax(binary tree) because of the oscillat- 2iπ −7 −6 ing terms of order Γ log 2 = 5.42 10 < 10 . Thus the estimate λmax = 0.346574 10−6 log p λ is valid. When log q is irrational, the quantity max is exactly 2 ± −p log p−q log q but in this case the value is smaller than λmax in the unbiaised case. And in any case it is smaller than the e−1 of constrained Aloha.

3.4. Q-ary tree algorithm. An obvious question: can we adapt the tree algorithm to Q-ary trees? The answer is yes, see 3, the idea came to Flajolet Mathys [12]; the collision algorithm requires a Q-sided coin toss. After each collision the contenders set C(t) as a random integer between 0 and Q 1 and let it evolve as follow: − – If C(t) = 0 then the computer transmits on the next slot; – If C(t) > 0 and the next slot is a collision, then C(t + 1) = C(t) + Q 1; − – If C(t) > 0 and the next slot is not a collision, then C(t + 1) = C(t) 1. − The function L(z) now satisﬁes the general equation

i=Q−1 L(z) = 1 Q(1 + z)e−z + L(p z) , − i i=0 X where the pi is the probability that the coin tossing produces the integer i after a collision. Or otherwise: Q L∗(s) = (1 + s)Γ(s) , 1 p−s − i i P 82FLAJOLET’S WORK ON TELECOMMUNICATION PROTOCOLS AND COLLISION RESOLUTION AGORITHMS

{A,B,C,D}

{A,B} {C} {D} €

€ ∅ {A}€{ B} €

€ € €

Figure 11.3. A 3-ary tree collision resolution of an initial collision of four nodes: A, B, C, D { } which leads to the asymptotic estimate formally similar to the binary case: Qz L(z) = (1 + r (log z)) . p log p Q − i i i where in general the r (x) is o(1)P when x or is periodic when the log p ’s Q → ∞ i are commensurable. However the amplitude is still of very small order, although slowly increasing with Q. Similarly with the binary case, for a given value Q the 1 minimum is attained for the unbiaised case: pi = Q and the error term (Q) is of 2 order Γ 2iπ exp( π ). Thus log Q ≈ − log Q log Q λ (Q-ary tree) = (1 + (Q)) , max Q

2 with (Q) exp( π ). ≈ − log Q If the quantity Q were not constrained to the integer values, then the maximum −1 value of λmax(Q-ary tree) would be e attained for Q = e. In fact the maximum value for integer Q is 3, the closest integer to e, and λmax(ternary tree) = 0.366204 which falls very short of e−1 = 0.367879 by very few digits. Notice that within at least six digits we have λmax(quaternary tree) = λmax(binary tree) and beyond λmax(Q-ary tree) is a decreasing function of the integer Q. 4. THE FREE ACCESS TREE ALGORITHM 83

The following table in ﬁgure 4, borrowed from [12], summarizes the different values of λmax when Q varies in the uniform coin tossing case.

Q λmax 2 0.346574 4 10−7 ± 3 0.367879 1 10−5 ± 4 0.346574 7 10−4 ± 6 0.298627 4 10−3 ± 8 0.259930 7 10−3 ±

Figure 11.4. Different values of λmax versus Q uniform coin tossing case. blocked access mode

−1 The question remains: can we have a λmax larger than e , or is this last quantity an ultimate limit. For long this was indeed a conjecture, and we will see how the works of Philippe Flajolet answer that e−1 is not an ultimate limit.

4. The free access tree algorithm The previous analysis was made under the implicit hypothesis that the tree algorithm works in blocked access mode. The free access mode exists as well, in this case any newly generated packet is tentatively transmitted in the very next slot of its generation time. The consequence is that new generated packets can participate to the current collision resolution, thus there is no need to manage a global counter K(t). Despite this sensible difference, the algorithm remains the same, i.e. after each collision contenders initialize a counter C(t) which evolves exactly as before.

4.1. Resolution of free access recursion. The analogy with the trie analysis is perfect as long as the new packets have a blocked access. If we leave a free access to new packet, i.e. they are transmitted on the first current slot, then the analysis signif- icantly departs from the trie orthodoxy. Philippe Flajolet developped the analysis as well [9].Although more complicated it can be tackled with some elegance via generating functions. If x and y indicates the number of new arrivals on the slots starting the left and right subtree, for n > 1 the situation is illustrated by figure 5: n −n n L = 1 + P (x)P (y) 2 (L + L − ) . n k k+x n k+y x,y X kX=0 For the analogy with trie it would correspond to the possibility of new insertions via internal node, instead of the usual root insertion. The packet arrival being Poisson of λx −λ mean λ then P (x) = x! e , the generating function L(z) considerably simplifies this equation: z L(z) = 1 2L(λ)(1 + z)e−z L0(λ)ze−z + 2L( + λ) . − − 2 84FLAJOLET’S WORK ON TELECOMMUNICATION PROTOCOLS AND COLLISION RESOLUTION AGORITHMS

Figure 11.5. Free arrival collision resolution

When the coin tossings are biaised we have a sensibly similar equation: L(z) = 1 2L(λ)(1 + z)e−z L0(λ)ze−z + L(pz + λ) + L(qz + λ) . − − How to solve this equation? Let’s rewrite it in (I H)L(z) = 1 2L(λ)f(z) L0(λ)g(z) , − − − with f(z) = (1 + z)e−z, g(z) = ze−z and H is the linear operator deﬁned on analytical function φ(z): Hφ(z) = φ(pz + λ) + φ(qz + λ). Operator I is the identity operator. Inverting I H by the classic formula (I H)−1 = I+H+H2 +... unfortunately − − this operation does not work because of a diverging series. However the resolution comes from from a similar but convergent series with L(z) = 1 2L(λ)Tf(z) L0(λ)Tg(z) , − − with the deﬁnition of the operator T on any analytical function φ(z): Tφ(z) = Hkφ(z) Hkφ(0) z(Hkφ)0(0) − − ≥ kX0 This series converge because if one take the second derivative of L(z) we have the identity (I H00)L00(z) = 2L(λ)f 00(z) L0(λ)g00(z) − − − with H00φ(z) = p2φ(pz + λ) + q2φ(qz + λ). It turns out that the iteration (I H00) = − I + H00 + (H00)2 + converges under mild conditions since p2 + q2 < 1. Therefore ··· L00(z) = 2L(λ) (H00)kf 00(z) L0(λ) (H00)kf 00(z) . − − ≥ ≥ kX0 kX0 z y 00 z y 00 k 00 Using the identity L(z) = 1+ 0 0 L (x)dxdy (no z term) and the fact that 0 0 (H ) φ (x)dxdy = Hkφ(z) Hkφ(z) z(Hkφ)0(0) leads to the claimed result. − − R R R R 4. THE FREE ACCESS TREE ALGORITHM 85

It remains to determine L(λ) and L0(λ) this is done by simple identiﬁcation in:

−1 L(λ) 1 + 2Tf(λ) Tg(λ) 1 =  L0(λ)   2(Tf)0(λ) 1 + (Tg)0(λ)   0        In passing (Tφ)0(z) = (H0)kφ0(z) (H0)kφ0(0). It turns out that the vector k≥0 − L(λ) P is determined as long as the above matrix is not singular, i.e. as long as  L0(λ)    D(λ) = (1 + 2Tf(λ))(1 + (Tg)0(λ)) 2Tg(λ)(Tf)0(λ) = 0 , − 6 If this condition is fulfilled we have 1 + (Tg)0(λ) L(λ) = D(λ) Flajolet could also give the asymptotic estimate of L when n (it turns out to n → ∞ be linear in n with some potential periodic terms), but this tour de force was not really necessary for the estimation of the λmax of the free access tree algorithm. Indeed L(λ) is exactly the average duration of a collision resolution interval with a initial collision of Poisson multiplicity. If the average L(λ) < 0 then the system having ergodic renewal times, is therefore stable. Therefore λmax is the first value of λ that makes L(λ) diverging, ie. λmax is the first non negative root of D(λ):

D(λmax) = 0 . Therefore the quantity max as root of a purely analytic function is known up to an arbitrary accuracy, contrary to the blocked access case:

λmax(free binary tree) = 0.360177 ...

Notice that although being signiﬁcantly larger than the λmax(binary tree) it is smaller than the λmax(ternary tree). We will return to this comment when we will discuss the free Q-ary tree algorithms. The above methodology can be applied to other parameters of interest. For example let Cn be the average number of contenders in the resolution of of collision of initial multiplicity n. We have C0 = 0 and C1 = 1 but when n > 1 we have Cn > n. In fact: n −n n C = P (x)P (y) 2 (C + C − ) , n k k+x n k+y x,y X kX=0 which after some algebra translates into the functional equation: (I H)C(z) = g(z) 2C(λ)f(z) C0(λ)g(z) , − − − zn −z z y 00 with C(z) = n Cn n! e . Since C(z) = z + 0 0 C (x)dxdy it resolves into P C(z) = z + Tg(z) 2C(λ)TfR(zR) C0(λ)Tg(z) − − 86FLAJOLET’S WORK ON TELECOMMUNICATION PROTOCOLS AND COLLISION RESOLUTION AGORITHMS and −1 C(λ) 1 + 2Tf(λ) Tg(λ) λ + Tg(λ) =  C0(λ)   2(Tf)0(λ) 1 + (Tg)0(λ)   1 + (Tg)0(λ)        Thus 1 C(λ) = (1 + (Tg)0(λ))(λ + Tg(λ) Tg(λ)) = λL(λ) . D(λ) − Notice that this identity was predictable since L(λ) is precisely the average renewal period. The average number of packets per renewal period should be λL(λ) in order to keep the arrival rate equal to λ. We will see in the following section that the methodology can be extended to some non trivial case that give insight to the performance of the tree algorithm as a telecommunication system.

4.2. Delivery delay analysis of the free tree algorithm. After having determined the condition of stability of a communication system, the telecom engineers are primarily interested into the packet delivery delay. The delivery delay is the time that separate the generation of the packet and its successful reception by its intended receiver. The system can be stable in general but could be prone to unacceptable large delivery delay. Therefore the analysis of the delivery delay is a key point and this was done in [?] noticeable in the literature as the first full analysis of the performance of a collision resolution algorithm. Let Wn be the average cumulated delays of packets participating to the resolution collision of initial multiplicity n. Clearly W0 = 0, W1 = 1. When n > 1, as illustrated in figure 6 we have the recursion n −n n W = n + P (x)P (y) 2 (W + (n k)L + W − ) . n k k+x − k+x n k+y x,y X kX=0 As an obvious operation we introduce the exponential generating function W (z) = zn −z n Wn n! e which satisfies the functional equation: (1)P (I H)W (z) = z + qzL(pz + λ) qL(λ)g(z) 2W (λ)f(z) W 0(λ)g(z) − − − − The key of the analysis is in the remark that the collision resolution intervals form a renewal sequence and that W (λ) is in fact the average cumulated delays per resolution intervals. Since C(λ) = λL(λ) is the average number of packets involved per collision resolution interval, thus the unconditional average packet delay is W (λ) . C(λ) Using the functional resolution methodology, we get from equation (1) the expression W (z) = z + qTh(z) qL(λ)Tg(z) 2W (λ)Tf(z) W 0(λ)Tg(z) , − − − 5. Q-ARY FREE ACCESS TREE ALGORITHM 87

Figure 11.6. The cumulated delays in a collision resolution with h(z) = zL(pz + λ). Thus −1 W (λ) 1 + 2Tf(λ) Tg(λ) λ + qTh(λ) =  W 0(λ)   2(Tf)0(λ) 1 + (Tg)0(λ)   1 + q(Th)0(λ)  and       (1 + (Tg)0(λ))(λ + qTh(λ) Tg(λ)(1 + q(Th)0(λ)) W (λ) = − . D(λ)

5. Q-ary free access tree algorithm A last and lancinant question remains: is e−1 an actual upper bound of all possible maximum throughput attainable by a collision resolution algorithm? Flajolet with Mathys [?] was able to give an original answer to this question. The answer is in the extension of the free access algorithm to Q-ary free access tree algorithm. To make the analysis fast, the generating function L(z) now satisfies: i=Q−1 L(z) = 1 QL(λ)f(z) L0(λ) + L(p z + λ) , − − i i=0 X or (I H)L(z) = 1 QL(λ)f(z) L0(λ)g(z) , − − − i=Q−1 by appropriately redefining the operator H: Hφ(z) = i=0 φ(piz + λ). This redefinition implies a redefinition of operator T and using the same argument as for P the binary case, we determine λmax as the root of D(λ): D(λ) = 0 with D(λ) = (1 + QTf(λ))(1 + (Tg)0(λ)) QTg(λ)(Tf)0(λ) . − It turns out that the maximum values of λmax are obtained when the coin tossing are uniform, i.e. when i p = 1 . The maximum value is obtained with Q = 3 with ∀ i Q λ = 0.401599 which is larger than e−1 of 10%. This way Flajolet could max ··· disprove the conjecture that e−1 would be an absolute barrier of stability conditions for collision resolution algorithms in basic slotted models. 88FLAJOLET’S WORK ON TELECOMMUNICATION PROTOCOLS AND COLLISION RESOLUTION AGORITHMS

The following table summarizes the different values of λmax when Q varies in the uniform coin tossing case (extracted from [12]).

Q λmax 2 0.360177 ··· 3 0.401599 ··· 4 0.399223 ··· 6 0.373354 ··· 8 0.347002 ···

Figure 11.7. Different values of λmax versus Q uniform coin tossing case. Free access mode Bibliography

[1] Abramson, the ALOHA system, another alternative to computer communications, 1968 [2] Fayolle, Gelenbe, Labetoulle, "Stability and Optimal Control of the Packet Switching Broadcast Chan- nel", JACM, 1977. [3] Capetanakis, "Generalized TDMA: The multi-accessing tree protocol", IEEE Transaction on Commu- nication, 1979. [4] Tsybakov, Vvedenskaya, "Random Multiple-Access Stack Algorithm", Probl. Peredachi Inf., 1980. [5] Flajolet, Nigel "Probabilistic counting algorithms for data base applications" Journal of Computer and System Sciences, 1985 [661 citations] [6] Flajolet, Greenberg, Ladner "Estimating the multiplicities of conﬂicts to speed their resolution in multiple access channels 1987 [77 citations] [7] Massey, "collision resolution algorithms and random access communications", CISM Course and Lec- ture Notes, 1981. [8] Fayolle, Hofri, On the Capacity of a Collision Resolution Channel under Stack based Collision Reso- lution Algorithm 1983 [9] Fayolle, Flajolet, Hofri, On a functional equation arising in the analysis of a protocol for a multi-access broadcast channel, 1982 [69 citations] [10] Tsybakov, "Survey of USSR contributions to random multiple-access communications", IEEE Trans- action on Information Theory, 1985. [11] Fayolle, Flajolet, Hofri, Jacquet, The evaluation of packet transmission characteristics in a multi- access channel with stack collision resolution protocol, IEEE Transaction on Information Theory, 1985 [99 citations] [12] Flajolet Mathys, Q-ary collision resolution algorithms in random-access systems with free or blocked channel access, IEEE Transaction on Information Theory, 1985 [183 citations] [13] Flajolet, Gourdon, Dumas, "Meilin transforms and asymptotics: Harmonic sums", TCS, 1995. [14] Crow, Widjaja, Kim, Sakai, "IEEE 802.11 Wireless Local Area Networks", IEEE Communication Magazine, 1997 [15] Jacquet, Muhlethaler, Robert, "Performant Implementations of Tree Collision Resolution Algorithms for CATV Networks", INRIA-RR 4107, 2001.

PAPER 12 Paper 49

Bibliography

[1]A BRAMOWITZ,M., AND STEGUN,I.A. Handbook of Mathematical Functions. National Bureau of Standards, Applied Mathematics Series 55. U.S. Government Printing Office, Washington, D.C., 1972. 10th printing, with corrections. Reprinted by Dover, with additional corrections. [2]A HO,A., AND CORASICK, M. Efficient String Matching: An Aid to Bibliographic Search. Com- munications of the ACM 18 (1975), 333–340. [3]B ANDERIER,C.,FLAJOLET, P., SCHAEFFER,G., AND SORIA, M. Planar maps and Airy phenomena. In Proceedings of the 27th International Colloquium on Automata, Languages and Pro- gramming (ICALP 2000) (Berlin/Heidelberg, 2000), U. Montanari, J. D. Rolim, and E. Welzl, Eds., vol. 1853 of Lecture Notes in Computer Science, Springer, pp. 388–402. This version is subsumed by [4]. [4]B ANDERIER,C.,FLAJOLET, P., SCHAEFFER,G., AND SORIA, M. Random maps, coalescing sad- dles, singularity analysis, and Airy phenomena. Random Structures & Algorithms 19 (2001), 194– 246. For preliminary versions, see [3, 5]. [5]B ANDERIER,C.,FLAJOLET, P., SCHAEFFER,G., AND SORIA, M. Random maps, coalescing sad- dles, singularity analysis, and Airy phenomena. Technical Report ALCOMFT-TR-01-20, ALCOM Project, 2001. For published version, see [4]. [6]B ARRON,A. Logically Smooth Density Estimation. Ph.d. thesis, Stanford University, 1985. [7]B ASSINO, F., CLÉMENT,J., AND NICODÈME, P. Counting occurrences for a finite set of words: combinatorial methods. To appear in Transactions on Algorithms. [8]B ASSINO, F., CLÉMENT,J., AND NICODÈME, P. Counting occurrences for a finite set of words: Combinatorial methods. ACM Transactions on Algorithms (2012+). Accepted for Publication. [9]B ENDER,E., AND KOCHMAN, F. The distribution of subword counts is usually normal. European Journal of Combinatorics 14 (1993), 265–275. [10]B OURDON,J., AND VALLÉE, B. Generalized pattern matching statistics. In Proceedings of colloquium on Mathematics and Computer Science : Algorithms and Trees (2002), Birkhauser, Trends in Mathematics, pp. 249–265. [11]B OURDON,J., AND VALLÉE, B. Pattern matching statistics on correlated sources. In Proceedings of LATIN’06 (2006), LNCS 3887, pp. 224–237. [12]B ROUTIN,N., AND FLAJOLET, P. The height of random binary unlabelled trees. In Fifth Colloquium on Mathematics and Computer Science (2008), U. Rösler, Ed., vol. AI of DMTCS Proceedings, pp. 121–134. This version is subsumed by [13]. [13]B ROUTIN,N., AND FLAJOLET, P. The distribution of height and diameter in random non-plane binary trees. To appear; arXiv:1009.1515v2, arXiv. 34 pages. For preliminary version, see [12], May 2011. [14]C HEN, L. Poisson approximations for dependent trials. Ann. Proba. 3 (1975), 534–545. [15]C HEN, L. Stein’s method, Palm theory and Poisson process approximation. Ann. Proba. 32 (2004), 2545–2569. [16]C HEUNG, Y. K., FLAJOLET, P., GOLIN,M., AND LEE, C. Y. J. Multidimensional divide-and- conquer and weighted digital sums (extended abstract). In Proceedings of the Sixth Workshop on Analytic Algorithmics and Combinatorics (ANALCO ’09) (2009), C. Martínez and R. Sedgewick, Eds., SIAM, pp. 58–65.

93 94 BIBLIOGRAPHY

[17]C HOMSKY,N., AND SCHÜTZENBERGER, M. The algebraic theory of context-free languages. Com- puter Programming and Formal Languages (1963), 118–161. P. Braffort and D. Hirschberg, eds, North Holland. [18]C LÉMENT,J.,FLAJOLET, P., AND VALLÉE, B. The analysis of hybrid trie structures. Research Report 3295, Institut National de Recherche en Informatique et en Automatique (INRIA), 1997. 9 pages. For published version, see [19]. [19]C LÉMENT,J.,FLAJOLET, P., AND VALLÉE, B. The analysis of hybrid trie structures. In Pro- ceedings of the Ninth Annual ACM–SIAM Symposium on Discrete Algorithms (SODA ’98) (1998), pp. 531–539. For preliminary version, see [18]. [20]C LÉMENT,J.,FLAJOLET, P., AND VALLÉE, B. Dynamical sources in information theory: A general analysis of trie structures. Research Report 3645, Institut National de Recherche en Informatique et en Automatique (INRIA), 1999. 61 pages. For published version, see [21]. [21]C LÉMENT,J.,FLAJOLET, P., AND VALLÉE, B. Dynamical sources in information theory: A general analysis of trie structures. Algorithmica 29 (2001), 307–369. For preliminary version, see [20]. [22]C ORMEN, T. H., LEISERSON,C.E.,RIVEST,R.L., AND STEIN,C. Introduction to Algorithms, Second Edition. The MIT Press and McGraw-Hill Book Company, 2001. [23]C SISZAR,I., AND SHIELDS, P. Redundancy rates for renewal and other processes. IEEE Trans. Information Theory 42 (1996), 2065–2072. [24]D AVISSON, L. D. Universal noiseless coding. IEEE Trans. Information Theory 19 (1973), 783–795. [25] DE BRUIJN, N. A combinatorial problem. Koninklijke Nederlandse Akademie v. Wetenschappen 49 (1946), 167–169. [26]D E BRUIJN, N. G. On Mahler’s partition problem. Indagationes Mathematicae 10 (1948), 210–220. Reprinted from Koninklijke Nederlandsche Akademie van Wetenschappen, Ser. A. [27] DE BRUIJN,N.G.,KNUTH,D.E., AND RICE, S. O. The average height of planted plane trees. In Graph theory and computing. Academic Press, New York, 1972, pp. 15–22. [28] DELA BRIANDAIS, R. File searching using variable length keys. In Papers presented at the the March 3-5, 1959, Western Joint Computer Conference (New York, 1959), IRE-AIEE-ACM ’59 (Western), pp. 295–298. [29]D ELANGE, H. Sur la fonction sommatoire de la fonction somme des chiffres. Enseign. Math. 21, 1 (1975), 31–47. [30]D EVROYE, L. A probabilistic analysis of the height of tries and of the complexity of triesort. Acta Informatica 21 (1984), 229–237. [31]D OETSCH,G. Handbuch der Laplace Transformation, Vol. 1–3. Birkhäuser Verlag, Basel, 1955. [32]D RMOTA,M.,REZNIK, Y., AND SZPANKOWSKI, W. Tunstall code, khodak variations, and random walks. IEEE Trans. Information Theory 56 (2010), 2928–2937. [33]D RMOTA,M., AND SZPANKOWSKI, W. Precise minimax redundancy and regrets. IEEE Trans. In- formation Theory 50 (2004), 2686–2707. [34]D UMAS, P., AND FLAJOLET, P. Asymptotique des récurrences mahlériennes : le cas cyclotomique. Research Report 2163, Institut National de Recherche en Informatique et en Automatique (INRIA), 1994. 23 pages. For published version, see [35]. [35]D UMAS, P., AND FLAJOLET, P. Asymptotique des récurrences mahlériennes : le cas cyclotomique. Journal de Théorie des Nombres de Bordeaux 8 (1996), 1–30. For preliminary version, see [34]. [36]F AYOLLE,G.,FLAJOLET, P., AND HOFRI, M. On a functional equation arising in the analysis of a protocol for a multi-access broadcast channel. Research Report 131, Institut National de Recherche en Informatique et en Automatique (INRIA), 1982. 36 pages. For published version, see [37]. [37]F AYOLLE,G.,FLAJOLET, P., AND HOFRI, M. On a functional equation arising in the analysis of a protocol for a multi-access broadcast channel. Advances in Applied Probability 18 (1986), 441–472. For preliminary version, see [36]. [38]F AYOLLE,G.,FLAJOLET, P., HOFRI,M., AND JACQUET, P. Analysis of a stack algorithm for random multiple-access communication. Research Report 245, Institut National de Recherche en Informatique et en Automatique (INRIA), 1983. 19 pages. For published version, see [39]. [39]F AYOLLE,G.,FLAJOLET, P., HOFRI,M., AND JACQUET, P. Analysis of a stack algorithm for random multiple-access communication. IEEE Transactions on Information Theory IT-31 (1985), 244–254. For preliminary version, see [38]. BIBLIOGRAPHY 95

[40]F AYOLLE, J. An average-case analysis of basic parameters of the suffix-tree. In Mathematics and Computer Science III: Algorithms, Trees, Combinatorics and Probabilities (2004), M. D. et al.. Trends in Mathematics, Ed., Birkhaüser Verlag, pp. 217–227. [41]F AYOLLE, J. An average-case analysis of basic parameters of the suffix tree. In Mathematics and Computer Science III: Algorithms, Trees, Combinatorics and Probabilities, M. Drmota, P. Flajo- let, D. Gardy, and B. Gittenberger, Eds., Trends in Mathematics. Birkhäuser Verlag, Basel, 2004, pp. 217–227. Proceedings of the 3rd International Colloquium of Mathematics and Computer Sci- ence, held at Technische Universität Wien. [42]F LAJOLET, P. Deux problèmes d’analyse d’algorithmes. Séminaire Delange-Pisot-Poitou (Théorie des Nombres) 20 (1978/79), 14-01–14-10. [43]F LAJOLET, P. Methods in the analysis of algorithms: Evaluations of a recursive partitioning process. In Proceedings of the 1983 International Conference on Foundations of Computation Theory (FCT ’83) (Berlin/Heidelberg, 1983), M. Karpinski, Ed., vol. 158 of Lecture Notes in Computer Science, Springer, pp. 141–158. [44]F LAJOLET, P. On the performance evaluation of extendible hashing and trie searching. Acta Infor- matica 20 (1983), 345–369. [45]F LAJOLET, P. Mathematical methods in the analysis of algorithms and data structures. Research Re- port 400, Institut National de Recherche en Informatique et en Automatique (INRIA), 1985. 72 pages. For published version, see [46]. [46]F LAJOLET, P. Mathematical methods in the analysis of algorithms and data structures. In Trends in Theoretical Computer Science, E. Börger, Ed. Computer Science Press, Rockville, Maryland, 1988, ch. 6, pp. 225–304. Volume 12 in the series “Principles of computer science;” Lecture Notes from 1984 for A Graduate Course in Computation Theory, Udine, Italy. For preliminary version, see [45]. [47]F LAJOLET, P. On adaptive sampling. Research Report 1025, Institut National de Recherche en In- formatique et en Automatique (INRIA), 1989. 11 pages. For published version, see [48]. [48]F LAJOLET, P. On adaptive sampling. Computing 43 (1990), 391–400. The article has volume 34 printed at the top, but it is actually contained in volume 43. For preliminary version, see [47]. [49]F LAJOLET, P. Singularity analysis and asymptotics of Bernoulli sums. Research Report 3401, Institut National de Recherche en Informatique et en Automatique (INRIA), 1998. 10 pages. For published version, see [50]. [50]F LAJOLET, P. Singularity analysis and asymptotics of Bernoulli sums. Theoretical Computer Science 215 (1999), 371–381. For preliminary version, see [49]. [51]F LAJOLET, P. Singular combinatorics. In Proceedings of the International Congress of Mathemati- cians (ICM-2002) (2002), L. Tatsien, Ed., vol. III, World Scientific, pp. 561–571. [52]F LAJOLET, P. Calcul, mathématique. In Encyclopædia Universalis. Encyclopædia Univer- salis, 2004. Available electronically at http://www.universalis.fr/encyclopedie/ calcul-mathematique/. [53]F LAJOLET, P. The ubiquitous digital tree. In Proceedings of the 23rd Annual Symposium on The- oretical Aspects of Computer Science (STACS 2006) (Berlin/Heidelberg, 2006), B. Durand and W. Thomas, Eds., vol. 3884 of Lecture Notes in Computer Science, Springer, pp. 1–22. [54]F LAJOLET, P., FUSY,É.,GANDOUET,O., AND MEUNIER, F. Hyperloglog: the analysis of a near- optimal cardinality estimation algorithm. In Proceedings of the 2007 Conference on Analysis of Al- gorithms (AofA ’07) (2007), P. Jacquet, Ed., vol. AH of DMTCS Proceedings, pp. 127–146. [55]F LAJOLET, P., GERHOLD,S., AND SALVY, B. Lindelöf representations and (non-)holonomic sequences. Electronic Journal of Combinatorics 17, #R3 (2010), 1–28. [56]F LAJOLET, P., AND GOLIN, M. Mellin transforms and asymptotics: The mergesort recurrence. Research Report 1612, Institut National de Recherche en Informatique et en Automatique (INRIA), 1992. 11 pages. For published version, see [58]. [57]F LAJOLET, P., AND GOLIN, M. Exact asymptotics of divide-and-conquer recurrences. In Proceed- ings of the 20th International Colloquium on Automata, Languages and Programming (ICALP 93) (Berlin/Heidelberg, 1993), A. Lingas, R. Karlsson, and S. Carlsson, Eds., vol. 700 of Lecture Notes in Computer Science, Springer, pp. 137–149. This version is subsumed by [58]. [58]F LAJOLET, P., AND GOLIN, M. Mellin transforms and asymptotics: The mergesort recurrence. Acta Informatica 31 (1994), 673–696. For preliminary versions, see [57, 56]. 96 BIBLIOGRAPHY

[59]F LAJOLET, P., GOURDON,X., AND DUMAS, P. Mellin transforms and asymptotics: Harmonic sums. Research Report 2369, Institut National de Recherche en Informatique et en Automatique (INRIA), 1994. 55 pages. For published version, see [60]. [60]F LAJOLET, P., GOURDON,X., AND DUMAS, P. Mellin transforms and asymptotics: Harmonic sums. Theoretical Computer Science 144 (1995), 3–58. For preliminary version, see [59]. [61]F LAJOLET, P., GRABNER, P., KIRSCHENHOFER, P., PRODINGER,H., AND TICHY, R. F. Mellin transforms and asymptotics: digital sums. Research Report 1498, Institut National de Recherche en Informatique et en Automatique (INRIA), 1991. 23 pages. For published version, see [62]. [62]F LAJOLET, P., GRABNER, P., KIRSCHENHOFER, P., PRODINGER,H., AND TICHY, R. F. Mellin transforms and asymptotics: digital sums. Theoretical Computer Science 123 (1994), 291–314. For preliminary version, see [61]. [63]F LAJOLET, P., GUIVARC’H, Y., SZPANKOWSKI, W., AND VALLÉE, B. Hidden Pattern Statistics. In Automata, Languages and Programming (2001), Springer Verlag, pp. 152–165. In F. Orejas, P. Spi- rakis, and J. van Leeuwen, editors. Number 2076 in Lecture Notes in Computer Science. Proceedings of the 28th ICALP Conference, Crete, July 2001. [64]F LAJOLET, P., AND JACQUET, P. Analytic models for tree communication protocols. In Flow Control of Congested Networks, A. R. Odoni, L. Bianco, and G. Szegö, Eds., vol. 38 of NATO Advance Science Institute Series, Series F: Computer and Systems Sciences. Springer-Verlag, Berlin/Heidelberg, 1987, pp. 223–234. For preliminary version, see [65]. [65]F LAJOLET, P., AND JACQUET, P. Analytic models for tree communication protocols. Research Re- port 648, Institut National de Recherche en Informatique et en Automatique (INRIA), 1987. 12 pages. For published version, see [64]. [66]F LAJOLET, P., KIRSCHENHOFER, P., AND TICHY, R. F. Deviations from Uniformity in Random Strings. Probability Theory and Rel. Fields 80 (1988), 139–150. [67]F LAJOLET, P., KIRSCHENHOFER, P., AND TICHY, R. F. Discrepancy of Sequences in Discrete Spaces. Algorithms and Combinatorics 8 (1989), 61–70. In G. Halász and V.T. Sós, editors, Irregu- larities of Partitions. [68]F LAJOLET, P., AND MARTIN, G. N. Probabilistic counting. In Proceedings of the 24th Annual Symposium on Foundations of Computer Science (1983), IEEE Computer Society Press, pp. 76–82. This version is subsumed by [70]. [69]F LAJOLET, P., AND MARTIN, G. N. Probabilistic counting algorithms for data base applications. Research Report 313, Institut National de Recherche en Informatique et en Automatique (INRIA), 1984. 26 pages. For published version, see [70]. [70]F LAJOLET, P., AND MARTIN, G. N. Probabilistic counting algorithms for data base applications. Journal of Computer and System Sciences 31 (1985), 182–209. For preliminary versions, see [68, 69]. [71]F LAJOLET, P., NEBEL,M., AND PRODINGER, H. The scientiﬁc works of Rainer Kemp (1949– 2004). Theoretical Computer Science 355 (2006), 371–381. [72]F LAJOLET, P., AND ODLYZKO, A. Exploring binary trees and other simple trees. In Proceedings of the 21st Annual Symposium on Foundations of Computer Science (1980), IEEE Computer Society Press, pp. 207–216. This version is subsumed by [74]. [73]F LAJOLET, P., AND ODLYZKO, A. The average height of binary trees and other simple trees. Re- search report, Institut National de Recherche en Informatique et en Automatique (INRIA), 1981. 55 pages. For published version, see [74]. [74]F LAJOLET, P., AND ODLYZKO, A. The average height of binary trees and other simple trees. Journal of Computer and System Sciences 25 (1982), 171–213. For preliminary versions, see [72, 73]. [75]F LAJOLET, P., AND ODLYZKO, A. Singularity analysis of generating functions. Research Report 826, Institut National de Recherche en Informatique et en Automatique (INRIA), 1988. 25 pages. For published version, see [76]. [76]F LAJOLET, P., AND ODLYZKO, A. Singularity analysis of generating functions. SIAM Journal on Discrete Mathematics 3 (1990), 216–240. For preliminary version, see [75]. [77]F LAJOLET, P., PELLETIER,M., AND SORIA, M. On Buffon machines and numbers. In Proceedings of the Twenty-Second Annual ACM–SIAM Symposium on Discrete Algorithms (SODA ’11) (2011), pp. 172–183. BIBLIOGRAPHY 97

[78]F LAJOLET, P., AND PRODINGER, H. Register allocation for unary-binary trees. Research Report 266, Institut National de Recherche en Informatique et en Automatique (INRIA), 1984. 17 pages. For published version, see [80]. [79]F LAJOLET, P., AND PRODINGER, H. Level number sequences for trees. Research Report 484, Insti- tut National de Recherche en Informatique et en Automatique (INRIA), 1986. 7 pages. For published version, see [81]. [80]F LAJOLET, P., AND PRODINGER, H. Register allocation for unary-binary trees. SIAM Journal on Computing 15 (1986), 629–640. For preliminary version, see [78]. [81]F LAJOLET, P., AND PRODINGER, H. Level number sequences for trees. Discrete Mathematics 65 (1987), 149–156. For preliminary version, see [79]. [82]F LAJOLET, P., AND PUECH, C. Partial match retrieval of multidimensional data. Research Report 233, Institut National de Recherche en Informatique et en Automatique (INRIA), 1983. 45 pages. For published version, see [84]. [83]F LAJOLET, P., AND PUECH, C. Tree structures for partial match retrieval. In Proceedings of the 24th Annual Symposium on Foundations of Computer Science (1983), IEEE Computer Society Press, pp. 282–288. This version is subsumed by [84]. [84]F LAJOLET, P., AND PUECH, C. Partial match retrieval of multidimensional data. Journal of the ACM 33 (1986), 371–407. For preliminary versions, see [83, 82]. [85]F LAJOLET, P., AND RAMSHAW, L. A note on Gray code and odd-even merge. SIAM Journal on Computing 9 (1980), 142–158. [86]F LAJOLET, P., RAOULT,J.-C., AND VUILLEMIN, J. On the average number of registers required for evaluating arithmetic expressions. In Proceedings of the 18th Annual Symposium on Foundations of Computer Science (1977), IEEE Computer Society Press, pp. 196–205. This version is subsumed by [87]. [87]F LAJOLET, P., RAOULT,J.-C., AND VUILLEMIN, J. The number of registers required for evaluating arithmetic expressions. Theoretical Computer Science 9 (1979), 99–125. For preliminary version, see [86]. [88]F LAJOLET, P., RÉGNIER,M., AND SEDGEWICK, R. Some uses of the Mellin integral transform in the analysis of algorithms. In Combinatorial Algorithms on Words, A. Apostolico and Z. Galil, Eds., vol. 12 of NATO Advance Science Institute Series, Series F: Computer and Systems Sciences. Springer-Verlag, Berlin/Heidelberg, 1985, pp. 241–254. Proceedings of the NATO Advanced Re- search Workshop on Combinatorial Algorithms on Words held at Maratea, Italy. For preliminary version, see [89]. [89]F LAJOLET, P., RÉGNIER,M., AND SEDGEWICK, R. Some uses of the mellin integral transform in the analysis of algorithms. Research Report 398, Institut National de Recherche en Informatique et en Automatique (INRIA), 1985. 14 pages. For published version, see [88]. [90]F LAJOLET, P., RÉGNIER,M., AND SOTTEAU, D. Algebraic methods for trie statistics. Research Re- port 298, Institut National de Recherche en Informatique et en Automatique (INRIA), 1984. 32 pages. For published version, see [91]. [91]F LAJOLET, P., RÉGNIER,M., AND SOTTEAU, D. Algebraic methods for trie statistics. In Analysis and Design of Algorithms for Combinatorial Problems, G. Ausiello and M. Lucertini, Eds., vol. 25 of Annals of Discrete Mathematics. North-Holland, 1985, pp. 145–188. Papers based on the workshop held at the International Centre for Mechanical Sciences (CISM) in Udine, Italy. Equivalently cited as North-Holland Mathematics Studies, volume 109. For preliminary version, see [90]. [92]F LAJOLET, P., AND RICHMOND, B. Generalized digital trees and their difference-differential equations. Research Report 1423, Institut National de Recherche en Informatique et en Automatique (INRIA), 1991. 15 pages. For published version, see [93]. [93]F LAJOLET, P., AND RICHMOND, B. Generalized digital trees and their difference-differential equations. Random Structures & Algorithms 3 (1992), 305–320. For preliminary version, see [92]. [94]F LAJOLET, P., ROUX,M., AND VALLÉE, B. Digital trees and memoryless sources: from arithmetics to analysis. In 21st International Meeting on Probabilistic, Combinatorial, and Asymptotic Methods in the Analysis of Algorithms (AofA ’10) (2010), M. Drmota and B. Gittenberger, Eds., vol. AM of DMTCS Proceedings, pp. 233–260. 98 BIBLIOGRAPHY

[95]F LAJOLET, P., AND SAHEB, N. Digital search trees and the generation of an exponentially distributed variate. In Proceedings of the 8th Colloquium on Trees in Algebra and Programming (CAAP ’83) (Berlin/Heidelberg, 1983), G. Ausiello and M. Protasi, Eds., vol. 159 of Lecture Notes in Computer Science, Springer, pp. 221–235. This version is subsumed by [96]. [96]F LAJOLET, P., AND SAHEB, N. The complexity of generating an exponentially distributed variate. Journal of Algorithms 7 (1986), 463–488. For preliminary versions, see [95, ?]. [97]F LAJOLET, P., AND SALVY, B. Euler sums and contour integral representations. Research Report 2917, Institut National de Recherche en Informatique et en Automatique (INRIA), 1996. 23 pages. For published version, see [98]. [98]F LAJOLET, P., AND SALVY, B. Euler sums and contour integral representations. Experimental Math- ematics 7 (1998), 15–35. For preliminary version, see [97]. [99]F LAJOLET, P., SALVY,B., AND ZIMMERMANN, P. Automatic average-case analysis of algorithms. Research Report 1233, Institut National de Recherche en Informatique et en Automatique (INRIA), 1990. 69 pages. For published version, see [100]. [100]F LAJOLET, P., SALVY,B., AND ZIMMERMANN, P. Automatic average-case analysis of algorithms. Theoretical Computer Science 79 (1991), 37–109. For preliminary version, see [99]. [101]F LAJOLET, P., AND SEDGEWICK, R. Digital search trees revisited. Research Report 311, Institut National de Recherche en Informatique et en Automatique (INRIA), 1984. 20 pages. For published version, see [102]. [102]F LAJOLET, P., AND SEDGEWICK, R. Digital search trees revisited. SIAM Journal on Computing 15 (1986), 748–767. For preliminary version, see [101]. [103]F LAJOLET, P., AND SEDGEWICK, R. The average case analysis of algorithms: Complex asymptotics and generating functions. Research Report 2026, Institut National de Recherche en Informatique et en Automatique (INRIA), 1993. 100 pages. Preliminary material for the book [115]. [104]F LAJOLET, P., AND SEDGEWICK, R. The average case analysis of algorithms: Counting and generating functions. Research Report 1888, Institut National de Recherche en Informatique et en Au- tomatique (INRIA), 1993. 116 pages. Preliminary material for the book [115]. [105]F LAJOLET, P., AND SEDGEWICK, R. The average case analysis of algorithms: Saddle point asymptotics. Research Report 2376, Institut National de Recherche en Informatique et en Automatique (INRIA), 1994. 55 pages. Preliminary material for the book [115]. [106]F LAJOLET, P., AND SEDGEWICK, R. Mellin transforms and asymptotics: Finite differences and Rice’s integrals. Research Report 2231, Institut National de Recherche en Informatique et en Au- tomatique (INRIA), 1994. 21 pages. For published version, see [107]. [107]F LAJOLET, P., AND SEDGEWICK, R. Mellin transforms and asymptotics: Finite differences and Rice’s integrals. Theoretical Computer Science 144 (1995), 101–124. For preliminary version, see [106]. [108]F LAJOLET, P., AND SEDGEWICK, R. The average case analysis of algorithms: Mellin transform asymptotics. Research Report 2956, Institut National de Recherche en Informatique et en Automa- tique (INRIA), 1996. 93 pages. [109]F LAJOLET, P., AND SEDGEWICK, R. The average case analysis of algorithms: Multivariate asymptotics and limit distributions. Research Report 3162, Institut National de Recherche en Informatique et en Automatique (INRIA), 1997. 123 pages. Preliminary material for the book [115]. [110]F LAJOLET, P., AND SEDGEWICK, R. Analytic combinatorics: Functional equations—rational and algebraic functions. Technical Report ALCOMFT-TR-01-8, ALCOM Project, 2001. 98 pages. Pre- liminary material for the book [115]. [111]F LAJOLET, P., AND SEDGEWICK, R. Analytic combinatorics: Functional equations, rational and algebraic functions. Research Report 4103, Institut National de Recherche en Informatique et en Automatique (INRIA), 2001. 98 pages. Preliminary material for the book [115]. [112]F LAJOLET, P., AND SEDGEWICK, R. Analytic combinatorics: Symbolic combinatorics. Technical Report ALCOMFT-TR-02-123, ALCOM Project, 2002. 186 pages. Preliminary material for the book [115]. [113]F LAJOLET, P., AND SEDGEWICK, R. Analytic combinatorics: Basic complex asymptotics. Techni- cal Report ALCOMFT-TR-03-100, ALCOM Project, 2003. 133 pages. Preliminary material for the book [115]. BIBLIOGRAPHY 99

[114]F LAJOLET, P., AND SEDGEWICK,R. Analytic Combinatorics. Cambridge University Press, 2009. [115]F LAJOLET, P., AND SEDGEWICK,R. Analytic Combinatorics. Cambridge University Press, 2009. xiv+810 pages. Also available electronically from the authors’ home pages. For preliminary versions, see [104, 103, 105, 109, 111, 110, 112, 113]. [116]F LAJOLET, P., AND SOTTEAU, D. A recursive partitioning process of computer science. In Proceed- ings of the Second World Conference on Mathematics at the Service of Man (Las Palmas, Canary Islands, Spain, 1982), A. Ballester, D. Cardús, and E. Trillas, Eds., Universidad Politécnica de Las Palmas, pp. 25–30. [117]F LAJOLET, P., AND STEYAERT, J.-M. A branching process arising in dynamic hashing, trie searching and polynomial factorization. In Proceedings of the 9th International Colloquium on Automata, Languages and Programming (ICALP 82) (Berlin/Heidelberg, 1982), M. Nielsen and E. M. Schmidt, Eds., vol. 140 of Lecture Notes in Computer Science, Springer, pp. 239–251. [118]F LAJOLET, P., AND SZPANKOWSKI, W. Analytic variations on redundancy rates of renewal processes. Research Report 3553, Institut National de Recherche en Informatique et en Automatique (INRIA), 1998. 10 pages. For published version, see [120]. [119]F LAJOLET, P., AND SZPANKOWSKI, W. Analytic variations on the redundancy rate of renewal processes. In Proceedings of the 2000 IEEE International Symposium on Information Theory (2000), IEEE Information Theory Society, p. 499. This version is subsumed by [120]. [120]F LAJOLET, P., AND SZPANKOWSKI, W. Analytic variations on redundancy rates of renewal processes. IEEE Transactions on Information Theory 48 (2002), 2911–2921. For preliminary versions, see [119, 118]. [121]F LAJOLET, P., SZPANKOWSKI, W., AND VALLÉE, B. Hidden Word Statistics. Journal of the ACM 53, 1 (January 2006), 147–183. [122]F LAJOLET, P., AND VALLÉE, B. Continued fractions, comparison algorithms, and fine structure constants. In Constructive, experimental, and nonlinear analysis (Providence, RI, USA, 2000), M. A. Théra, Ed., vol. 27 of Conference Proceedings, Canadian Mathematical Society, American Mathe- matical Society, pp. 53–82. For preliminary versions, see [123, 124]. [123]F LAJOLET, P., AND VALLÉE, B. Continued fractions, comparison algorithms, and fine structure constants. Research report, Institut National de Recherche en Informatique et en Automatique (IN- RIA), 2000. 30 pages. For published version, see [122]. [124]F LAJOLET, P., AND VALLÉE, B. Continued fractions, comparison algorithms, and fine structure constants. Technical Report ALCOMFT-TR-01-1, ALCOM Project, 2001. For published version, see [122]. [125]F LAJOLET, P., AND VEPSTAS, L. On differences of zeta values. Journal of Computational and Applied Mathematics 220 (2008), 58–73. [126]F REDKIN, E. Trie memory. Communications of the ACM 3 (1960), 490–499. [127]G OULDEN,I., AND JACKSON, D. An inversion theorem for clusters decompositions of sequences with distinguished subsequences. J. London Math. Soc. 2, 20 (1979), 567–576. [128]G OULDEN,I., AND JACKSON,D. Combinatorial Enumeration. John Wiley, 1983. New-York. [129]G UIBAS,L., AND ODLYZKO, A. Maximal prefix-synchronized codes. SIAM J. Appl. Math. 35 (1978), 401–418. [130]G UIBAS,L., AND ODLYZKO, A. Periods in strings. J. Combin. Theory A, 30 (1981), 19–42. [131]G UIBAS,L., AND ODLYZKO, A. Strings overlaps, pattern matching, and non-transitive games. J. Combin. Theory A, 30 (1981), 108–203. [132]H AN,G., AND MARCUS, B. Asymptotics of the input-constrained binary symmetric channel capacity. Annals of Applied probability 19 (2009), 1063–1091. [133]H EUBERGER, C. Hwang’s Quasi-Power-Theorem in Dimension Two. Quaest. Math. 30 (2007), 507–512. [134]H WANG,H.-K. Théorèmes limites pour les structures combinatoires et les fonctions arithmétiques. PhD thesis, École polytechnique, Palaiseau, France, Dec. 1994. [135]H WANG, H.-K. Large deviations for combinatorial distributions I: Central limit theorems. Ann in Appl. Probab. 6 (1996), 297–319. [136]H WANG, H.-K. On convergence rates in the central limit theorems for combinatorial structures. European J. Combinatorics 19 (1998), 329–343. 100 BIBLIOGRAPHY

[137]J ACQUET, P., SEROUSSI,G., AND SZPANKOWSKI, W. On the entropy of a hidden markov process. In Data Compression Conference (2004), vol. IEEE, pp. 362–371. [138]J ACQUET, P., AND SZPANKOWSKI, W. Autocorrelation on words and its applications. Analysis of sufﬁx trees by string-ruler approach. Journal of Combinatorial Theory A66 (1994), 237–269. [139]J ACQUET, P., AND SZPANKOWSKI, W. Asymptotic behavior of the lempel-ziv parsing scheme and digital search trees. Theoretical Computer Science 144 (1995), 161–197. [140]J ACQUET, P., AND SZPANKOWSKI, W. Analytical depoissonization and its applications. Theor. Comput. Sci. 201, 1-2 (1998), 1–62. [141]J ACQUET, P., AND SZPANKOWSKI, W. Markov types and minimax redundancy for markov sources. IEEE Trans. Information Theory 50 (2004), 1393–1402. [142]J ACQUET, P., AND SZPANKOWSKI, W. Noisy constrained capacity for bsc. IEEE Trans. Information Theory 56 (2010), 5412–5423. [143]K ALPATHY,R.,MAHMOUD,H.M., AND WARD, M. D. Asymptotic properties of a leader election algorithm, 2011. [144]K EMP, R. The average number of registers needed to evaluate a binary tree optimally. Acta Inform. 11, 4 (1978/79), 363–372. [145]K IEFFER,J.,FLAJOLET, P., AND YANG, E.-H. Data compression via binary decision diagrams. In Proceedings of the 2000 IEEE International Symposium on Information Theory (2000), IEEE Information Theory Society, p. 296. See also unpublished memoir [146]. [146]K IEFFER,J.,FLAJOLET, P., AND YANG, E.-H. Universal lossless data compression via binary decision diagrams. Unpublished memoir, 2000. 21 pages. For preliminary version, see [145]. [147]K LEENE, S. C. Representation of events in nerve nets and ﬁnite automata. In Automata Studies (1956), pp. 3–40. C.E Shannon and J. McCarthy, editors, Princeton, N.J. [148]K NESSL,C., AND SZPANKOWSKI, W. On the Number of Full Levels in Tries. Random Structures & Algorithms 25 (2004), 247–276. [149]K NUTH,D.,MORRIS,J., AND PRATT, V. Fast pattern matching in strings. SIAM journal of computing (1977), 323–350. [150]K NUTH,D.E. The art of computer programming. Volume 3. Addison-Wesley Publishing Co., Read- ing, Mass.-London-Don Mills, Ont., 1973. Sorting and searching, Addison-Wesley Series in Com- puter Science and Information Processing. [151]K NUTH,D.E. The Art of Computer Programming, Volume I: Fundamental Algorithms, 2nd Edition. Addison-Wesley, 1973. [152]K NUTH,D.E. The Art of Computer Programming, 2nd ed., vol. 3. Addison-Wesley, Reading, MA, 1998. [153]K ONTOYIANNIS, I. An implementable lossy version of the lempel-ziv algorithm — part i : Optimal- ity for memoryless sources. IEEE Trans. Information Theory 45 (1999), 2285–2292. [154]L INDELÖF,E. Le calcul des résidus et ses applications à la théorie des fonctions. Les Grands Clas- siques Gauthier-Villars. [Gauthier-Villars Great Classics]. Éditions Jacques Gabay, Sceaux, 1989. Reprint of the 1905 original. [155]L ONARDI,S.,SZPANKOWSKI, W., AND WARD, M. Error resilient lz’77 data compression: Algo- rithms, analysis, and experiments. IEEE Trans. Information Theory 53 (2007), 1799–1813. [156]L ONARDI,S.,SZPANKOWSKI, W., AND WARD, M. D. Error resilient LZ’77 data compression: algorithms, analysis, and experiments. IEEE Transactions on Information Theory 53 (2007), 1799– 1813. [157]L OTHAIRE,M. Applied Combinatorics on Words. Encyclopedia of Mathematics. Cambridge Uni- versity Press, 2005. [158]L OUCHARD,G., AND SZPANKOWSKI, W. On the average redundancy rate of the lempel-ziv code. IEEE Trans. Information Theory 43 (1997), 2–8. [159]L UCZAK, T., AND SZPANKOWSKI, W. A suboptimal lossy data compression based in approximate pattern matc hing. IEEE Trans. Information Theory 43 (1997), 1439–1451. [160]M AHMOUD,H.,FLAJOLET, P., JACQUET, P., AND RÉGNIER, M. Analytic variations on bucket selection and sorting. Research Report 3399, Institut National de Recherche en Informatique et en Automatique (INRIA), 1998. 22 pages. For published version, see [161]. BIBLIOGRAPHY 101

[161]M AHMOUD,H.,FLAJOLET, P., JACQUET, P., AND RÉGNIER, M. Analytic variations on bucket selection and sorting. Acta Informatica 36 (2000), 735–760. For preliminary version, see [160]. [162]M ATHYS, P., AND FLAJOLET, P. Q-ary collision resolution algorithms in random-access systems with free or blocked channel access. Research Report 334, Institut National de Recherche en Infor- matique et en Automatique (INRIA), 1984. 90 pages. For published version, see [163]. [163]M ATHYS, P., AND FLAJOLET, P. Q-ary collision resolution algorithms in random-access systems with free or blocked channel access. IEEE Transactions on Information Theory IT-31 (1985), 217– 243. For preliminary version, see [162]. [164]M ORRISON, D. R. Patricia-practical algorithm to retrieve information coded in alphanumeric. JACM 15, 4 (1968), 514–534. [165]N ICODÈME, P. Regexpcount, a symbolic package for counting problems on regular expressions and words. Fundamenta Informaticae 56, 1-2 (2003), 71–88. [166]N ICODÈME, P., SALVY,B., AND FLAJOLET, P. Motif Statistics. In Algorithms, ESA’99 (1999), J. Nešestril, Ed., pp. 194–211. Volume 1643 of Lecture Notes in Computer Science. [167]N ICODÈME, P., SALVY,B., AND FLAJOLET, P. Motif Statistics. Theoretical Computer Science 287, 2 (2002), 593–618. [168]N OONAN,J., AND ZEILBERGER, D. The Goulden-Jackson Method: Extensions, Applications and Implementations. J. of Difference Equations and Applications 5, 4-5 (1999), 355–377. [169]P ITTEL, B. Asymptotical growth of a class of random trees. Annals of Probability 13 (1985), 414– 427. [170]P RODINGER, H. How to select a loser. Discrete Mathematics 120 (1993), 149–159. [171]R ABIN,M., AND SCOTT, D. Finite automata and their decision problems. IBM J. Res. Develop. (1959), 114–225. [172]R ÉGNIER, M. A unified approach to word occurrences probabilities. Discrete Applied Mathematics, Special issue on Computational Biology 104, 1 (2000), 259–280. http://algo.inria.fr/regnier/publis/Regnier00.ps. [173]R ÉGNIER,M., AND DENISE, A. Statistiques extrêmes sur les mots. In XXXV-èmes Journées de Statistique (2003), vol. 2, pp. 799–802. In Proceedings XXXV-èmes Journées de Statistique, Lyon. [174]R ÉGNIER,M., AND DENISE, A. Rare events and conditional events on random strings. DMTCS 6, 2 (2004), 191–214. [175]R ÉGNIER,M., AND SZPANKOWSKI, W. On the approximate pattern occurrences in a text. In SE- QUENCES ’97: Proceedings of the Compression and Complexity of Sequences 1997 (Washington, DC, USA, 1997), IEEE Computer Society, pp. 253–264. [176]R ÉGNIER,M., AND SZPANKOWSKI, W. On Pattern Frequency Occurrences in a Markovian Se- quence. Algorithmica 22, 4 (1998), 631–649. [177]R ISSANEN, J. Fisher information and stochastic complexity. IEEE Trans. Information Theory 42 (1996), 40–47. [178]S AVARI, S. Redundancy of the lempel-ziv incremental parsing rule. IEEE Trans. Information Theory 43 (1997), 9–21. [179]S EDGEWICK, R. Data movement in odd-even merging. SIAM J. Comput. 7, 3 (1978), 239–272. [180]S EDGEWICK,R., AND FLAJOLET, P. An Introduction to the Analysis of Algorithms. Addison- Wesley Publishing Company, 1996. 512 pages. [181]S HTARKOV, Y. Universal sequential coding of single messages. Problems of Information Transmis- sion 23 (1987), 175–186. [182]S TEIN, C. A bound for the error in the normal approximation to the distribution of a sum of dependent random variables. In Proc. Sixth Berkeley Symp. Math. Statis. Probab. (1972), vol. 2, Univ. California Press, Berkeley, pp. 583–602. [183]S ZPANKOWSKI, W. Asymptotic properties of data compression and suffix trees. IEEE Trans. Infor- mation Theory 39 (1993), 1647–1659. [184]S ZPANKOWSKI, W. A generalized suffix tree and its (un)expected asymptotic behaviors. SIAM J. Computing 22 (1993), 1176–1198. 102 BIBLIOGRAPHY

[185]S ZPANKOWSKI, W. Towards analytical information theory: recent results on Lempel-Ziv data compression schemes (Seminar 18). In Algorithms Seminar, 1996–1997, B. Salvy, Ed., no. Research Report 3267. Institut National de Recherche en Informatique et en Automatique (INRIA), 1997, pp. 75–80. Summary by Philippe Flajolet. [186]S ZPANKOWSKI, W. On asymptotics of certain recurrences arising in universal coding. Problems of Information Transmission 34 (1998), 55–61. [187]S ZPANKOWSKI, W. Asymptotic average redundancy of huffman (and other) block codes. IEEE Trans. Information Theory 46 (2000). [188]S ZPANKOWSKI, W. Average Case Analysis of Algorithms on Sequences. John Wiley & Sons, New York, 2001. [189]S ZPANKOWSKI, W., AND VERDU, S. Minimum expected length of ﬁxed-to-variable lossless compression without preﬁx constraints. IEEE Trans. Information Theory 57 (2011), 4017–4025. [190]S ZPANKOWSKI, W., AND WEINBERGER, M. Minimax pointwise redundancy for memoryless models over large alphabets. IEEE Trans. Information Theory 58 (2012). [191]T HUE, A. Skrifter udgivne of Videnskabs-Selskabet i Christiania. Mathematisk-Naturvidenskabelig Klasse (1912). Reprinted in Thue’s Selected Mathematical Papers (Oslo: Universitetsforlaget, 1977), pages 413–477. [192]V ALLÉE,B.,CLÉMENT,J.,FILL,J.A., AND FLAJOLET, P. The number of symbol comparisons in QuickSort and QuickSelect. In Proceedings of the 36th International Colloquium on Automata, Lan- guages and Programming: Part I (ICALP 2009) (Berlin/Heidelberg, 2009), S. Albers, A. Marchetti- Spaccamela, Y. Matias, S. Nikoletseas, and W. Thomas, Eds., vol. 5555 of Lecture Notes in Computer Science, Springer, pp. 750–763. [193]V ITTER,J.S., AND FLAJOLET, P. Average-case analysis of algorithms and data structures. Re- search Report 718, Institut National de Recherche en Informatique et en Automatique (INRIA), 1987. 67 pages. For published version, see [194]. [194]V ITTER,J.S., AND FLAJOLET, P. Average-case analysis of algorithms and data structures. In Hand- book of Theoretical Computer Science, J. van Leeuwen, Ed., vol. A: Algorithms and Complexity. North-Holland, Amsterdam, 1990, ch. 9, pp. 431–524. For preliminary version, see [193]. [195]X IE,Q., AND BARRON, A. Minimax redundancy for the class of memoryless sources. IEEE Trans. Information Theory 43 (1997), 647–657. [196]Y ANG,E., AND KIEFFER, J. On the performance of data compression algorithms based upon string matching. IEEE Trans. Information Theory 44 (1998), 47–65. [197]Z IV,J., AND LEMPEL, A. A universal algorithm for sequential data compression. IEEE Transactions on Information Theory IT-23 (1977), 337–343. INDEX

We will provide a very comprehensive index. If you have suggestions for words that you deﬁnitely want placed into the index, please feel welcome to contact Mark by email: [email protected] You are welcome to contact your volume editor or Mark at any time about any questions regarding the Collected Works of Philippe Flajolet. We are delighted to have your participation. The editorial team wants to maintain the highest standards of quality, to honor the memory of our Philippe in all ways.

103