Nonlinear P´olya Urn Models and Self-Organizing Processes Tong Zhu A Dissertation in Mathematics Presented to the Faculties of the University of Pennsylvania in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy 2009 Robin Pemantle Supervisor of Dissertation Tony Pantev Graduate Group Chairperson Acknowledgments First and foremost, I am deeply grateful to my advisor Robin Pemantle. His patient guidance and kind support made this work possible. I am thankful for the chance to study with him and appreciate all the help he has given to me for all these years. Thank you Robin! I also want to thank Herman Gluck, Herbert Wilf and Tony Pantev for all the help they have given to me during my graduate study. I am so lucky to get your support. A lot of thanks to Janet Burns, Monica Pallanti and Paula Scarborough, and everyone in the Math Department. Thank you for all the help and making the department like a family. Last, but not least, I would like to express my deepest gratitude to my parents. Without their unconditional love and support, I would not be able to do what I have accomplished. ii ABSTRACT Nonlinear P´olya Urn Models and Self-Organizing Processes Tong Zhu Robin Pemantle, Advisor We consider a class of nonlinear P´olya urn models, which are applied to model self-organizing processes. The first two chapters are devoted to a survey of urn models, theories and methods. In the third chapter, we study the nonlinear urns, which show monopoly properties. We found the asymptotic distributions of the minority color and the attraction time. We showed the asymptotic behaviors of the final monopoly stage. Our main method is the exponential embedding. We have also found a simple probabilistic way to prove combinatorial identities involving symmetric functions. In the last part, we introduce the applications of P´olya urn models and the future studies. iii Contents 1 Introduction 1 1.1 Basic Models and Notations . 2 1.2 Classifications of GPU Models . 5 2 Methods and Theories 7 2.1 Exchangeability . 7 2.2 Martingale Method and Stochastic Approximation . 13 2.2.1 Martingale Method . 13 2.2.2 Stochastic Approximation . 17 2.3 Embedding into Multitype Continuous Markov Branching Process (MCMBP) . 34 2.4 Some Other Methods and Results . 43 3 A Close Look at the Monopoly Processes 51 3.1 Exponential Embedding Method . 52 3.2 Summation of independent exponential random variables . 54 iv 3.2.1 Probability proof for some combinatorial identities . 58 3.3 Monopoly . 72 3.4 The Number of Balls of The Minority Color . 74 3.5 Attraction time . 92 3.6 Large Time Minority Has Little Chance to Win Back . 95 3.7 The Decay/Growth is Slow . 98 4 Applications of GPU Models 101 4.1 Reinforced Random Walk (RRW) . 101 4.2 Statistics . 104 4.3 In Computer Science . 108 4.4 In Clinical Trials . 112 4.5 In Biology . 115 4.6 In Social Science . 116 4.6.1 In Psychology: . 116 4.6.2 In Economics: . 117 5 Further Development 118 v Chapter 1 Introduction Urn models are a group of simple and useful probabilistic models ([68], [69]). One significant star in this family is the P´olya urn model and its generalizations. In 1923, Eggenberger and P´olya ([39]) first introduced the original Eggenberger- P´olyaurn model (usually simplified as P´olyaurn) to study the contagious diseases. Through the past 80 years, people have generalized this model in many different ways. These models are commonly named generalized P´olyaurn models (GPU). Not only have the GPU models been widely developed in theories, but they have also been applied to numerous fields, for example, statistics, computer science, biomedicine and economics, etc. 1 1.1 Basic Models and Notations A single-urn GPU model involves an urn containing balls of different colors (or types). At each discrete time (some generalizations consider continuous time inter- vals), some balls are added or removed according to probabilities depending on the composition of balls of different colors at the time. Let (Ω; F; P) be a probability space. The randomness we need on this space is defined by a sequence of IID random variables uniform on [0; 1], say fUnk : n ≥ 1; k ≥ 1g. Let Fn (n ≥ 1) be the σ-field generated by fUik : 1 ≤ 1 ≤ n; k ≥ 1g and F0 be the trivial one. In most cases, fUn1g is enough, hence simplified as fUng. For an event A 2 F, the indicator function of A is 8 > <> 1 x 2 A 1A(x) := > :> 0 x2 = A: d Let Z+ denote the d-dimensional nonnegative integer space. For any vector d r = (r1; :::; rd) 2 Z+, define jrj = r1 + ::: + rd. Let R(n) = (R1(n); :::; Rd(n)) be a d homogeneous Markov chain with state space Z+. In GPU models, Ri(n) represents the number of balls of color i at time n. Define the normalized vector of R(n): 8 > R(n) < jR(n)j if jR(n)j 6= 0 X(n) := > :> 0 if jR(n)j = 0; d−1 which shows the composition of colors in the urn. Denote ∆ := f(x1; :::; xd) 2 d Pd R : i=1 xi = 1; and xi ≥ 0 for all ig to be the standard (d − 1)-simplex. Hence, 2 d−1 d d X(n) 2 ∆ . Let Υ : Z+ × Z+ ! [0; 1] be the transition kernel of R(n), i.e. Υ(r; r0) = P(R(n + 1) = r0jR(n) = r). In most GPU models, the transition kernel Υ depends only on X(n). In this dissertation, we refer fR(n)g to be a generalized (P´olya)urn process. Associated with fR(n)g, there are some other random processes. Let fξng be the drawn color sequence, with ξn 2 f1; :::; dg and P(ξn+1jFn) := µ(X(n)). Also define δ(n) = (δ1(n); :::; δd(n)) := ν(X(n)) to be the adding sequence, i.e. δi(n) := Ri(n + 1) − Ri(n), showing the replacement schemes, which could be either time-homogeneous or inhomogeneous. In the following, we introduce some examples of GPU models: Original P´olya Urn: Initially, the urn has R1(0) balls of color 1 and R2(0) balls of color 2. Each time, one ball is drawn out uniformly at random and returned to the urn with another ball of the same color. Formally, at time n or after the n-th operation, Ri(n) P(ξn+1 = ijFn) = ; for i = 1; 2: R1(n) + R2(n) And δi(n + 1) = 1fξn+1=ig. Friedman's Urn: In 1949, B. Friedman generalized the P´olya urn model in such a way that, each time one ball is drawn from the urn at random, it is returned to the urn with α balls of the drawn color, as well as β balls of the opposite color. Hence, the distribution of ξn+1 is the same as in the original P´olya urn. And 3 δi(n + 1) = α1fξn+1=ig + β1fξn+16=ig. This gives a matrix form to the replacement scheme, sometimes called a replacement matrix, which is important in determining the behavior of the urn process, 0 1 α β B C A := B C : @ A β α The replacement scheme can also be time-inhomogeneous. Generally, suppose the replacement matrix is A(n) = (aij(n))i;j=1;:::;d: Then the adding sequence can be represented by (δ1(n + 1); :::; δd(n + 1)) = (1fξn+1=1g; :::; 1fξn+1=dg) · A(n) GPU with Urn Functions: In many cases, the measure µ(Xn) = Xn. But for some generalizations, the draws depend on some urn function. This was first introduced in [57]. An urn function F(X(n)) = (F1(X1(n)); :::; Fd(Xd(n))) is a function defined on the (d − 1)-simplex F : ∆d ! ∆d: In other words, P(ξn+1 = ijFn) = Fi(Xi(n)). 4 1.2 Classifications of GPU Models Given a lot of generalizations, it is good to classify them. In any given iteration of the urn process, it is not hard to see that the drawing and replacing schemes play the key role. Hence, we may classify GPU models based on µ(Xn) and ν(Xn). For example, the replacement matrix An could be deterministic or random; a de- terministic one could be classified by different forms (see for example [44]); even eigenvalues determine the probability behaviors (see for example [66]). Remark 1.2.1. In Johnson and Kotz's book [68], they have pointed out that "The way in which the replacement is modified determines the particular subclass of distributions arising. In particular, the kind of replacement depends on the results of the sampling." (pp.176). See their book for more details. These types of classifications are sometimes more or less physical. More essen- tially, classifying GPU models based on some characteristics is another idea, like (partial) exchangeability, which we will discuss more in later chapters, or asymp- totic behaviors. But on the other hands, these are results of the physical operations. Knowing their relations might be another important job. For example, Pouyanne classified some GPU according to their asymptotic behaviors, as well as discussed the eigenvalue of the matrices. Outline of this dissertation is as follows. In Section 2, we introduce several methods to study GPU models and their main results. Section 3 is devoted to the 5 discussion of one special type of GPU processes, which shows monopoly properties. Applications of GPU models are introduced in Section 4. And finally in Section 5, we will list some open problems.