IIE Transactions (2005) 37, 651–656 Copyright C “IIE” ISSN: 0740-817X print / 1545-8830 online DOI: 10.1080/07408170590948512 Relationships among some distributions

WHEYMING TINA SONG Department of Industrial Engineering and Engineering Management, National Tsing Hua University, Hsinchu, Taiwan, Republic of China E-mail: [email protected], wheyming [email protected]

Received March 2004 and accepted August 2004

The purpose of this paper is to graphically illustrate the parametric relationships between pairs of 35 univariate distribution families. The families are organized into a seven-by-five matrix and the relationships are illustrated by connecting related families with arrows. A simplified matrix, showing only 25 families, is designed for student use. These relationships provide rapid access to information that must otherwise be found from a time-consuming search of a large number of sources. Students, teachers, and practitioners who model random processes will find the relationships in this article useful and insightful.

1. Introduction 2. A seven-by-five matrix

An understanding of probability concepts is necessary Figure 1 illustrates 35 univariate distributions in 35 if one is to gain insights into systems that can be rectangle-like entries. The row and column numbers are modeled as random processes. From an applications labeled on the left and top of Fig. 1, respectively. There are point of view, univariate probability distributions pro- 10 discrete distributions, shown in the first two rows, and vide an important foundation in probability theory since 25 continuous distributions. Five commonly used sampling they are the underpinnings of the most-used models in distributions are listed in the third row. For example, the practice. is indexed as R3C1, which indicates Univariate distributions are taught in most probability Row3and Column 1. The range, probability mass (density) and courses in schools of business, engineering, function, mean, and variance for discrete and continuous and science. A figure illustrating the relationships among distributions are summarized in Tables 1 and 2, respectively. univariate distributions is useful to indicate how distributi- Distributions are ordered alphabetically in Tables 1 and 2. ons correspond to one another. Nakagawa and Yoda According to the index number shown in the second col-

Downloaded By: [CDL Journals Account] At: 21:26 29 October 2008 (1977), Leemis (1986), and Kotz and van Dorp (2004) of- umn of Tables 1 and 2, users can easily find the distribution fer diagrams of relations among univariate distributions, required in Fig. 1. but their diagrams are not formatted in a matrix form, In Fig. 1, the distribution name, parameters and range so it is difficult for the instructor to refer to any partic- are shown in each entry.The parameters adopted satisfy the ular distribution from the diagram, and it might be dif- following conventions: n and k are integers; 0 ≤ p ≤ 1; a is ficult for users to find the required distribution quickly. the minimum; b is the maximum; µ is the expected value; Twoversions of the figure presented in this article over- m is either the median or mode; σ is the standard devi- come this shortcoming. Figure 1 is the complete figure with ation; θ is the location parameter; β and γ are scale pa- 35 univariate distributions, whereas Fig. 2 is a simpler ver- rameters; α, α1,α2 are shape parameters; and υ,υ1,υ2 are sion with 25 distributions designed for more elementary degrees of freedom. We do not always use α to denote the needs. shape parameter. For example, the parameter m (denoted Some useful references for studying univariate distribu- as the mode) in the Triangular (a, m, b); the parameter k tions can be found in Patil, Boswell, Joshi, and Ratnaparkhi (denoted as an integer) in the Erlang (k,β); υ (denoted as (1985), Patil, Boswell, and Ratnaparkhi (1985), Stigler the degrees of freedom) in the Chi-Squared (υ); and υ1,υ2 (1986), Hald (1990), Johnson et al. (1993, 1994, 1995), (both denoted as the degrees of freedom) in the F(υ1,υ2) Stuart and Ord (1994), Wimmer and Altmann (1999), are shape parameters. Evans, Hastings and Peacock (2000), and Balakrishnan Relations between entries are indicated with a dashed and Nevzorov (2003). These books individually do not arrow or solid arrow. A dashed arrow shows asymptotic re- provide all of the simple relationships shown in this lations, and a solid arrow shows transformations or special article. cases. The X is used for all distributions.

0740-817X C 2005 “IIE” 652 Song Downloaded By: [CDL Journals Account] At: 21:26 29 October 2008

Fig. 1. Relationships among 35 distributions. (An arrow beginning and ending at the same rectangle indicates that it remains in the same distribution family, but the parameter values might change.)

Forexample, the arrow from the Normal (R3C1) to the In Fig. 1, if more than one random variable is involved Standard Normal (R3C2) indicates that subtracting the to form a transformation, the relationship between these mean from a Normal and dividing by its standard devi- random variables is denoted by “iid” (independent and ation yields a Standard Normal distribution. identically distributed), “indep” (independent), or “k-dim Relationships among some univariate distributions 653

Table 1. Discrete distributions

Distribution Index Probability Expected name no. Range mass function value Variance

Bernoulli (p) R1C3 x = 0, 1 px(1 − p)1−x pp(1 − p) , = , ,..., n x − n−x − Binomial (n p) R2C3 x 0 1 nCxp (1 p) np np(1 p) = , ,..., / n / n 2/ − n / 2 Discrete R1C5 x x1 x2 xn 1 n i=1 xi n i=1 xi n ( i=1 xi n) Uniform (x1,...,xn) Equal-Spaced R2C5 x = a, a + c, 1/n (a + b)/2 c2(n2 − 1)/12 , , + ,..., = + b−a Uniform (a b c) a 2c b where n 1 c Geometric (trials) (p) R2C1 x = 1, 2,... p(1 − p)x−1 1/p (1 − p)/p2 , , = { − + , } K N−K / N − N−n Hyper-Geometric (N K n) R1C2 x max n N K 0 , Cx Cn−x Cn np np(1 p)( N−1 ) ...,min{K, n} where p = K/N , = , + ,... x−1 k − x−k / − / 2 Negative Binomial (k p) R2C2 x k k 1 Ck−1 p (1 p) k pk(1 p) p CK CN−K + + − Negative R1C1 x = k,..., k−1 x−k K−k+1 k(N 1) k(N 1)(N K) (K + 1 − k) CN N−x+1 K+1 (K+1)2(K+2) Hyper-Geometric (N, K, k) N − K + k x−1 Poisson (µ) R2C4 x = 0, 1, 2,... µx exp(−µ)/x! µµ − − + β Polya (n, p,β) R1C4 x = 0, 1, 2,...,nCn x 1(p + jβ) np np(1 p)(1 n ) x j=0 1+β · n−x−1 − + β k=0 (1 p k ) · n−1 + β −1 [ i=0 (1 i )] n = / − Note: Cm n! [m!(n m)!]. Downloaded By: [CDL Journals Account] At: 21:26 29 October 2008

Fig. 2. Relationships among 25 distributions. 654 Song

Table 2. Continuous distributions

Index Probability Expected Distribution name no. Range density function value Variance Arc-Sine R4C5 [0,1] [π x(1 − x)]−1 1/21/8 α − α −  α +α 1 1 − 2 1 α α α α ,α ( 1 2)x (1 x) 1 1 2 Beta ( 1 2) R5C4 [0,1] 2 (α1)(α2) α1+α2 (α1+α2) (α1+α2+1) θ,β R {βπ + x−θ 2 }−1 Cauchy ( ) R4C4 [1 ( β ) ] NA NA υ −1 − / υ R+ x 2 e x 2 υ υ Chi-Squared ( ) R3C5 (υ/2)2υ/2 2 − ,β R+ xk 1 exp(−x/β) β β2 Erlang (k ) R6C3 βk (k−1)! k k Exponential (β) R6C2 R+ β−1 exp(−x/β) ββ2 Extreme Value (θ,γ) R4C1 R γ −1 exp[−(x − θ)/γ ] θ + 0.57722γγ2π 2/6 · exp{− exp[−(x − θ)/γ ]} υ +υ 1 2 υ /2 υ / − 2 ( )(υ /υ ) 1 ( 1 2) 1 υ 2υ (υ +υ −2) υ ,υ R+ 2 1 2 x 2 ,v > 2 1 2 ,v > F( 1 2) R3C4  υ /  υ / −1 (υ +υ )/2 υ − 2 2 υ υ − 2 υ − 2 4 ( 1 2) ( 2 2) (1+υ1υ x) 1 2 2 2 1( 2 2) ( 2 4) α− 2 α, β R+ (x/β) 1exp(−x/β) αβ αβ 2 Gamma ( ) R5C3 β(α) β 1 +α  2 +α 2 1 +α + α α α − α ( α 1) ( α 1) ( α 1) R 2 x 1 2 1 − x 2 2 β2{ 2 − 2 } Generalized Gamma R5C2 ( ) exp[ ( ) ] 2 (α1)β β β (α1) (α1)  (α1) (α1,α2,β) Laplace (µ, β) R7C1 R (2β)−1 exp(−|x − µ|/β) µ 2β2 µ, β R exp[−(x−µ)/β] µπ2β2/ Logistic ( ) R7C2 β{1+exp[−(x−µ)/β]}2 3 −α −1 + α2 exp(−α1)x 2 Loglogistic (α ,α ) R7C4 R −α δ exp(−η)csc(δ) δ[exp(−2η)] 1 2 [1+exp(−α )x 2 ]2 1 · [tan(δ) − δ]csc2(δ) π α δ = ,η = 1 where α α √ 2 2 ,α R+ α π −1 − 1 ln(x/m) 2 α2/ 2 α2 Lognormal (m ) R4C2 (x 2 ) exp[ 2 ( α ) ] m exp( 2) m exp( ) · [exp(α2) − 1] − / x−µ 2 Normal (µ, σ 2) R3C1 R exp[( √1 2)( σ ) ] µσ2 2πσ θ,α θ,∞ αθα/ α+1 αθ ,α > αθ2 ,α > Pareto ( ) R7C5 [ ) x α−1 1 (α−1)2(α−2) 2 √ √ β R+ /β2 − /β 2 β π/ β2 − β π 2 Rayleigh ( ) R6C1 (2x )exp[ (x ) ] 2 ( 2 ) Rectangular (a, b) R6C5 [a, b](b − a)−1 (a + b)/2(b − a)2/12 Standard Cauchy R4C3 R [π(1 + x2)]−1 NA NA (θ = 0,β = 1) R exp(−x) π 2/ Standard Logistic R7C3 [1+exp(−x)]2 0 3 (µ = 0,β = 1) Downloaded By: [CDL Journals Account] At: 21:26 29 October 2008 Standard Normal R3C2 R √1 exp(−x2/2) 0 1 2π (µ = 0,σ = 1) Standard Uniform R6C4 [0,1] 1 1/21/12 (a = 0, b = 1)  υ+ / 2 T(υ) R3C3 R [( 1)√ 2] (1 + x )−(υ+1)/2 0, υ>1 υ , υ>2 (υ/2) πυ υ υ−2 2(x−a) , ≤ ≤ − − a x m Triangular (a, m, b) R5C5 [a, b] (m a)(b a) 1 (a + m + b) 1 {a2 + m2 + b2 2(b−x) , < ≤ 3 18 (b−a)(b−m) m x b − (am + ab + mb)} α, β R+ αβ −α α−1 − /β α β + 1 β2 + 2 Weibull ( ) R5C1 x exp[ (x ) ] (1 α ) (1 α ) − β + 1 2 [ (1 α )]

Note: R+ denotes [0, ∞) and R denotes (−∞, ∞). NA denotes “not applicable.”

normal” (k-dimensional Normal Distribution). For exam- indep,” which indicates that the sum of independent Chi- ple, the relation from the Geometric (R2C1) to the Negative Squared random variables yields a Chi-Squared distribu- k tion. The relation from the Normal (R2C1) to the Normal is Binomial (R2C2) is marked “ i=1 Xi, iid,” which indicates k that the sum of k iid Geometric random variables yields marked “ i=1 Xi, k-dim normal,” which indicates that the a Negative . The relation from Chi- sum of k random variables has a Normal distribution if they Squared (R3C5) back to the Chi-Squared is marked “ Xi, are components of a k-dimensional Normal distribution. Relationships among some univariate distributions 655

The multivariate Normal is discussed in many places, in- 3. A simplified matrix cluding Johnson et al. (1997), Johnson and Wichern (1998), and Kotz et al. (2000). Figure 2 is a simpler version of Fig. 1, not including the The transformation relationships in Fig. 1 can be Negative Hypergeometric (R1C1), Polya (R1C4), Extreme combined to form other relationships. For instance, a Value (R4C1), Standard Cauchy (R4C3), Cauchy (R4C4), path from the Standard Normal (R3C2) to the Chi- Arc-Sine (R4C5), Generalized Gamma (R5C2), and the five Squared (R3C5) to the Gamma (R5C3) to the Ex- distributions in Row 7. In a probability introductory course, ponential (R6C2) indicates that the random variable we have found that students are motivated to learn the trans- 2 + 2 β = X1 X2 has the Exponential ( 2) distribution if formation of random variables and uses of the moment X1 and X2 are independent Standard Normal random generating functions, so that they can derive all 44 relations variables. in Fig. 2. Discussions on the transformation of random There is a special relationship between the Standard Uni- variables and moment generating functions are typically form (R6C4) and any continuous distribution. That is, optional in standard probability and statistics texts, such FX (X) ∼ Uniform(0, 1), where FX is the cumulative distri- as Walpole et al. (2002). To be able to prove all relations bution function (cdf) of any continuous random variable in Fig. 1, some additional mathematical background is X. Therefore, an arrow with the relation FX (X) could be required. drawn from any continuous distribution to the Standard We add two distributions in the first row of Fig. 2: the −1 Uniform, and an arrow with the relation FX (U)(where Geometric (failures) distributions and the Standard Dis- U ∼ Uniform (0,1)) could be drawn from the Standard Uni- crete Uniform (with range 1, 2,...,n), which is a special form to any continuous distribution where the random vari- case of the Equal-Spaced Uniform. The Geometric (fail- −1 ures) random variable is the total number of failures before able FX (U)isknown as the transformation of inverse-cdf, which is a popular random variate generation technique. the first success occurs. The Geometric (R2C1) in Fig. 1 Random variate generation is discussed in many places, now becomes Geometric (trials) in which the random vari- including Schmeiser (1980), Devroye (1986) and Gentle able indicates the number of trials needed until the first suc- (2003). cess occurs. To avoid confusion with index numbers used in Let us continue the discussion of the relationship be- Tables 1 and 2, we do not list row and column numbers in tween the Standard Uniform and any continuous distri- Fig. 2. bution. We do not draw arrows connecting the Standard Uniform U with all continuous distributions X in Fig. 1 because that would render the diagram unnecessarily dif- Acknowledgements ficult to read. In Fig. 1, we simply draw arrows from the Standard Uniform to certain continuous distributions. For This paper is based upon work supported by the Na- example, we show the arrows from the Standard Uni- tional Science Council under grant NSC-90-2218-E-007- form to the (R6C2), Rectangle 018. Many people contributed information that was of (R6C5), Loglogistic (R7C4), Logistic (R7C2), and Pareto value in the preparation of this paper and for that I am (R7C5). The relations from the Standard Uniform to some most appreciative. In particular, the author sincerely thanks Downloaded By: [CDL Journals Account] At: 21:26 29 October 2008 other continuous distributions, including the Triangular professors Bruce Schmeiser and David Goldsman for their (R5C5), Standard Logistic (R7C4), and Rayleigh (R6C1), perceptive comments. Further, the author deeply thanks such that the closed-form inverse cdf exists, are not shown Ph.D. students Mingchang Chih and Sn-Long Liu for their in Fig. 1 because of space limitations. These relations can very helpful proofreading. be easily derived via the combined paths, as discussed earlier. We mark some entries to show the relationship between References the corresponding discrete and continuous distributions. To indicate that the (R2C1) corre- Balakrishnan, N. (1992) The Handbook of the Logistic Distribution, sponds to the Exponential distribution (R6C2), we mark a Marcel Dekker, New York, NY. circle in the bottom of these two entries. To indicate that the Balakrishnan, N. and Nevzorov V.B. (2003) A Primer on Statistical Dis- Negative Binomial distribution (R2C2) corresponds to the tributions,Wiley, New York, NY. Devroye, L. (1986) Non-Uniform Random Variate Generation, Springer- Erlang (R6C3), we mark two circles in the bottom of these Verlag, New Yo rk , N Y. two entries. To indicate that the Equal-Spaced Uniform Evans, M., Hastings, N. and Peacock, B. (2000) Statistical Distributions, (R2C5) corresponds to the Continuous Uniform (R6C5), Wiley, New York, NY. we mark a rectangle in the bottom of these two entries. Gentle, J.E. (2003) Random Number Generation and Monte Carlo Meth- One final comment is that the Logistic, Standard Logis- ods, Springer-Verlag, New York, NY. Hald, A. (1990) A History of Probability and Statistics and Their Appli- tic, and Loglogistic are analogous to the Normal, Standard cations Before 1750,Wiley, New York, NY. Normal, and Lognormal, respectively. See Balakrishnan Johnson, N.L., Kotz, S. and Balakrishnan, N. (1994) Continuous Univari- (1992, p. 190) for a detailed explanation. ate Distributions—1, 2nd eds., Wiley, New York, NY. 656 Song

Johnson, N.L., Kotz, S. and Balakrishnan, N. (1995) Continuous Univari- Wimmer, G. and Altmann, G. (1999) Thesaurus of Univariate Dis- ate Distributions—2, 2nd eds., Wiley, New York, NY. crete Distribution Probability Distributions,STAMM Verlag, Essen, Johnson, N.L., Kotz, S. and Balakrishnan, N. (1997) Discrete Multivariate Germany. Distributions,Wiley, New York, NY. Johnson, N.L., Kotz, S. and Kemp, W.W. (1993) Univariate Discrete Dis- tributions, 2nd eds., Wiley, New York, NY. Johnson and Wichern (1998) Applied Multivariate Statistical Analysis, Prentice-Hall, Englewood Cliffs, NJ. Biography Kotz, S., Balakrishnan, N. and Johnson, N.L. (2000). Continuous Multi- variate Distributions, 2nd eds., Wiley, New York, NY. Wheyming Tina Song is a Professor in the Department of Indus- Kotz, S. and Jogn Renev´ an Dorp (2004) Beyond Beta—Other Continu- trial Engineering at the National Tsing Hua University in Taiwan. ous families of Distributions with Bounded Support and Applications, She received her undergraduate degree in Statistics and Master’s de- World Scientific, NJ, p. 251. gree in Industrial Engineering at Cheng-Kung University in Taiwan Leemis, M.L. (1986) Relationships among common univariate distribu- in 1979. She then received Master’s degrees in Applied Mathemat- tions. The American Statistician, 40(2), 143–145. ics in 1983 and Industrial Engineering in 1984, both from the Uni- Nakagawa, T. and Yoda, H. (1977) Relationships among distributions. versity of Pittsburgh. She received her Ph.D. from the School of In- IEEE Transactions on Reliability, 26(5), 352–353. dustrial Engineering at Purdue University in 1989. She joined Tsing Patil, G.P., Boswell, M.T., Joshi, S.W. and Ratnaparkhi, M.V. (1985) Hua in 1990 after spending a year as a Visiting Assistant Professor at Discrete Continuous Models, International Co-operative Publishing Purdue. She received the outstanding university-wide teaching award House, Burtonsville, MD. from the National Tsing Hua University in 1993. Her research hon- Patil, G.P., Boswell, M.T. and Ratnaparkhi, M.V. (1985) Univariate Con- ors include the 1988 Omega Rho student paper award, the 1990 IIE tinuous Models, International Co-operative Publishing House, Bur- Doctoral Dissertation award, and the 1996 distinguished research award tonsville, MD. from the National Science Council of the Republic of China. Her re- Schmeiser, B.W. (1980) Random variate generation: a survey, in Simula- search interests are in applied operations research; probability and statis- tion with Discrete Models: A State-of-the-Art View,Oren, T.I., Shub, tics; and the statistical aspects of stochastic simulation, including in- C.M. and Roth, P.F. (eds). Institute of Electrical and Electronic En- put modeling, output analysis, variance reduction, and ranking and gineers, New York, NY, pp. 79–104. selection. Stigler, S.M. (1986) The History of Statistics, Cambridge, MA: The Balknap Press. Walpole, R.E., Myers, R.H., Myers, S.L. and Ye, K. (2002) Probability & Statistics for Engineering and Scientists, 7th ed., Prentice Hall, Upper Saddle River, NJ. Contributed by the Simulation Department Downloaded By: [CDL Journals Account] At: 21:26 29 October 2008