1 Appendix: Common Distributions
Total Page:16
File Type:pdf, Size:1020Kb
1 Appendix: Common distributions This chapter provides details for common univariate and multivariate distributions, in- cluding de…nitions, moments, and simulation. Many distributions can be parameterized in di¤erent ways. Deyroye (1986) provides a complete treatments of random number genera- tion, although care must be taken to insure the parameterizations are consistent. Uniform A random variable X has a uniform distribution on the interval [ ; ], denoted [ ; ] ; if the probability density function (pdf) is U 1 p (x ; ) = (1) j for x [ ; ] and 0 otherwise. The mean and variance of a uniform random variable 2 + ( )2 are E (x) = 2 and var (x) = 12 , respectively. The uniform distribution plays a foundational role in random number generation. In particular, uniform random numbers are required for the inverse transform simulation method, accept-reject algorithms, and the Metropolis algorithm. Fast and accurate pre-programmed algorithms are available in most statistical software packages and programming languages. Bernoulli A random variable X 0; 1 has a Bernoulli distribution with parameter , denoted 2 f g X er () if the probability mass function is B x 1 x Prob (X = x ) = (1 ) . (2) j The mean and variance of a Bernoulli random variable are E(X) = and var(X) = (1 ), respectively. To simulate X er (), B 1. Draw U (0; 1) U 2. Set X = 1 if U > . 1 Binomial A random variable X 0; 1; :::; n has a binomial distribution with parameters n 2 f g and , denoted, X in (n; ), if the probability mass function is B n! x n x Prob (X = x n; ) = (1 ) , (3) j x!(n x)! where n! = n (n 1)! = n (n 1) 2 1. The mean and variance of a binomial ran- dom variable are E(X) = n and var(X) = n (1 ), respectively. The Binomial distribution arises as the distribution of a sum of n independent Bernoulli trials. The binomial is closely related to a number of other distributions. If W1; :::; Wn are i.i.d. n er (p), then Wi in (n; p). As n increases and p 0 and np = , B i=1 B ! 1 ! X in (n; p) converges in distribution to a Poisson distribution with parameter . B P To simulate X in (n; ), B 1. Draw W1; :::; Wn independently Wi er () B 2. Set X = # (Xi = 1) . Poisson A random variable X N+ (the non-negative integers) has a Poisson distribution 2 with parameter , denoted X oi () ; if the probability mass function is P x Prob (X = x ) = e : (4) j x! The mean and variance of a Poisson random variable are E(X) = and var(X) = ; respectively. To simulate X oi (), P 1. Draw Zi i 1 independently, Zi exp (1) f g n 2. Set X = inf n 0 : Zi > . i=1 n X o Exponential 2 A random variable X R+ has an exponential distribution with parameter , de- 2 noted, X exp (), if the pdf is 1 x p (x ) = exp . (5) j The mean and variance of an exponential random variable are E (X) = and var (X) = 2, respectively. The inverse transform method is easiest way to simulate exponential random vari- x ables, since cumulative distribution function is F (x) = 1 e : To simulate X exp (), 1. Draw U [0; 1] U 2. Set X = ln (1 U) . Gamma A random variable X R+ has a gamma distribution with parameters and , 2 denoted X ( ; ), if the pdf is G 1 p (x ; ) = x exp ( x) , (6) j ( ) where the gamma function is de…ned in Appendix 4. The mean and variance of 1 2 a gamma random variable are E(X) = and var(X) = , respectively. It is important to note that there are di¤erently parameterizations of the gamma distribution. For example, some authors (and MATLAB) parameterize the gamma density as 1 1 p (x ; ) = x exp ( x= ) . j ( ) Notice that if Y ( ; 1) and X = Y= , then X ( ; ). To see this, note that G G the inverse transform is Y = X and dY=dX = , which implies that 1 1 1 p (x ; ) = (x ) exp ( x)( ) = x exp ( x) , j ( ) ( ) which is the density of ( ; ) random variable. The exponential distribution is a G 1 special case of the gamma distribution when = 1: X (1; ) implies that G X exp (). 3 Gamma random variable simulation is standard, with built-in generators in most software packages. These algorithms typically use accept/reject algorithms that are customized to the speci…c values of and . To simulate X ( ; ) when is G integer-valued, 1. Draw X1; :::; X independently Xi exp(1) 2. Set X = Xi. i=1 X For non-integer , accept-reject methods provide fast and accurate algorithms for gamma simulation. To avoid confusion over parameterizations, the transformation method can be used. To simulate X ( ; ) ; G 1. Draw Y ( ; 1) G Y 2. Set X = . Beta A random variable X [0; 1] has a beta distribution with parameters and , 2 denoted X ( ; ), if the pdf is B 1 1 x (1 x) p (x ; ) = , (7) j B ( ; ) where ( ) ( ) B ( ; ) = ( + ) is the beta function. Since p (x ; ) dx = 1, j R 1 1 1 B ( ; ) = x (1 x) dx: Z0 The mean and variance of a beta random variable are E (X) = and var (X) = , (8) + ( + )2 ( + + 1) respectively. If = = 1, then X (0; 1). U 4 If and are integers, to simulate X ( ; ), B 1. Draw X1 ( ; 1) and X2 ( ; 1) G G X 2. Set X = 1 : X1 + X2 For the general case, fast algorithms involving accept-reject, composition, and trans- formation methods are available in standard software packages. Chi-squared A random variable X R+ has a Chi-squared distribution with parameter , denoted 2 X 2 if the pdf is X 1 x 2 1 p (x ) = x exp . (9) j 2 2 2 2 The mean and variance of X are E (X) = and var (X) = 2, respectively. The 2-distribution is a special case of the gamma distribution: 2 = ; 1 . X X2 G 2 2 Simulating chi-squared random variables typically uses the transformation method. For integer values of . the following two-step procedure simulates a 2 random X variable: 1. Draw Z1; :::; Z independently Zi (0; 1) N 2 2.Set X = Zi . i=1 X When is large, simulating using normal random variables is computationally costly and alternative more computationally e¢ cient algorithms use gamma random variable generation. Inverse gamma A random variable X R+ has an inverse gamma distribution, denoted by X 2 ( ; ), if the pdf is IG exp p (x ; ) = x . (10) ( ) x +1 j 5 The mean and variance of the inverse gamma distribution are 2 E (X) = and var (X) = (11) 1 ( 1)2 ( 2) 1 for > 2. If Y ( ; ) ; then X = Y ( ; ). To see this, note that G IG 0 1 1 1 1 1 1 = y exp ( y) dy = exp 2 dx 0 ( ) ( ) x x x Z Z1 1 1 = exp dx. ( ) x +1 x Z0 The following two-steps simulate an ( ; ) IG 1: Draw Y ( ; 1) G 2: Set X = . Y Again, as in the case of the gamma distribution, some authors use a di¤erent para- meterization for this distribution as, so it is important to be careful to make sure you are drawing using the correct parameters. In the case of prior distributions over scale parameters, 2, it is additional complicated because some authors such as Zellner (1971) parameterize models in terms of instead of 2. Normal A random variable X R has a normal distribution with parameters and 2, 2 denoted X (; 2), if the pdf is N 2 2 1 (x ) p x ; = exp 2 : (12) j p22 2 ! The mean and variance are E (X) = and var (X) = 2. Given the importance of normal random variables, all software packages have func- tions to draw normal random variables. The algorithms typically use transformation methods drawing uniform and exponential random variables or look-up tables. Lognormal 6 2 A random variable X R+ has a lognormal distribution with parameters and , 2 denoted by X (; 2) if the pdf is LN 1 1 p x ; 2 = x ; 2 = exp (ln x )2 : (13) j j xp22 22 + 1 2 The mean and variance of the normal distribution are E (X) = e 2 and similarly var (X) = exp (2 + 2) (exp (2) 1). It is related to a normal distribution via the transformation X = e+Z . Although …nite moments of the lognormal exist, the distribution does not admit a moment-generating function. Simulating lognormal random variables via the transformation method is straightfor- ward since X = e+" where " (0; 1) is (; 2). N LN Truncated Normal A random variable X has a truncated normal distribution with parameters ; 2 and truncation region ( ; b) if the pdf is (x ; 2) p (x a < x < b) = j ; j (b ; 2) (a ; 2) j j where and it is clear that b (x ; 2) dx = (b ; 2). The mean of a truncated 1 j j normal distribution is R a b E(X a < X < b) = ; j b a where x is the standard normal density evaluated at (x ) = and x is the stan- dard normal CDF evaluated at (x ) =. The inversion method can be used to simulate a truncated normal random variable. A two-step algorithm provides a draw from a truncated standard normal, 1: U U [0; 1] 1 2: X = [ (a) + U ( (b) (a))] ; a 1=2 2 where (a) = (2) exp ( x =2) dx. For a general truncated normal, X 2 1 (; ) 1[a;b] TN R 1: Draw U U [0; 1] 1 a b a 2: X = + + U ; 1 where is the inverse of the error function.