MLEtools.nb 1
Reference: some basic distributions
An excellent reference for probability distributions is: Evans, Merran, Nicholas Hastings, and Brian Peacock. 2000. Statistical Distributions, 3rd Edition. John Wiley & Sons. New York.
Continuous
Expected, or mean E Z = zf z „ z Variance VAR Z = E Z - E Z 2 = z - E z 2 f z „ z @ D Ÿ H L For continuous variables, the probability density function is the probability of the value z given the parameters H L @H @ DL D Ÿ H @ DL H L
ü Uniform
Uniform Distribution: a uniform distribution on the interval [a,b] where a < b probability density function: 1 f z = ÅÅÅÅÅÅÅÅÅÅb-a , where a § z § b b+a mean: ÅÅÅÅÅÅÅÅÅÅ2 b-a 2 variance: ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ12 H L H L ü r
dunif z, min = a, max = b
# generate 1 uniform random number over the range 1.4 to 2.4 runif H1, 1.4, 2.4 L
ü mathematicaH L
1 f z_, a_, b_ := b − a
∗ random number over range 0,1 ∗ @ D Random
H0.0407265 L @D ∗ random number of specified range ∗
H L MLEtools.nb 2
Random Real, 3, 6
4.87348 @ 8 ü Normal Normal Distribution: has two parameters: m is the location parameter, in this case the mean; s is the scale parameter, in this case the standard deviation probability density function: 2 f z = ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ1 exp - ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅz-m 2 ps2 2 s2 mean: m H L variance: s2 è!!!!!!!!!!! !!! H L I M note: plotting f z will give you the familiar bell curve ü r H L probability density function dnorm z, mean, sd # plot the pdf curve dnorm x, 4, 1 ,0,7 H L random number generation, where n is the number random numbers to generation. In other words, n = 1 will create 1 random number. H H L L rnorm n, mean, sd ü mathematicaH L 1 z − m 2 f z_, m_, σ_ := Exp − 2 2 π σ2 2 σ Plot f z, 4, 1 , z, 0, 7 ; H L @ D è!!!!!!!!!!!!! A E 0.4 @ @ D 8 0.2 0.1 1 2 3 4 5 6 7 MLEtools.nb 3 Show that pdf integrates to 1 Integrate f z, 1, 1 , z, −∞, ∞ 1 @ @ D 8 H1.87451 L @ @ DD ü Lognormal Lognormal Distribution: has two parameters: m is the location parameter; s is the shape parameter probability density function: 2 f z = ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ1 ÅÅÅÅÅÅÅÅ exp - ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅln z -logÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅm , where z > 0, m >0, and s >0 z 2 ps2 2 s2 2 mean: m exp ÅÅÅÅÅÅÅs 2 H H L H LL median: m è!!!!!!!!!!! !!! H L I M s2 is the variance of the ln(z) variance: m2I expMs2 exp s2 - 1 note: This describes a variable in which the logarithm of the variable is a normal distribution. Whereas the normal distribu- tion is additive, the lognormal distribution is multiplicative. H L H H L L ü r dlnorm z, meanlog = m, sdlog =σ R uses a slightly different form of the lognormal model. Use the following form so that m is defined in the same way as described in the f(zH) equation above. L dlnorm z, log m , σ # generate random number rlnorm H1, meanlogH L = 1,L sdlog = 1.2 ü mathematica H L 1 Log z − Log m 2 f z_, m_, σ_ := Exp − 2 z 2 π σ2 2 σ H @ D @ DL @ D è!!!!!!!!!!!!! A E MLEtools.nb 4 Plot f z, 1.7, 1 , z, 0, 20 ; @ @ D 8 0.2 0.1 5 10 15 20 Show that pdf integrates to 1 Integrate f z, 1, 1 , z, 0, ∞ 1 @ @ D 8 Integrate zf z, m, σ , z, 0, ∞ , Assumptions → σ>0, m > 0, σ∈Reals, m ∈ Reals 2 σ 2 m @ @ D 8 < 8 Integrate z − Integrate zf z, m, σ , z, 0, ∞ , Assumptions → σ>0, m > 0, σ∈Reals, m ∈ Reals 2 f z, m, σ , z, 0, ∞ , Assumptions → σ>0, m > 0, σ∈Reals, m ∈ Reals @ 2 2 Hσ −1 + σ m2 @ @ D 8 < 8 ü Exponential Exponential Distribution: has three parameters: m is the location parameter; s is the shape parameter probability density function: f z = a Exp -az, where z > 0 1 mean: ÅÅÅÅa 1 variance: ÅÅÅÅÅÅÅa2 note: ContinuousH L distribution@ D analog of Poisson. Can be used for dispersal distance. Widely used in Baysian Analysis ü r dexp z, rate = a # generate random number rexp H1, rate = aL ü mathematicaH L f z_, a_ := a Exp −az Plot f z, 0.5 , z, 0, 15 , PlotRange → All ; @ D @ D 0.5 @ @ D 8 < D 0.4 0.3 0.2 0.1 2 4 6 8 10 12 14 << Statistics`ContinuousDistributions` ∗ exponential random number with a = 0.5 ∗ Random ExponentialDistribution 0.5 H2.1514 L @ @ DD MLEtools.nb 6 ü Gamma Gamma Distribution: has two parameters: a is the rate parameter; n is the shape parameter probability density function: an n-1 f z = ÅÅÅÅÅÅÅÅÅÅÅG n Exp -az z , where z > 0 n mean: ÅÅÅÅa n variance: ÅÅÅÅÅÅÅ2 H L H aL @ D note: can have a long tail. When n = 1, the gamma distribution becomes the exponential distribution. When n = degrees of freedom / 2 and a = 2, the gamma distribution becomes the chi-square distribution. ü r dgamma z, shape = n , rate = a # generate random number rgamma H1, shape = n, rate = a L ü mathematica H L an f z_, a_, n_ := Exp −az zn−1 Gamma n Plot f z, 1, 1 , z, 0, 12 , PlotRange → All ; @ D @ D 1 @ D @ @ D 8 < D 0.8 0.6 0.4 0.2 2 4 6 8 10 12 MLEtools.nb 7 Plot f z, 1, 2 , z, 0, 12 , PlotRange → All ; 0.35 @ @ D 8 < D 0.3 0.25 0.2 0.15 0.1 0.05 2 4 6 8 10 12 Plot f z, 0.5, 2 , z, 0, 12 , PlotRange → All ; 0.175 @ @ D 8 < D 0.15 0.125 0.1 0.075 0.05 0.025 2 4 6 8 10 12 Derive expected value Integrate zf z, a, n , z, 0, ∞ , Assumptions → a > 0, n > 0, a ∈ Reals, n ∈ Reals n a @ @ D 8 < 8 Integrate z − Integrate zf z, a, n , z, 0, ∞ , Assumptions → a > 0, n > 0, a ∈ Reals, n ∈ Reals 2 f z, a, n , z, 0, ∞ , Assumptions → a > 0, n > 0, a ∈ Reals, n ∈ Reals @ n H @ @ D 8 < 8 2.82494H L @ @ ê DD MLEtools.nb 8 ü Chi-Square Note: this section is a bit incomplete Chi-Square Distribution: A squared normal distribution is a chi-square distribution. probability density function for 1 degree of freedom: f z = ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ1 ÅÅÅÅÅÅÅÅ exp - ÅÅÅÅÅÅÅÅÅÅÅz 2 ps2 z 2 s2 probability densityè!!!!!!! function!!!! !!!!!! for n degrees of freedom: H L 1 Hn 2 -1 L z f z = ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅn n 2ÅÅÅÅÅÅÅÅÅÅÅÅÅÅn z exp - ÅÅÅÅÅÅÅÅÅÅÅ2 , for z ¥ 0 s 2 G ÅÅÅÅ2 2 s mean: n s2 H ê L 4 H ê L variance: 2 n s H L H L H L notes: - n can also be thought of as a shape parameter in the context of a general distribution - often assume unit variance, s = 1 1 - G ÅÅÅÅ2 = p è!!!! ü rH L dchisq ü mathematica HL 1 z f z_, σ_ := Exp − 2 2 π σ2 z 2 σ Plot f z, 2.1 , z, 0, 7 ; @ D è!!!!!!!!!!!!!!!!! A E 1.75 1.5 @ @ D 8 1.25 1 0.75 0.5 0.25 1 2 3 4 5 6 7 Show that pdf integrates to 1 MLEtools.nb 9 Integrate f z, 2.1 , z, 0, ∞ 1. @ @ D 8 0.111756H L @ @ DD Add non-centrality, or location, parameter with a single degree of freedom, although no scale parameter mean: 1 + m Two forms are given below −1 z +μ 1 z 4 f z_, μ_ := Exp − BesselI −1 2, μ z 2 2 μ 1 1 1 è!!!!!!! z +μ f@z_, μ_D := Hypergeometric0F1A E J N , z μ A ê ExpE− 2 4 2 π z 2 NIntegrate f z, 2.0 , z, 0, 100000 @ D A E è!!!!!!!!!!!! A E 1. @ @ D 8 3. @ @ D 8 1 +μ @ @ D 8 < 8 Cosh z μ A E 1 1 z +μ è!!!!!!! μ + − μ − ExpA zE Exp z Exp 2 2 π z 2 è!!!!!!! è!!!!!!! 1 z +μ ExpI Az μ +EExp − A z μ EM è!!!!!!!! !!!! Exp −A E 2 2 π z 2 1è!!!!!!! è!!!!!!!z +μ 1 z +μ I A ExpE zAμ− EM +è!!!!!!!! !!!! AExp − zEμ− 2 2 π z 2 2 2 π z 2 è!!!!!!! è!!!!!!! è!!!!!!!!!!!! A E è!!!!!!!!!!!! A E MLEtools.nb 10 ü Bimodal Normal Normal Distribution: has two parameters: m is the location parameter, in this case the mean; s is the scale parameter, in this case the standard deviation probability density function: 2 f z = ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ1 exp - ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅz-m 2 ps2 2 s2 mean: m H L variance: s2 è!!!!!!!!!!! !!! H L I M note: plotting f z will give you the familiar bell curve Here I extend that to a bimodal normal distribution. I am assuming that I can just divide each unimodal pdf by 2, but it would be nice toH confirmL this ü r # assuming equal weighting to each mode dnorm z, m1, s1 + dnorm z, m2, s2 2 # weighting c for modes H1 1 +Hexp −c L ∗ dnorm Hz, m1, s1LL+ ê 1 1 + exp c dnorm z, m2, s2 ü mathematicaH êH H LLL H L H êH H LLL H L 1 z − m1 2 1 z − m2 2 f z_, m1_, σ1_, m2_, σ2_ := Exp − + Exp − 2 2 2 2 π σ12 2 σ1 2 2 π σ22 2 σ2 Plot f z, 5, 1, 2, 0.7 , z, 0, 8 ; H L H L @ D è!!!!!!!!!!!!!!!! A E è!!!!!!!!!!!!!!!! A E 0.25 @ @ D 8 0.2 0.15 0.1 0.05 2 4 6 8 Show that pdf integrates to 1 Integrate f z, 5, 1, 2, 0.7 , z, −∞, ∞ 1. @ @ D 8 Therefore the support function is ∗ bimodal normal distribution ∗ sf z_, m1_, σ1_, m2_, σ2_ := 1 z − m1 2 1 z − m2 2 H Log 2 2 π − Log Exp − L + Exp − ; 2 σ 2 2 σ 2 @ σ1D 2 1 σ2 2 2 bimodalRandomè!!!!!!!!! m1_, σ1_, m2_, σ2_ H:= BlockL indef, x , H L A E A è!!!!!!!!!! A E è!!!!!!!! A EE indef x_Real := NIntegrate bimodalPDF z, m1, σ1, m2, σ2 , z, −∞,x ; m1 + m2 m1 + m2 x . FindRoot@ Random indef Dx , x, A8 ∗ 0.5,< ∗ 1.0 2 2 @ D @ @ D 8 c z − m1 2 1 − c z − m2 2 f z_, m1_, σ1_, m2_, σ2_, c_ := Exp − + Exp − 2 2 2 π σ12 2 σ1 2 π σ22 2 σ2 Plot f z, 5, 1, 2, 0.7, 0.3 , z, 0, 8 ; H L H L @ D è!!!!!!!!!!!!!!!! A E è!!!!!!!!!!!!!!!! A E 0.4 @ @ D 8 0.2 0.1 2 4 6 8 Show that pdf integrates to 1 Integrate f z, 5, 1, 2, 0.7, 0.3 , z, −∞, ∞ 1. @ @ D 8 ü Beta Beta Distribution: This distribution is good for modeling fractions because it is constrained to the domain between 0 and 1. It has two parame- ters: both p and q are shape parameters with p>0 and q>0, but z is restrict to the range (0,1) probability density function: p-1 q-1 1 q-1 f z = ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅz 1-ÅÅÅÅÅÅÅÅÅÅÅÅz , B is the beta function, B(p,q) = up-1 1 - u „u) B p,q 0 p-1 q-1 f z = ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅz ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ1-z ÅÅÅÅÅÅÅÅÅ 1up-1 1-u q-1 „u 0 H L mean: ÅÅÅÅÅÅÅÅÅÅÅp H L p+Hq L H L Ÿ H L pq variance: ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅŸÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅH L H Lp+q 2 p+q+1 ü r H L H L dbeta z, shape1 = p, shape2 = q # random number generation rbeta H1, shape1 = p, shape2 = qL ü mathematicaH L zp−1 1 − z q−1 f z_, p_, q_ := Beta p, q Plot f z, 2.2, 3.6 , Hz, 0,L 1 ; @ D 2 @ D @ @ D 8 1 0.5 0.2 0.4 0.6 0.8 1 Therefore the support function is ∗ beta distribution ∗ sf z_, p_, q_ := Log Beta p, q − p − 1 Log z − q − 1 Log 1 − z The Beta Hdistribution can be transformedL to be in terms of a mean parameter and variance parameter instead of p and q. This can be more@ useful becauseD one@ is often@ interestedDD H in modelingL @ theD mean.H L @H LD MLEtools.nb 13 μ 1 −μ μ substiution = q → μ 1 −μ −σ2 ,p−> μ 1 −μ −σ2 ; σ2 μ σ2 1−μ 1−μ μ−σ2 μ 1−μ μ−σ2 −1+ −1+ 9 1 − zH H Lσ2 L z σ2 H H L L= f z_, μ_, σ_ := ; μ 1H−μ LIHμ−σ2 L M1−μ 1−μIH μ−σL 2 M Beta σ2 , σ 2 2 2 H L−HH1 +μL −μL + μH +σLHH L L sf@ z_, μ_, σ_D := 1 − Log 1 − z + A σ2 E −μ2 +μ3 +σ2 +μσ2 μ −μ + μ2 +σ2 −1 +μ −μ+μ2 +σ2 ji H Log zLH+ Log Beta L−zy , ; @ σ2D j z @Hσ2 LD σ2 k { where m œ [0,1]ji zy H L H LH L j z @ D A A EE and s2 < m 1 -k m { mean: m variance: sH2 L The Beta distribution can be transformed to be in terms of a mean parameter and dispersion parameter, p = s(1-m) and q =s m. This just scales p and q without the specific constraints on the variance version. Here m œ (0,1), s œ (0,¶). ü Bimodal Beta extend the Beta Distribution by adding 2 pdf's together and dividing by 2 probability density function: p-1 q-1 r-1 s-1 1 q-1 f z z 1-z z 1-z , B is the beta function, B(p,q) = up-1 1 u u) = ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ2 B pÅÅÅÅÅÅÅÅÅÅÅÅ,q + ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ2 B rÅÅÅÅÅÅÅÅÅÅÅ,s 0 - „ mean: H L H L variance: H L H L H L Ÿ H L ü mathematica zv−1 1 − z w−1 zx−1 1 − z y−1 f z_, v_, w_, x_, y_ := + 2 Beta v, w 2 Beta x, y f 0.4, 2.2, 3.6, 2.2, 3.6 H L H L @ D @ D @ D 1.8448 @ D MLEtools.nb 14 Plot f z, 1.3, 3.6, 4.1, 1.6 , z, 0, 1 ; 1.2 @ @ D 8 0.2 0.4 0.6 0.8 1 0.8 0.6 0.4 Integrate f z, 1.3, 3.6, 4.1, 1.6 , z, 0, 1 1. @ @ D 8 ∗ bimodal beta distribution ∗ zv−1 1 − z w−1 zx−1 1 − z y−1 sf z_, v_, w_, x_, y_ :=−Log + 2 Beta v, w 2 Beta x, y H L H L H L @ D A E @ D @ D Discrete Expected, or mean E Z = z zfz Variance 2 2 VAR Z = E Z - E Z = z z - E z fz @ D ⁄ where Z is a random variable, z is a particular instance of that variable, fz is the frequency of occurance of z. For discrete variables,H L @H the probability@ DL D ⁄ distributionH @ DL function is the probability of the value z given the parameters MLEtools.nb 15 ü Binomial Binomial Distribution: has two parameters: N is the number of times the trial was conducted in which one of two outcomes can occur. p is the probability of one outcome, and z is the total number of times that outcome occurs. For example, flip a coin 10 times and count the number of heads. The result would be a binomial distribution, and if the coin is fair, p = 0. 5. Whereas the other discrete distributions are used to model discrete observations of continuous processes, the binomial models discrete pro- cesses. probability distribution function: N! z N-z Pr Z = z = ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅN-z ! z! p 1 - p mean: N p variance: Np1 - p H L 8 < I M H L ü r H L dbinom z, n, p plot dbinom seq 1:20 , 20, 0.3 # randomH numberL generation rbinomH 1, n,H p H L LL # see function description below curve 1H 1 + expL −x , −10, 10 ü mathematicaH êH H LL L n! f z_, n_, p_ := pz 1 − p n−z n − z ! z! ListPlot Table z, f z, 20, 0.25 , z, 0, 20 , PlotStyle → PointSize 0.03 ; @ D H L H L 0.2 @ @8 @ D< 8 0.15 0.1 0.05 5 10 15 20 Sums to 1 Plus @@ Table f z, 20, 0.3 , z, 0, 20 1. @ @ D 8 << Statistics`DiscreteDistributions` ∗ binomial random number with n = 3 and p = 0.5 ∗ Random BinomialDistribution 3 , 0.5 1H L @ @ DD Here is a useful function for modeling a probability. The funciton is constrained to a range between 0 and 1 for all real values of f x , both positive and negative. This function is derived by the following logic: 1) Probabilities œ [0,1]. An odds ratio is a ratio of the probability of sucess over the probability of failure, and odds ratiosH L œ [0,¶). Therefore ln(odds ratios) œ (-¶,¶). 2) For a binary model, only two outcomes are examined, therefore the Pr{failure} = 1 - Pr{success}. Pr success ln ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅPr failureÅÅÅÅÅÅÅÅÅÅ = f x ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅPr successÅÅÅÅÅÅÅÅÅÅ = exp f x Pr failure8 < ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅPr success8ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ< = exp f x 1-I8Pr success< M H L Pr 8success< = ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅexp f ÅÅÅÅÅÅÅÅÅÅÅÅx 8 < expH Hf xLL+1 Pr success8 < = ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ1ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ 1+exHHp -H fLLLLx H H LL 8 < parameters = a →−4, b → 0.7 ; H H LL 8 f< x_ := a + bx . parameters; 1 Plot 8 . parameters,< x, 0, 10 , PlotRange → 0, 1 ; 1 + Exp −f x @ D ê 1 A ê 8 < 8 0.6 0.4 0.2 2 4 6 8 10 The Hill function is also useful for modeling probabilities as it contains interpretable parameters: a is the point at which the probability equals 0.50 and b is the "slope" at the 0.50 point. MLEtools.nb 17 parameters = a → 5, b → 4 ; 1 Plot . parameters, x, 0, 10 , PlotRange → 0, 1 ; 1 + a b x 8 < 1 A ê 8 < 8 0.6 0.4 0.2 2 4 6 8 10 ü Multinomial Multinomial Distribution: This is an extension of the binomial to m possible outcomes each with a probability pi. probability distribution function: N! z z z Pr Zm = zm = ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ p1 1 p2 2 ...pm m z1! z2! ...zm! mean: Npm variance: Npm 1 - pm 8 < I M ü mathematicaH L n! f z1_, z2_, n_, p1_, p2_ := p1z1 p2z2 1 − p1 − p2 n−z1−z2 z1! z2! n − z1 − z2 ! ListPlot Table z, f z, 3, 20, 0.25, 0.25 , z, 0, 20 , PlotStyle → PointSize 0.03 ; @ D H L H L 0.025 @ @8 @ D< 8 0.015 0.01 0.005 5 10 15 20 Sums to 1 MLEtools.nb 18 Plus @@ Flatten Table f z, i, 5, 0.25, 0.25 , i, 0, 5 , z, 0, 5 1. @ @ @ D 8 < 8 ∗ multinomial random vector for an event that occurs 10 times; the probability vector follows ∗ Random MultinomialDistribution 10, 0.2, 0.2, 0.3, 0.3 H 1, 3, 2, 4 L @ @ 8 ü r 8 < dmultinom # random number generation HL ü Poisson Poisson Distribution: has one parameter that can really be broken into two parameters: r is the rate constant and t is the time. One description of a poisson distribution: Imagine a process with a constant probability of occurance r. The event can occur at any time, and repeatedly. However, if we measure at discrete time intervals, and count the number of times the event has occurred for discrete entities, the result will be a Poisson Distribution. Imagine rain falling onto floor tiles. If you count the number of drops that hit each tile in one minute, the result would be distribute as a Poisson distribution. probability distribution function: e-rt rt z Pr Z = z = ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅz! mean: r t variance: r t H L note: if evenly8 spaced< time intervals, set t equal to 1 ü r dpois z, lambda # random number generation rpois H1, lambdaL ü mathematicaH L Exp −λ λz f z_, λ_ := z! @ D @ D MLEtools.nb 19 ListPlot Table z, f z, 3 , z, 0, 15 , PlotStyle → PointSize 0.03 ; 0.2 @ @8 @ D< 8 0.15 0.1 0.05 2 4 6 8 10 12 14 Sums to 1 Plus @@ Table f z, 20, 0.3 , z, 0, 100 1. @ @ D 8 2H L @ @ DD ü Negative binomial Negative Binomial Distribution: The negative binomial is similar to the poisson except that the data are overdispersed. As a consequence, the variance will always be larger than the mean. The negative binomial can be derived assuming the rate constant in the poisson is no longer constant but is gamma distributed. As k ض, the distribution approaches a Poisson. k is the shape parameter. probability distribution function: G k+z k k m z Pr Z = z = ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅG k z! ÅÅÅÅÅÅÅÅÅÅÅk+m ÅÅÅÅÅÅÅÅÅÅÅk+m ¶ where G is the gamma function, G n = Exp -t tn-1 „t H L 0 mean: m H L 8 m2< H L H L variance: m + ÅÅÅÅÅÅÅÅk H L Ÿ H L Alternative form: used to describe the number the number of failures that occur before n sucesses have occured, where the probability of success is p. ü r # note, "mu =" is needed to specify this particular parameterization dnbinom z, size = k, mu = m rnbinom 1, size = k, mu = m H L H L MLEtools.nb 20 ü mathematica Gamma k + z k k m z f z_, k_, m_ := Gamma k z! k + m k + m ListPlot Table z, f z,@ 10,D 3.2i , yz, 0, 15 , PlotStyle → PointSize 0.03 ; @ D j z J N @ D k { 0.2 @ @8 @ D< 8 0.1 0.05 2 4 6 8 10 12 14 ListPlot Table z, f z, 0.26, 10 , z, 0, 100 , PlotStyle → PointSize 0.03 ; 0.02 @ @8 @ D< 8 0.015 0.01 0.005 20 40 60 80 100 Sums to 1 Plus @@ Table f z, 10, 3 , z, 0, 100 N 1. @ @ D 8 Sum zf z, k, m , z, 0, ∞ m @ @ D 8 Sum z − Sum zf z, k, m , z, 0, ∞ 2 f z, k, m , z, 0, ∞ Simplify m k + m k @H @ @ D 8 H L MLEtools.nb 21 Mathematica does not contain a built-in random number generator for this form of the negative binomial. While not very elegant, the function below can be used to randomly generate numbers from a negative binomial distribution. NegBiRandom m_, k_ := Gamma k + x k k m x funtable = Table , x, 0, 250 ; Gamma k x! k + m k + m @ D 9 CDF = Table Sum funtable i , i, endsum , endsum, 1, Length funtable ; @ D i y x = Random ; A j z J N 8 NegBiRandomD 10, 3 =@@ DD 5 @ D ∗ negative binomial distribution ∗ Clear sf ; sf z_, m_ ?NumberQ, k_ ?NumberQ := If m > 0&&k> 0, −Log Gamma k + z + Log z! + H Log Gamma k − k Log k + k Log k +L m − z Log m + z Log k + m , 10000.0 ; SetAttributes@ D sf, Listable ; @ D @ @ @ DD @ D @ @ DD @ D @ D @ D @ D D @ D ü Generalized Poisson The generalized Poisson distribution (Consul and Jain. 1973. Technometrics. 15: 791-799) can be used to represent a Poisson process in which the data are over-dispersed. It is also known as Lagrangian Poisson Distribution (Johnson, Kotz, and Kemp. 1992. Univariate Discrete Distributions). Despite the claims of the original authors, this distribution cannot be used for under-dispersed data because the sum of the probabilities do not add to 1. This distribution explicity models an increase in the rate of moving to subsequent categories and can represent data with very long right-tails as compared to the negative binomial. This distribution worked very well for the distribution of pollen deposited on flowers (Castellanos et al. 2003. Evolution 57:2742-2752.) When l2 = 0 this distribution becomes the Poisson distribution. - l +z l z-1 e 1 2 l1 l1+z l2 Pr Z = z = ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅz! ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ , where 0 Consul and Jain (1973) list the followingH methodL for estimating@ H l andLD l from the mean and variance, but I don't trust these @ D 1 2 as the the best estimates, just a close estimate. mean l2 = 1 - ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅvariance "############### MLEtools.nb 22 l1 = mean 1 -l2 H L ü Beta-Binomial Beta Binomial Distribution: This distribution assumes that p in the binomial distribution has a beta distribution with parameters v and w (see beta distribution above). probability distribution function: N! B v+z,N+w-z Pr Z = z = ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅN-z ! z! ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅB vÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ,w , where B(v,w) is the beta function mean: H L variance: H L H L 8 < I M The Beta distribution can be transformed to be in terms of a mean parameter and dispersion parameter, p = s(1-m) and q =s m. This just scales p and q without the specific constraints on the variance version. Here m œ (0,1), s œ (0,¶). The m, s version is the one used for sfBetaBinomial in Mathematica. This form has been used to describe correlated instances of the Binomial Distribution, although more mechanistic distributions for these cases exist. ü r library rmutil # this is based on a parameterization so that m is the mean and s is # an overdispersion parameters. As s approaches infintiy, # the BetaH − BinomialL collapses to a Biomial Distribution dbetabinom z, N, m, s ü mathematica H L n! Beta s m + z, n + s 1 − m − z f z_, n_, m_, s_ := n − z ! z! Beta s m, s 1 − m n! Beta@v + z, n + w − zH L D f@z_, n_, v_, w_D := Hn − zL! z! Beta@v, w H LD ListPlot Table z, f z, 20, 2.0, 3.2 @ , z, 0, 20 D, PlotStyle → PointSize 0.03 ; @ D H L @ D 0.08 @ @8 @ D< 8 0.06 0.04 0.02 5 10 15 20 MLEtools.nb 23 Sums to 1 Plus @@ Table f z, 20, 2.0, 3.2 , z, 0, 20 1. @ @ D 8 ∗ pdf for beta distribution ∗ zp−1 1 − z q−1 f z_, p_, q_ := Beta p, q H L ∗ pdf for binomialH distributionL ∗ @ D n!@ D f z_, n_, p_ := pz 1 − p n−z n − z ! z! H L n! pv−1 1 − p w−1 Integrate@ D pz 1 − p Hn−z L , n − z H! z! L Beta v, w p, 0, 1 , Assumptions → v > 0, w > 0, z ≥ 0, n ≥ 0 H L A H L 1 H L @ D Beta8 v, w< n − z ! z! 8 @ D @ D ü BimodalH Beta-BinomialL @ D Using the same assumptions from the derivation of the unimodal beta binomial distribution, multiply the bimodal beta distribition by the binomial distribution and integrate p from 0 to 1 ∗ pdf for binomial distribution ∗ n! f z_, n_, p_ := pz 1 − p n−z n − z ! z! H L ∗ pdf for bimodal beta ∗ @ D H L H L zv−1 1 − z w−1 zx−1 1 − z y−1 f z_, v_, w_, x_, y_ := + 2 Beta v, w 2 Beta x, y H L H L H L @ D @ D @ D MLEtools.nb 24 ∗ multiply both distributions by each other ∗ n! pv−1 1 − p w−1 px−1 1 − p y−1 pz 1 − p n−z + n − z ! z! 2 Beta v, w 2 Beta x, y H L n! pv−1 1 − p w−1 n! px−1 1 − p y−1 pz 1 − p n−z ji H L + H L pz zy1 − p n−z n − z ! z! H L j2 Beta v, w n − z ! z! z 2 Beta x, y H L k @ D @ D { n! pz+v−1 1 − p w+n−z−1 n! pz+x−1 1 − p y+n−z−1 H + L H L n − z ! z! H2 BetaL v, w n − z ! z! 2 Beta Hx, y L H L @ D H L @ D n! pz+v−1 1 − p w+n−z−1 pz+x−1 1 − p y+n−z−1 H L + H L n − z ! z! 2 Beta v, w 2 Beta x, y H L @ D H L @ D 1 ! z+v−1 − w+n−z−1 z+x−1 − y+n−z−1 n i p H 1 L p p H 1 L p y j + z p n − z ! zj! 2 Beta v, w 2 Beta x, y z H0 L k @ D @ D { ! 1 z+v−1 − w+n−z−1 ! 1 z+x−1 − y+n−z−1 n ip 1H p L Hn L p y 1 p j p + z p ‡ n − z ! z! 0 j 2 Beta v, w n − z ! z! 0 z2 Beta x, y H L k @ D @ D { n! Beta z + v, w + n − z n! Beta z + x, y + n − z H L + H L n − z ! z! ‡ 2 Beta v, w n − z ! z! ‡2 Beta x, y H L @ D H L @ D n! Beta z + v, w + n − z Beta z + x, y + n − z @ D + @ D n − z ! z! 2 Beta v, w 2 Beta x, y H L @ D H L @ D ∗ pdf for bimodali @ beta−binomalD distribution@ ∗ D y j z H L k @ D n! Beta@ z + v,D w +{n − z Beta z + x, y + n − z f z_, n_, v_, w_, x_, y_ := + n − z ! z! 2 Beta v, w 2 Beta x, y H L Bimodal Beta-Binomial Distribution: i @ D @ D y @ D j z has 5 parameters: shape parameters for one nodeH areL v and wk; shape parameters@ Dfor the other node are@ x andD y; n is{ the total number of individuals in the set and z is the number of success in that set. Note: this distribution and the unimodal beta-binomial assume a different draw from beta for the p of each set, but it assumes the same p for all the individuals within a set. This statement is based on Skellam. 1948. Journal of the Royal Statistical Society. Series B, 10: 257-261. probability density function: p-1 q-1 1 q-1 f z = ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅz 1-ÅÅÅÅÅÅÅÅÅÅÅÅz , B is the beta function, B(p,q) = up-1 1 - u „u) B p,q 0 p-1 q-1 f z = ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅz ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ1-z ÅÅÅÅÅÅÅÅÅ 1up-1 1-u q-1 „u 0 H L mean: H L H L H L Ÿ H L variance: H L H L Ÿ MLEtools.nb 25 ListPlot Table f z, 30, 1.2, 3.6, 3.6, 1.2 , z, 0, 30 , PlotStyle → PointSize 0.02 , PlotRange → 0, 0.04 ; 0.04 @ @ @ D 8 0.02 0.01 5 10 15 20 25 30 zv−1 1 − z w−1 zx−1 1 − z y−1 Plot + . v −> 1.2, w −> 3.6, x −> 3.6, y −> 1.2 , z, 0, 1 ; 2 Beta v, w 2 Beta x, y H L H L 1.2 A ê 8 < 8 0.8 0.6 0.4 0.2 0.2 0.4 0.6 0.8 1 Plus @@ Table f z, 30, 1.2, 3.6, 3.6, 1.2 , z, 0, 30 1. @ @ D 8 ∗ bimodal beta−binomial distribution ∗ sf z_, n_, v_, w_, x_, y_ := Beta z + v, w + n − z Beta z + x, y + n − z HLog n − z ! + Log z! − Log n! − Log L + 2 Beta v, w 2 Beta x, y @ D @ D @ D @H L D @ D @ D A E ü Bimodal Binomial @ D @ D n! n! f z_, n_, p_, q_ := pz 1 − p n−z + qz 1 − q n−z 2 n − z ! z! 2 n − z ! z! @ D H L H L H L H L MLEtools.nb 26 ListPlot Table z, f z, 20, 0.15, 0.45 , z, 0, 20 , PlotStyle → PointSize 0.03 ; 0.12 @ @8 @ D< 8 0.08 0.06 0.04 0.02 5 10 15 20 Sums to 1 Plus @@ Table f z, 20, 0.3 , z, 0, 20 1. @ @ D 8 H L @ D @ H L D @ D @ D @ H L H L D