DOCSLIB.ORG

Exponential Families I

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Exponential Families I Exponential Families Exponential Families MIT 18.655 Dr. Kempthorne Spring 2016 1 MIT 18.655 Exponential Families One Parameter Exponential Family Exponential Families Multiparameter Exponential Family Building Exponential Families Outline 1 Exponential Families One Parameter Exponential Family Multiparameter Exponential Family Building Exponential Families 2 MIT 18.655 Exponential Families One Parameter Exponential Family Exponential Families Multiparameter Exponential Family Building Exponential Families Definition q Let X be a random variable/vector with sample space X ⊂ R and probability model Pθ: The class of probability models P = fPθ; θ 2 Θg is a one-parameter exponential family if the density/pmf function p(x j θ) can be written: p(x j θ) = h(x)expfη(θ)T (x) − B(θ)g where h : X! R η :Θ ! R B :Θ ! R: Note: By the Factorization Theorem, T (X ) is sufficient for θ p(x j θ) = h(x)g(T (x); θ) Set g(T (x); θ) = expfη(θ)T (x) − B(θ)g T (X ) is the Natural Sufficient Statistic: 3 MIT 18.655 Exponential Families One Parameter Exponential Family Exponential Families Multiparameter Exponential Family Building Exponential Families Examples Poisson Distribution (1.6.1) : X ∼ Poisson(θ); where E[X ] = θ: θx −θ p(x j θ) = x! e ; x = 0; 1;::: 1 = x! expf(log(θ)x − θg = h(x)expfη(θ)T (x) − B(θ) where: 1 h(x) = x! η(θ) = log(θ) T (x) = x 4 MIT 18.655 Exponential Families One Parameter Exponential Family Exponential Families Multiparameter Exponential Family Building Exponential Families Examples Binomial Distribution (1.6.2) : X ∼ Binomial(θ; n) n r p(x j θ) = θ1(1 − θ)n−x x = 0; 1;:::; n x r n θ = expflog()x + nlog(1 − θ)g x 1−θ = h(x)expfη(θ)T (x) − B(θ) where: r n h(x) = x θ η(θ) = log( 1−θ ) T (x) = x B(θ) = −nlog(1 − θ) 5 MIT 18.655 Exponential Families One Parameter Exponential Family Exponential Families Multiparameter Exponential Family Building Exponential Families Examples 2 Normal Distribution : X ∼ N(µ, σ0): (Known variance) 1 2 − (x−µ) 1 2σ2 p 0 p(x j θ) = 2 e 2πσ0 1 x2 µ µ2 = [p exp{− g] × expf x − g 2 2σ2 σ2 2σ2 2πσ0 0 0 0 = h(x)expfη(θ)T (x) − B(θ) where: 1 x2 h(x) = [−p exp{− g] 2 2σ2 2πσ0 0 µ η(θ) = 2 σ0 T (x) = x µ2 B(θ) = 2 2σ0 6 MIT 18.655 Exponential Families One Parameter Exponential Family Exponential Families Multiparameter Exponential Family Building Exponential Families Examples 2 Normal Distribution : X ∼ N(µ0; σ ): (Known mean) 1 2 1 − (x−µ0) p(x j θ) = p e 2σ2 2πσ2 1 1 2 1 2 = [p ] × exp{− 2 (x − µ0) ) − log(σ )g 2πσ2 2σ 2 = h(x)expfη(θ)T (x) − B(θ) where: h(x) = [p1 ] 2π 1 η(θ) = − 2σ2 2 T (x) = (x − µ0) 1 2 B(θ) = 2 log(σ ) 7 MIT 18.655 Exponential Families One Parameter Exponential Family Exponential Families Multiparameter Exponential Family Building Exponential Families Samples from One-Parameter Exponential Family Distribution Consider a sample: X1;:::; Xn; where Xi are iid P where P 2 P= fPθ; θ 2 Θg is a one-parameter exponential family distribution with density function p(x j θ) = h(x)expfη(θ)T (x) − B(θ)g The sample X = (X1;:::; Xn) is a random vector with density/pmf: Qn p(x j θ) = i=1 (h(xi )exp[η(θ)T (xi ) − B(θ)]) Qn Pn = [ i=1 h(xi )] × exp[η(θ) i=1 T (xi ) − nB(θ)] ∗ ∗ ∗ ∗ = h (x)expfη (θ)T (x) − B (θ)g where: ∗ Q n h (x) = i=1 h(xi ) ∗ η (θ) = η(θ) ∗ Pn T (x) = i=1 T (xi ) ∗ B (θ) = nB(θ) ∗ Note: The Sufficient Statsitic T is one-dimensional for all n. 8 MIT 18.655 Exponential Families One Parameter Exponential Family Exponential Families Multiparameter Exponential Family Building Exponential Families Samples from One-Parameter Exponential Family Distribution Theorem 1.6.1 Let fPθg be a one-parameter exponential family of discrete distributions with pmf function: p(x j θ) = h(x)expfη(θ)T (x) − B(θ)g Then the family of distributions of the statistic T (X ) is a one-parameter exponential family of discrete distributions whose frequency functions are ∗∗ Pθ(T (x) = t) = p(t j θ) = h (t)expfη(θ)t − B(θ)g where ∗∗ X h (t) = h(x) fx:T (x)=tg Proof: Immediate 9 MIT 18.655 Exponential Families One Parameter Exponential Family Exponential Families Multiparameter Exponential Family Building Exponential Families Canonical Exponential Family Re-parametrize setting η = η(θ) { the Natural Parameter The density has the form p(x; η) = h(x)expfηT (x) − A(η)g The function A(η) replaces B(θ) and is defined as the normalization constant: log(A(η)) = RR ··· R h(x)expfηT (x)]dx if X continuous or X log(A(η)) = h(x)expfηT (x)] x2X if X discrete The Natural Parameter Space fη : η = η(θ); θ 2 Θg = E (Later, Theorem 1.6.3 gives properties of E) T (x) is the Natural Sufficient Statistic. 10 MIT 18.655 Exponential Families One Parameter Exponential Family Exponential Families Multiparameter Exponential Family Building Exponential Families Canonical Representation of Poisson Family Poisson Distribution (1.6.1) : X ∼ Poisson(θ); where E[X ] = θ: θx −θ p(x j θ) = x! e ; x = 0; 1;::: 1 = x! expf(log(θ)x − θg = h(x)expfη(θ)T (x) − B(θ)g where: 1 h(x) = x! η(θ) = log(θ) T (x) = x Canonical Representation η = log(θ): η A(η) = B(θ) = θ = e . 11 MIT 18.655 Exponential Families One Parameter Exponential Family Exponential Families Multiparameter Exponential Family Building Exponential Families MGFs of Canonical Exponenetial Family Models Theorem 1.6.2 Suppose X is distributued according to a canonical exponential family, i.e., the density/pmf function is given by q p(x j η) = h(x)exp[ηT (x) − A(η)]; for x 2 X ⊂ R : If η is an interior point of E, the natural parameter space, then The moment generating function of T (X ) exists and is given by sT (X ) MT (s) = E [e j η] = expfA(s + η) − A(η)g for s in some neighborhood of 0: 0 E [T (X ) j η] = A (η): 00 Var[T (X ) j η] = A (η): Proof: sT (X ) RR (s+η)T (x)−A(η) MT (s) = E [e ) j η] = ··· h(x)e dx [A(s+η)−A(s)] RR (s+η)T (x)−A(s+η) = [e ] × ··· h(x)e dx = [e[A(s+η)−A(s)]] × 1 Remainder follows from properties of MFGs. 12 MIT 18.655 Exponential Families One Parameter Exponential Family Exponential Families Multiparameter Exponential Family Building Exponential Families Moments of Canonical Exponential Family Distributions η Poisson Distribution: A(η) = B(θ) = θ = e . 0 η E (X j θ) = A (η) = e = θ: 00 η Var(X j θ) = A (η) = e = θ: Binomial Distribution: r n X n−x p(x j θ) = θ (1 − θ) x θ = h(x)expflog()(1−θ) x + nlog(1 − θ)g η = h(x)expfηx − nlog(e + 1)g η θ So A(η) = nlog(e + 1; ) with η = log()1−θ 0 eη A (η) = n eη +1 = nθ 00 1 η η −1 η A (η) = neη +1 e + ne × (eη +1)2 e eη eη = n[]eη +1 × (1 − (eη +1) ) = nθ(1 − θ) 13 MIT 18.655 Exponential Families One Parameter Exponential Family Exponential Families Multiparameter Exponential Family Building Exponential Families Moments of the Gamma Distribution X1;:::; Xn i.i.d Gamma(p; λ) distribution with density λp xp−1 e−λx p(x j λ, p) = Γ(p) , 0 < x < 1 where R 1 p p−1 −λx Γ(p) = 0 λ x e dx xp−1 p(x j λ, p) = [ Γ(p) ]exp{−λx + plog(λ)g = h(x)expfηT (x) − A(η)g where η = −λ A(η) = −plog(λ) = −plog(−η) Thus 0 E (X ) = A (η) = −p/η = p/λ 00 2 2 Var(X ) = A (η) = (p/η ) = p/λ 14 MIT 18.655 Exponential Families One Parameter Exponential Family Exponential Families Multiparameter Exponential Family Building Exponential Families Notes on Gamma Distribution Gamma(p = n=2; λ = 1=2) corresponds to the Chi-Squared distribution with n degrees of freedom. p = 2 corresponds to the Exponential Distribution p For p = 1; Γ(1=2) = π Γ(p + 1) = pΓ(p) for positive integer p. 15 MIT 18.655 Exponential Families One Parameter Exponential Family Exponential Families Multiparameter Exponential Family Building Exponential Families Rayleigh Distribution Sample X1 :::; Xn iid with density function x 2 2 p(x j θ) = θ2 exp(−x =2θ ) 1 2 2 = [x] × exp{− 2θ2 x − log(θ )g = h(x)expfηT (x) − A(η)g where 1 η = − 2θ2 2 T (X ) = X : 2 −1 A(η) = log(θ ) = log( 2η ) = −log(−2η) By the mgf 2 0 2 1 2 E (X ) = A (η) = − 2η= − η = 2θ 2 00 1 4 Var(X ) = A (η) = + η2 = 4θ For the n− sample: X = (X1;:::; Xn) Pn 2 T (X) = 1 X i 2 E [T (X)] = −n/η = 2nθ 1 4 Var[T (X)] = n η2 = 4nθ : x2 Note: P(X ≤ x) = 1 − exp{− 2θ2 g (Failure time model) 16 MIT 18.655 Exponential Families One Parameter Exponential Family Exponential Families Multiparameter Exponential Family Building Exponential Families Outline 1 Exponential Families One Parameter Exponential Family Multiparameter Exponential Family Building Exponential Families 17 MIT 18.655 Exponential Families One Parameter Exponential Family Exponential Families Multiparameter Exponential Family Building Exponential Families Definition k fPθ; θ 2 Θg,Θ ⊂ R , is a k-parameter exponential family if the the density/pmf function of X ∼ Pθ is k X p(x j θ) = h(x)exp[ ηj (θ)Tj (x) − B(θ)], j=1 q where x 2 X ⊂ R ; and η1; : : : ; ηk and B are real-valued functions mappying Θ ! R.
Load more

Recommended publications
  • A Random Variable X with Pdf G(X) = Λα Γ(Α) X ≥ 0 Has Gamma
    DISTRIBUTIONS DERIVED FROM THE NORMAL DISTRIBUTION Definition: A random variable X with pdf λα g(x) = xα−1e−λx x ≥ 0 Γ(α) has gamma distribution with parameters α > 0 and λ > 0. The gamma function Γ(x), is defined as Z ∞ Γ(x) = ux−1e−udu. 0 Properties of the Gamma Function: (i) Γ(x + 1) = xΓ(x) (ii) Γ(n + 1) = n! √ (iii) Γ(1/2) = π. Remarks: 1. Notice that an exponential rv with parameter 1/θ = λ is a special case of a gamma rv. with parameters α = 1 and λ. 2. The sum of n independent identically distributed (iid) exponential rv, with parameter λ has a gamma distribution, with parameters n and λ. 3. The sum of n iid gamma rv with parameters α and λ has gamma distribution with parameters nα and λ. Definition: If Z is a standard normal rv, the distribution of U = Z2 called the chi-square distribution with 1 degree of freedom. 2 The density function of U ∼ χ1 is −1/2 x −x/2 fU (x) = √ e , x > 0. 2π 2 Remark: A χ1 random variable has the same density as a random variable with gamma distribution, with parameters α = 1/2 and λ = 1/2. Definition: If U1,U2,...,Uk are independent chi-square rv-s with 1 degree of freedom, the distribution of V = U1 + U2 + ... + Uk is called the chi-square distribution with k degrees of freedom. 2 Using Remark 3. and the above remark, a χk rv. follows gamma distribution with parameters 2 α = k/2 and λ = 1/2.
    [Show full text]
  • Stat 5101 Notes: Brand Name Distributions
    Stat 5101 Notes: Brand Name Distributions Charles J. Geyer February 14, 2003 1 Discrete Uniform Distribution Symbol DiscreteUniform(n). Type Discrete. Rationale Equally likely outcomes. Sample Space The interval 1, 2, ..., n of the integers. Probability Function 1 f(x) = , x = 1, 2, . , n n Moments n + 1 E(X) = 2 n2 − 1 var(X) = 12 2 Uniform Distribution Symbol Uniform(a, b). Type Continuous. Rationale Continuous analog of the discrete uniform distribution. Parameters Real numbers a and b with a < b. Sample Space The interval (a, b) of the real numbers. 1 Probability Density Function 1 f(x) = , a < x < b b − a Moments a + b E(X) = 2 (b − a)2 var(X) = 12 Relation to Other Distributions Beta(1, 1) = Uniform(0, 1). 3 Bernoulli Distribution Symbol Bernoulli(p). Type Discrete. Rationale Any zero-or-one-valued random variable. Parameter Real number 0 ≤ p ≤ 1. Sample Space The two-element set {0, 1}. Probability Function ( p, x = 1 f(x) = 1 − p, x = 0 Moments E(X) = p var(X) = p(1 − p) Addition Rule If X1, ..., Xk are i. i. d. Bernoulli(p) random variables, then X1 + ··· + Xk is a Binomial(k, p) random variable. Relation to Other Distributions Bernoulli(p) = Binomial(1, p). 4 Binomial Distribution Symbol Binomial(n, p). 2 Type Discrete. Rationale Sum of i. i. d. Bernoulli random variables. Parameters Real number 0 ≤ p ≤ 1. Integer n ≥ 1. Sample Space The interval 0, 1, ..., n of the integers. Probability Function n f(x) = px(1 − p)n−x, x = 0, 1, . , n x Moments E(X) = np var(X) = np(1 − p) Addition Rule If X1, ..., Xk are independent random variables, Xi being Binomial(ni, p) distributed, then X1 + ··· + Xk is a Binomial(n1 + ··· + nk, p) random variable.
    [Show full text]
  • Effect of Probability Distribution of the Response Variable in Optimal Experimental Design with Applications in Medicine †
    mathematics Article Effect of Probability Distribution of the Response Variable in Optimal Experimental Design with Applications in Medicine † Sergio Pozuelo-Campos *,‡ , Víctor Casero-Alonso ‡ and Mariano Amo-Salas ‡ Department of Mathematics, University of Castilla-La Mancha, 13071 Ciudad Real, Spain; [email protected] (V.C.-A.); [email protected] (M.A.-S.) * Correspondence: [email protected] † This paper is an extended version of a published conference paper as a part of the proceedings of the 35th International Workshop on Statistical Modeling (IWSM), Bilbao, Spain, 19–24 July 2020. ‡ These authors contributed equally to this work. Abstract: In optimal experimental design theory it is usually assumed that the response variable follows a normal distribution with constant variance. However, some works assume other probability distributions based on additional information or practitioner’s prior experience. The main goal of this paper is to study the effect, in terms of efficiency, when misspecification in the probability distribution of the response variable occurs. The elemental information matrix, which includes information on the probability distribution of the response variable, provides a generalized Fisher information matrix. This study is performed from a practical perspective, comparing a normal distribution with the Poisson or gamma distribution. First, analytical results are obtained, including results for the linear quadratic model, and these are applied to some real illustrative examples. The nonlinear 4-parameter Hill model is next considered to study the influence of misspecification in a Citation: Pozuelo-Campos, S.; dose-response model. This analysis shows the behavior of the efficiency of the designs obtained in Casero-Alonso, V.; Amo-Salas, M.
    [Show full text]
  • Chapter 6 Continuous Random Variables and Probability
    EF 507 QUANTITATIVE METHODS FOR ECONOMICS AND FINANCE FALL 2019 Chapter 6 Continuous Random Variables and Probability Distributions Chap 6-1 Probability Distributions Probability Distributions Ch. 5 Discrete Continuous Ch. 6 Probability Probability Distributions Distributions Binomial Uniform Hypergeometric Normal Poisson Exponential Chap 6-2/62 Continuous Probability Distributions § A continuous random variable is a variable that can assume any value in an interval § thickness of an item § time required to complete a task § temperature of a solution § height in inches § These can potentially take on any value, depending only on the ability to measure accurately. Chap 6-3/62 Cumulative Distribution Function § The cumulative distribution function, F(x), for a continuous random variable X expresses the probability that X does not exceed the value of x F(x) = P(X £ x) § Let a and b be two possible values of X, with a < b. The probability that X lies between a and b is P(a < X < b) = F(b) -F(a) Chap 6-4/62 Probability Density Function The probability density function, f(x), of random variable X has the following properties: 1. f(x) > 0 for all values of x 2. The area under the probability density function f(x) over all values of the random variable X is equal to 1.0 3. The probability that X lies between two values is the area under the density function graph between the two values 4. The cumulative density function F(x0) is the area under the probability density function f(x) from the minimum x value up to x0 x0 f(x ) = f(x)dx 0 ò xm where
    [Show full text]
  • 1. How Different Is the T Distribution from the Normal?
    Statistics 101–106 Lecture 7 (20 October 98) c David Pollard Page 1 Read M&M §7.1 and §7.2, ignoring starred parts. Reread M&M §3.2. The eects of estimated variances on normal approximations. t-distributions. Comparison of two means: pooling of estimates of variances, or paired observations. In Lecture 6, when discussing comparison of two Binomial proportions, I was content to estimate unknown variances when calculating statistics that were to be treated as approximately normally distributed. You might have worried about the effect of variability of the estimate. W. S. Gosset (“Student”) considered a similar problem in a very famous 1908 paper, where the role of Student’s t-distribution was first recognized. Gosset discovered that the effect of estimated variances could be described exactly in a simplified problem where n independent observations X1,...,Xn are taken from (, ) = ( + ...+ )/ a normal√ distribution, N . The sample mean, X X1 Xn n has a N(, / n) distribution. The random variable X Z = √ / n 2 2 Phas a standard normal distribution. If we estimate by the sample variance, s = ( )2/( ) i Xi X n 1 , then the resulting statistic, X T = √ s/ n no longer has a normal distribution. It has a t-distribution on n 1 degrees of freedom. Remark. I have written T , instead of the t used by M&M page 505. I find it causes confusion that t refers to both the name of the statistic and the name of its distribution. As you will soon see, the estimation of the variance has the effect of spreading out the distribution a little beyond what it would be if were used.
    [Show full text]
  • Lecture.7 Poisson Distributions - Properties, Normal Distributions- Properties
    Lecture.7 Poisson Distributions - properties, Normal Distributions- properties Theoretical Distributions Theoretical distributions are 1. Binomial distribution Discrete distribution 2. Poisson distribution 3. Normal distribution Continuous distribution Discrete Probability distribution Bernoulli distribution A random variable x takes two values 0 and 1, with probabilities q and p ie., p(x=1) = p and p(x=0)=q, q-1-p is called a Bernoulli variate and is said to be Bernoulli distribution where p and q are probability of success and failure. It was given by Swiss mathematician James Bernoulli (1654-1705) Example • Tossing a coin(head or tail) • Germination of seed(germinate or not) Binomial distribution Binomial distribution was discovered by James Bernoulli (1654-1705). Let a random experiment be performed repeatedly and the occurrence of an event in a trial be called as success and its non-occurrence is failure. Consider a set of n independent trails (n being finite), in which the probability p of success in any trail is constant for each trial. Then q=1-p is the probability of failure in any trail. 1 The probability of x success and consequently n-x failures in n independent trails. But x successes in n trails can occur in ncx ways. Probability for each of these ways is pxqn-x. P(sss…ff…fsf…f)=p(s)p(s)….p(f)p(f)…. = p,p…q,q… = (p,p…p)(q,q…q) (x times) (n-x times) Hence the probability of x success in n trials is given by x n-x ncx p q Definition A random variable x is said to follow binomial distribution if it assumes non- negative values and its probability mass function is given by P(X=x) =p(x) = x n-x ncx p q , x=0,1,2…n q=1-p 0, otherwise The two independent constants n and p in the distribution are known as the parameters of the distribution.
    [Show full text]
  • Skewed Double Exponential Distribution and Its Stochastic Rep- Resentation
    EUROPEAN JOURNAL OF PURE AND APPLIED MATHEMATICS Vol. 2, No. 1, 2009, (1-20) ISSN 1307-5543 – www.ejpam.com Skewed Double Exponential Distribution and Its Stochastic Rep- resentation 12 2 2 Keshav Jagannathan , Arjun K. Gupta ∗, and Truc T. Nguyen 1 Coastal Carolina University Conway, South Carolina, U.S.A 2 Bowling Green State University Bowling Green, Ohio, U.S.A Abstract. Definitions of the skewed double exponential (SDE) distribution in terms of a mixture of double exponential distributions as well as in terms of a scaled product of a c.d.f. and a p.d.f. of double exponential random variable are proposed. Its basic properties are studied. Multi-parameter versions of the skewed double exponential distribution are also given. Characterization of the SDE family of distributions and stochastic representation of the SDE distribution are derived. AMS subject classifications: Primary 62E10, Secondary 62E15. Key words: Symmetric distributions, Skew distributions, Stochastic representation, Linear combina- tion of random variables, Characterizations, Skew Normal distribution. 1. Introduction The double exponential distribution was first published as Laplace’s first law of error in the year 1774 and stated that the frequency of an error could be expressed as an exponential function of the numerical magnitude of the error, disregarding sign. This distribution comes up as a model in many statistical problems. It is also considered in robustness studies, which suggests that it provides a model with different characteristics ∗Corresponding author. Email address: (A. Gupta) http://www.ejpam.com 1 c 2009 EJPAM All rights reserved. K. Jagannathan, A. Gupta, and T. Nguyen / Eur.
    [Show full text]
  • 5. the Student T Distribution
    Virtual Laboratories > 4. Special Distributions > 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 5. The Student t Distribution In this section we will study a distribution that has special importance in statistics. In particular, this distribution will arise in the study of a standardized version of the sample mean when the underlying distribution is normal. The Probability Density Function Suppose that Z has the standard normal distribution, V has the chi-squared distribution with n degrees of freedom, and that Z and V are independent. Let Z T= √V/n In the following exercise, you will show that T has probability density function given by −(n +1) /2 Γ((n + 1) / 2) t2 f(t)= 1 + , t∈ℝ ( n ) √n π Γ(n / 2) 1. Show that T has the given probability density function by using the following steps. n a. Show first that the conditional distribution of T given V=v is normal with mean 0 a nd variance v . b. Use (a) to find the joint probability density function of (T,V). c. Integrate the joint probability density function in (b) with respect to v to find the probability density function of T. The distribution of T is known as the Student t distribution with n degree of freedom. The distribution is well defined for any n > 0, but in practice, only positive integer values of n are of interest. This distribution was first studied by William Gosset, who published under the pseudonym Student. In addition to supplying the proof, Exercise 1 provides a good way of thinking of the t distribution: the t distribution arises when the variance of a mean 0 normal distribution is randomized in a certain way.
    [Show full text]
  • On a Problem Connected with Beta and Gamma Distributions by R
    ON A PROBLEM CONNECTED WITH BETA AND GAMMA DISTRIBUTIONS BY R. G. LAHA(i) 1. Introduction. The random variable X is said to have a Gamma distribution G(x;0,a)if du for x > 0, (1.1) P(X = x) = G(x;0,a) = JoT(a)" 0 for x ^ 0, where 0 > 0, a > 0. Let X and Y be two independently and identically distributed random variables each having a Gamma distribution of the form (1.1). Then it is well known [1, pp. 243-244], that the random variable W = X¡iX + Y) has a Beta distribution Biw ; a, a) given by 0 for w = 0, (1.2) PiW^w) = Biw;x,x)=\ ) u"-1il-u)'-1du for0<w<l, Ío T(a)r(a) 1 for w > 1. Now we can state the converse problem as follows : Let X and Y be two independently and identically distributed random variables having a common distribution function Fix). Suppose that W = Xj{X + Y) has a Beta distribution of the form (1.2). Then the question is whether £(x) is necessarily a Gamma distribution of the form (1.1). This problem was posed by Mauldon in [9]. He also showed that the converse problem is not true in general and constructed an example of a non-Gamma distribution with this property using the solution of an integral equation which was studied by Goodspeed in [2]. In the present paper we carry out a systematic investigation of this problem. In §2, we derive some general properties possessed by this class of distribution laws Fix).
    [Show full text]
  • 1 One Parameter Exponential Families
    1 One parameter exponential families The world of exponential families bridges the gap between the Gaussian family and general dis- tributions. Many properties of Gaussians carry through to exponential families in a fairly precise sense. • In the Gaussian world, there exact small sample distributional results (i.e. t, F , χ2). • In the exponential family world, there are approximate distributional results (i.e. deviance tests). • In the general setting, we can only appeal to asymptotics. A one-parameter exponential family, F is a one-parameter family of distributions of the form Pη(dx) = exp (η · t(x) − Λ(η)) P0(dx) for some probability measure P0. The parameter η is called the natural or canonical parameter and the function Λ is called the cumulant generating function, and is simply the normalization needed to make dPη fη(x) = (x) = exp (η · t(x) − Λ(η)) dP0 a proper probability density. The random variable t(X) is the sufficient statistic of the exponential family. Note that P0 does not have to be a distribution on R, but these are of course the simplest examples. 1.0.1 A first example: Gaussian with linear sufficient statistic Consider the standard normal distribution Z e−z2=2 P0(A) = p dz A 2π and let t(x) = x. Then, the exponential family is eη·x−x2=2 Pη(dx) / p 2π and we see that Λ(η) = η2=2: eta= np.linspace(-2,2,101) CGF= eta**2/2. plt.plot(eta, CGF) A= plt.gca() A.set_xlabel(r'$\eta$', size=20) A.set_ylabel(r'$\Lambda(\eta)$', size=20) f= plt.gcf() 1 Thus, the exponential family in this setting is the collection F = fN(η; 1) : η 2 Rg : d 1.0.2 Normal with quadratic sufficient statistic on R d As a second example, take P0 = N(0;Id×d), i.e.
    [Show full text]
  • Random Variables and Probability Distributions 1.1
    RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS 1. DISCRETE RANDOM VARIABLES 1.1. Definition of a Discrete Random Variable. A random variable X is said to be discrete if it can assume only a finite or countable infinite number of distinct values. A discrete random variable can be defined on both a countable or uncountable sample space. 1.2. Probability for a discrete random variable. The probability that X takes on the value x, P(X=x), is defined as the sum of the probabilities of all sample points in Ω that are assigned the value x. We may denote P(X=x) by p(x) or pX (x). The expression pX (x) is a function that assigns probabilities to each possible value x; thus it is often called the probability function for the random variable X. 1.3. Probability distribution for a discrete random variable. The probability distribution for a discrete random variable X can be represented by a formula, a table, or a graph, which provides pX (x) = P(X=x) for all x. The probability distribution for a discrete random variable assigns nonzero probabilities to only a countable number of distinct x values. Any value x not explicitly assigned a positive probability is understood to be such that P(X=x) = 0. The function pX (x)= P(X=x) for each x within the range of X is called the probability distribution of X. It is often called the probability mass function for the discrete random variable X. 1.4. Properties of the probability distribution for a discrete random variable.
    [Show full text]
  • ONE SAMPLE TESTS the Following Data Represent the Change
    1 WORKED EXAMPLES 6 INTRODUCTION TO STATISTICAL METHODS EXAMPLE 1: ONE SAMPLE TESTS The following data represent the change (in ml) in the amount of Carbon monoxide transfer (an indicator of improved lung function) in smokers with chickenpox over a one week period: 33, 2, 24, 17, 4, 1, -6 Is there evidence of significant improvement in lung function (a) if the data are normally distributed with σ = 10, (b) if the data are normally distributed with σ unknown? Use a significance level of α = 0.05. SOLUTION: (a) Here we have a sample of size 7 with sample mean x = 10.71. We want to test H0 : μ = 0.0, H1 : μ = 0.0, 6 under the assumption that the data follow a Normal distribution with σ = 10.0 known. Then, we have, in the Z-test, 10.71 0.0 z = − = 2.83, 10.0/√7 which lies in the critical region, as the critical values for this test are 1.96, for significance ± level α = 0.05. Therefore we have evidence to reject H0. The p-value is given by p = 2Φ( 2.83) = 0.004 < α. − (b) The sample variance is s2 = 14.192. In the T-test, we have test statistic t given by x 0.0 10.71 0.0 t = − = − = 2.00. s/√n 14.19/√7 The upper critical value CR is obtained by solving FSt(n 1)(CR) = 0.975, − where FSt(n 1) is the cdf of a Student-t distribution with n 1 degrees of freedom; here n = 7, so − − we can use statistical tables or a computer to find that CR = 2.447, and note that, as Student-t distributions are symmetric the lower critical value is CR.
    [Show full text]