Univariate Continuous Distribution Theory

M347 Mathematical statistics Univariate continuous distribution theory Contents 1 Pdfs and cdfs 3 1.1 Densities and normalising constants 3 1.1.1 Examples 4 1.2 Distribution functions and densities 6 1.2.1 Examples 8 1.3 Probabilities of lying within intervals 10 2 Moments 11 2.1 Expectation 11 2.2 The mean 13 2.3 Raw moments 14 2.4 Central moments 15 2.5 The variance 16 2.6 Linear transformation 18 2.6.1 Expectation and variance 18 2.6.2 Effects on the pdf 19 2.7 The moment generating function 21 2.7.1 Examples 22 Solutions 24 This publication forms part of an Open University module. Details of this and other Open University modules can be obtained from the Student Registration and Enquiry Service, The Open University, PO Box 197, Milton Keynes MK7 6BJ, United Kingdom (tel. +44 (0)845 300 6090; email [email protected]). Alternatively, you may visit the Open University website at www.open.ac.uk where you can learn more about the wide range of modules and packs offered at all levels by The Open University. To purchase a selection of Open University materials visit www.ouw.co.uk, or contact Open University Worldwide, Walton Hall, Milton Keynes MK7 6AA, United Kingdom for a brochure (tel. +44 (0)1908 858793; fax +44 (0)1908 858787; email [email protected]). Note to reader Mathematical/statistical content at the Open University is usually provided to students in printed books, with PDFs of the same online. This format ensures that mathematical notation is presented accurately and clearly. The PDF of this extract thus shows the content exactly as it would be seen by an Open University student. Please note that the PDF may contain references to other parts of the module and/or to software or audio-visual components of the module. Regrettably mathematical and statistical content in PDF files is unlikely to be accessible using a screenreader, and some OpenLearn units may have PDF files that are not searchable. You may need additional help to read these documents. The Open University, Walton Hall, Milton Keynes, MK7 6AA. First published 2015. Copyright c 2015 The Open University All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, transmitted or utilised in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without written permission from the publisher or a licence from the Copyright Licensing Agency Ltd. Details of such licences (for reprographic reproduction) may be obtained from the Copyright Licensing Agency Ltd, Saffron House, 6–10 Kirby Street, London EC1N 8TS (website www.cla.co.uk). Open University materials may also be made available in electronic formats for use by students of the University. All rights, including copyright and related rights and database rights, in electronic materials and their contents are owned by or licensed to The Open University, or otherwise used by The Open University as permitted by applicable law. In using electronic materials and their contents you agree that your use will be solely for the purposes of following an Open University course of study or otherwise as licensed by The Open University or its assigns. Except as permitted above you undertake not to copy, store in any medium (including electronic storage or use in a website), distribute, transmit or retransmit, broadcast, modify or show in public such electronic materials in whole or in part without the prior written consent of The Open University or in accordance with the Copyright, Designs and Patents Act 1988. Edited, designed and typeset by The Open University, using the Open University TEX System. Printed in the United Kingdom by Martins the Printers, Berwick upon Tweed. ISBN 978 1 4730 0341 5 1.1 1 Pdfs and cdfs 1 Pdfs and cdfs The distributions of continuous random variables are described by their ‘probability density functions (pdfs)’ and ‘cumulative distribution functions (cdfs)’. Pdfs are the topic of Subsection 1.1, where their basic properties are described and examples are developed. Cdfs, including their properties, some examples and their relationship to pdfs, are considered in Subsection 1.2. Finally, you will be reminded how differences between cdf values give probabilities of a random variable lying within an interval in Subsection 1.3. 1.1 Densities and normalising constants A continuous random variable X follows a distribution with probability density function, or pdf (or sometimes just density function or This pdf has nothing to do with density), f say. the ‘portable document format’ PDF files on your computer. The key properties of a pdf f are that it is non-negative and integrates to 1, that is, ∞ f(x) 0 for all x R and f(x) dx = 1. ≥ ∈ −∞ Z f(x) area = 1 0 x Figure 2.1 A pdf f(x) plotted as a function of x Note that in Figure 2.1, f(x) 0 for all x. Also, its integral, which is the area of the shaded region under≥ the pdf, is 1. In fact, any mathematical function which is non-negative, positive on at least one interval of values of x, and has a finite integral can be made into a pdf. (A function whose integral does not exist cannot be made into a pdf.) Suppose that g is such a function with ∞ g(x) dx = C, −∞ Z where 0 <C< . Then ∞ g(x) f(x)= C is a pdf. To see this, note that ∞ ∞ ∞ g(x) 1 1 f(x) dx = dx = g(x) dx = C = 1. −∞ −∞ C C −∞ C × Z Z Z C is called the normalising (or normalisation) constant. Some statisticians use the term ‘normalising constant’ for the The following nomenclature is not entirely standard in statistics, but can quantity 1/C instead of C. be useful. The part g of the pdf f that contains all the dependence of f on x will be referred to as the density core, or just core for short, in M347. 3 Univariate continuous distribution theory One thing to bear in mind is that a pdf is not a probability itself. In particular, f(x) = P (X = x). Indeed, for a continuous distribution, P (X = x) equals6 zero! However, there is a closely related probability. The probability that X lies within a specific interval [x0, x0 + ε), where ε > 0 is small, is approximately equal to ε times the density at x0 (see Figure 2.2). Formally, P (x X < x + ε) 0 ≤ 0 f(x ) as ε 0. ε → 0 → f(x) f(x0) 0 x x0 x0 + Figure 2.2 The pdf f(x) of Figure 2.1 with the area under the pdf and between x0 and x0 + ε shaded 1.1.1 Examples Example 2.1 The pdf of the uniform distribution on (0, 1) The distribution with density f which is constant on 0 < x < 1 and zero otherwise is called the uniform distribution on (0, 1), denoted U(0, 1). The constant value, k say, must be positive for f to be a density. What is the value of k? Well, the density is 0 for x 0, k for 0 < x < 1 and 0 again for x 1. So, splitting the range of integration≤ into three parts gives ≥ ∞ 0 1 ∞ f(x) dx = f(x) dx + f(x) dx + f(x) dx −∞ −∞ Z Z Z0 Z1 0 1 ∞ = 0 dx + k dx + 0 dx −∞ Z Z0 Z1 1 = 0 + k dx + 0 0 1 Z = kx dx = [k ]1 = k(1 0)= k. 0 − Z0 Integration of a constant However, this integral should be 1, so it must be that k = 1. That is, 1 if 0 < x < 1, f(x) = 0 otherwise. For short, it can be said that f(x) = 1 on 0 < x < 1. 4 1 Pdfs and cdfs The uniform pdf is shown in Figure 2.3. f(x) 1 0 1 0 1 2 x − Figure 2.3 Graph of the pdf of the uniform distribution on (0, 1) Notice that you could have replaced the integration step in calculating k by using the area of the square in Figure 2.3. When a pdf only takes positive values on a subset of R, then the limits of integration reduce to the limits of that subset. In Example 2.1, these limits were 0 and 1. In general, the domain on which f takes positive values is called the support of the distribution. Thus, in Example 2.1, the support of f is (0, 1) and the density is written in the form f(x)= function of x on x in its support. { } And, in general, the integral of f is the integral of f over its support. In M347, the support of f will always be a single interval, although one or both endpoints of that interval may be . ±∞ Example 2.2 The pdf of the exponential distribution The exponential distribution, M(λ), with parameter λ> 0, is claimed to have density − f(x)= λe λx on x> 0. (Thus, implicitly, f(x)=0 for x 0.) Is this truly a density? Well the exponential function is non-negative≤ everywhere, so f certainly is. Does it integrate to 1? Yes, it does: noting that the support of f is (0, ), ∞ ∞ ∞ − f(x) dx = λe λx dx −∞ 0 Integration of an exponential Z Z ∞ − = λ e λx dx 0 Z ∞ ∞ 1 − − = λ e λx = e λx = 0 ( 1) = 1. −λ 0 − 0 − − h i 5 Univariate continuous distribution theory The exponential pdf is shown in Figure 2.4 for λ = 1.

Load more