Chapter 5 The normal distribution This chapter deals in detail with one of the most versatile models for variation, the normal distribution or 'bell-shaped' curve. You will learn how to use printed tables to calculate normal probabilities. The normal curve also provides a useful approxi- mation to other probability distributions: this is one of the consequences of the central limit theorem. In Chapter 2, Section 2.4 you were introduced to an important continuous distribution called the normal distribution. It was noted that many real data sets can reasonably be treated as though they were a random sample from the normal distribution and it was remarked that the normal distribution turns out to play a central role in statistical theory as well as in practice. This entire chapter is devoted to the study of the normal distribution. The chapter begins with a review of all that has been said so far about the normal distribution. The main point to bear in mind is that in many cases a probability model for random variation follows necessarily as a mathematical consequence of certain assumptions: for instance, many random processes can be modelled as sets or sequences of Bernoulli trials, the distribution theory following from the twin assumptions that the trials are independent and that the probability of success from trial to trial remains constant. Quite often, however, data arise from a situation for which no model has been proposed: nevertheless, even when the data sets arise from entirely different sampling contexts, they often seem to acquire a characteristic peaked and symmetric shape that is essentially the same. This shape may often be adequately rep- resented through a normal model. The review is followed by an account of the genesis of the normal distribution. In Section 5.2, you will discover how to calculate normal probabilities. As for any other continuous probability distribution, probabilities are found by calculating areas under the curve of the probability density function. But for the normal distribution, this is not quite straightforward, because applying the technique of integration does not in this case lead to a formula that is easy to write down. So, in practice, probabilities are found by referring to printed tables, or by using a computer. The remaining sections of the chapter deal with one of the fundamental the- orems in statistics and with some of the consequences of it. It is called the central limit theorem. This is a theorem due to Pierre Simon Laplace (1749- 1827) that was read before the Academy of Sciences in Paris on 9 April 1810. The theorem is a major mathematical statement: however, we shall be con- cerned not with the details of its proof, but with its application to statistical problems. Elements of Statistics 5.1 Some history 5.1.1 Review The review begins with a set of data collected a long time ago. During the mapping of the state of Massachusetts in America, one hundred readings were taken on the error involved when measuring angles. The error was measured in minutes (a minute is 1/60 of a degree). The data are shown in Table 5.1. Table 5.1 Errors in angular measurements United States Coast Survey Report (1854). The error was calculated Error (in minutes) Frequency by subtracting each measurement Between +6 and +5 1 from 'the most probable' value. 2 Between +5 and +4 Frequency Between +4 and +3 2 Between +3 and +2 3 Between +2 and +l 13 Between +l and 0 26 Between 0 and -1 26 Between -1 and -2 17 Between -2 and -3 8 Between -3 and -4 2 10 A histogram of this sample is given in Figure 5.1. This graphical represen- tation shows clearly the main characteristics of the data: the histogram is unimodal (it possesses just one mode) and it is roughly symmetric about that mode. -4.0 -2.0 0.0 2.0 4.0 6.0 Error (minutes) Another histogram, which corresponds to a different data set, is shown in Figure 5.1 Errors in angular Figure 5.2. You have seen these data before. measurements (minutes) This is a graphical representation of the sample of Scottish soldiers' chest Frequency measurements that you met in Chapter 2, Section 2.4. This histogram is also unimodal and roughly symmetric. The common characteristics of the shape of both the histograms in Figures 5.1 and 5.2 are shared with the normal distribution whose p.d.f. is illustrated in Figure 5.3. 800 34 36 38 40 42 44 46 48 Chest (inches) Figure 5.2 Chest measurements of Scottish soldiers (inches) Figure 5.9 The normal p.d.f. For clarity, the vertical axis has been omitted in this graph of the normal density function. What is it about Figures 5.1, 5.2 and 5.3 that makes them appear similar? Well, each diagram starts at a low level on the left-hand side, rises steadily Chapter 5 Section 5.1 until reaching a maximum in the centre and then decreases, at the same rate that it increased, to a low value towards the right-hand side. The diagrams are unimodal and symmetric about their modes (although this symmetry is only approximate for the two data sets). A single descriptive word often used to describe the shape of the normal p.d.f., and likewise histograms of data sets that might be adequately modelled by the normal distribution, is 'bell-shaped'. Note that there is more than one normal distribution. No single distribution could possibly describe both the data of Figure 5.1, which have their mode around zero and which vary from about -4 minutes of arc to over 5 minutes, and those of Figure 5.2, whose mode is at about 40inches and which range from approximately 33inches to 48inches. In the real world there are many instances of random variation following this kind of pattern: the mode and the range of observed values will alter from random variable to random variable, but the characteristic bell shape of the data will be apparent. The four probability density functions shown in Figure 5.4 all correspond to different normal distributions. Figure 5.4 Four normal densities What has been described is another family of probability models, just like the binomial family (with two parameters, n and p) and the Poisson family (with one parameter, the mean p). The normal family has two parameters, one specifying location (the centre of the distribution) and one describing the degree of dispersion. In Chapter 2 the location parameter was denoted by p Elements of Statistics and the dispersion parameter was denoted by a; in fact, the parameter p is the mean of the normal distribution and a is its standard deviation. This information may be summarized as follows. The probability density function for the normal family of random variables is also given. The normal probability density function If the continuous random variable X is normally distributed with mean p and standard deviation a (variance a') then this may be written X N(P,0'); the probability density function of X is given by 1 1 2-11 f(z) = ~exp[-i (a)2] , -W < X < W. (5.1) As remarked already, you do not need to remember this formula in order to calculate normal A sketch of the p.d.f. of X is as follows. probabilities. The shape of the density function of X is often called 'bell-shaped'. The p.d.f. of X is positive for all values of X; however, observations more than about three standard deviations away from the mean are rather unlikely. The total area under the curve is 1. There are very few random variables for which possible observations include all negative and positive numbers. But for the normal distribution, extreme values may be regarded as occurring with negligible probability. One should not say 'the variation in Scottish chest measurements is normally distributed with mean 40 inches and standard deviation about 2 inches' (the implication being that negative observations are possible); rather, say 'the variation in Scottish chest measurements may be adequately modelled by a normal distri- bution with mean 40 inches and standard deviation 2 inches'. In the rest of this chapter we shall see many more applications in the real world where different members of the normal family provide good models of variation. But first, we shall explore some of the history of the development of the normal distribution. 186 Chapter 5 Section 5.1 Exercise 5.1 Without attempting geometrical calculations, suggest values for the par- ameters p and U for each of the normal probability densities that are shown in Figure 5.4. An early history Although the terminology was not standardized until after 1900, the normal distribution itself was certainly known before then (under a variety of dif- ferent names). The following is a brief account of the history of the normal distribution before the twentieth century. Credit for the very first appearance of the normal p.d.f. goes to Abraham de Moivre (1667-1754), a Protestant Frenchman who emigrated to London in 1688 to avoid religious persecution and lived there for the rest of his life, becoming an eminent mathematician. Prompted by a desire to compute the probabilities of winning in various games of chance, de Moivre obtained what is now recognized as the normal p.d.f., an approximation to a binomial prob- ability function (these were early days in the history of the binomial distri- bution). The pamphlet that contains this work was published in 1733.