Central Limit Theorem: the Cornerstone of Modern Statistics

Home , Central limit theorem

Statistical Round pISSN 2005-6419 • eISSN 2005-7563 KJA Central limit theorem: the Korean Journal of Anesthesiology cornerstone of modern statistics

Sang Gyu Kwak1 and Jong Hae Kim2 Departments of 1Medical Statistics, 2Anesthesiology and Pain Medicine, School of Medicine, Catholic University of Daegu, Daegu, Korea

According to the central limit theorem, the means of a random sample of size, n, from a population with mean, μ, and 2 2 μ σ variance, σ , distribute normally with mean, , and variance, n . Using the central limit theorem, a variety of parametric tests have been developed under assumptions about the parameters that determine the population probability distribution. Compared to non-parametric tests, which do not require any assumptions about the population probability distribution, parametric tests produce more accurate and precise estimates with higher statistical powers. However, many medical researchers use parametric tests to present their data without knowledge of the contribution of the central limit theorem to the development of such tests. Thus, this review presents the basic concepts of the central limit theorem and its role in binomial distributions and the Student’s t-test, and provides an example of the sampling distributions of small populations. A proof of the central limit theorem is also described with the mathematical concepts required for its near- complete understanding.

Key Words: Normal distribution, Probability, Statistical distributions, Statistics.

Introduction multiple parametric tests are used to assess the statistical validity of clinical studies performed by medical researchers; however, The central limit theorem is the most fundamental theory in most researchers are unaware of the value of the central limit modern statistics. Without this theorem, parametric tests based theorem, despite their routine use of parametric tests. Thus, on the assumption that sample data come from a population clinical researchers would benefit from knowing what the cen- with fixed parameters determining its probability distribution tral limit theorem is and how it has become the basis for para- would not exist. With the central limit theorem, parametric tests metric tests. This review aims to address these topics. The proof have higher statistical power than non-parametric tests, which of the central limit theorem is described in the appendix, with do not require probability distribution assumptions. Currently, the necessary mathematical concepts (e.g., moment-generating function and Taylor’s formula) required for understanding the proof. However, some mathematical techniques (e.g., differential Corresponding author: Jong Hae Kim, M.D. and integral calculus) were omitted due to space limitations. Department of Anesthesiology and Pain Medicine, School of Medicine, Catholic University of Daegu, 33, Duryugongwon-ro 17-gil, Nam-gu, Basic Concepts of Central Limit Theorem Daegu 42472, Korea Tel: 82-53-650-4979, Fax: 82-53-650-4517 In statistics, a population is the set of all items, people, or Email: [email protected] ORCID: http://orcid.org/0000-0003-1222-0054 events of interest. In reality, however, collecting all such ele- ments of the population requires considerable effort and is often Received: September 30, 2016. not possible. For example, it is not possible to investigate the Revised: November 10, 2016 (1st); November 17, 2016 (2nd). Accepted: January 2, 2017. proficiency of every anesthesiologist, worldwide, in performing awake nasotracheal intubations. To make inferences regarding Korean J Anesthesiol 2017 April 70(2): 144-156 the population, however, a subset of the population (sample) https://doi.org/10.4097/kjae.2017.70.2.144

CC This is an open-access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/ licenses/by-nc/4.0/), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. Copyright ⓒ the Korean Society of Anesthesiologists, 2017 Online access in http://ekja.org KOREAN J ANESTHESIOL Kwak and Kim

2 can be used. A sample of sufficient size that is randomly selected σ 112.5 population mean) and 37.5 = n = 3 , respectively; however, can be used to estimate the parameters of the population using the distribution of the means of samples is skewed (Fig. 1A). inferential statistics. A finite number of samples are attainable When a simple random sampling with replacement is per- from the population depending on the size of the sample and formed for samples with a size of 6, 4 × 4 × 4 × 4 × 4 × 4 = 46 = the population itself. For example, all samples with a size of 1 4,096 samples are possible (Table 2). The mean and variance of obtained at random, with replacement, from the population the 4,096 sample means are 12 (the population mean) and 18.75 2 {3,6,9,30} would be {3},{6},{9}, and {30}. If the sample size is 2, a = σ = 112.5, respectively. Compared to the distribution of the 2 n 6 total of 4 × 4 = 4 = 16 samples, which are {3,3},{3,6},{3,9},{3,30}, means of samples with a size of 3, that of the means of samples {6,3},{6,6}…{9,30},{30,3},{30,6},{30,9}, and {30,30}, would be with a size of 6 is less skewed. Importantly, the sample means n possible. In this way, 4 samples with a size of n would be ob- also gather around the population mean. (Fig. 1B). Thus, the tained from the population. Here, we consider the distribution larger the sample size (n), the more closely the sample means of the sample means. gather symmetrically around the population mean (μ) and have For example, here is the asymmetric population with a size of σ2 a corresponding reduction in the variance ( n ) (Figs. 1C and 4 as presented above: 1D). If Figs. 1A to 1D are converted to the probability density function by replacing the variable “frequency” with another variable “probability” on the vertical axis, their shapes remain The population mean, μ, {and3, 6 ,variance,9, 30} σ2, are: unchanged. In general, as the sample size from the population increases, its mean gathers more closely around the population mean with 3 + 6 + 9 + 30 a decrease in variance. Thus, as the sample size approaches in- 𝜇𝜇 = = 12 4 finity, the sample means approximate the normal distribution σ2 2 2 2 2 with a mean, μ, and a variance, n . As shown above, the skewed 2 (3 − 12) + (6 − 12) + (9 − 12) + (30 − 12) 𝜎𝜎 = = 112.5 distribution of the population does not affect the distribution 4 A simple random sampling with replacement from the popu- of the sample means as the sample size increases. Therefore, 3 lation produces 4 × 4 × 4 = 4 = 64 samples with a size of 3 (Table the central limit theorem indicates that if the sample size is suf- 1). The mean and variance of the 64 sample means are 12 (the ficiently large, the means of samples obtained using a random sampling with replacement are distributed normally with the

Table 1. Samples with a Size of 3 and Their Means Table 2. Samples with a Size of 6 and Their Means Number Sample Sample mean Number Sample Sample mean 1 3 3 3 3 2 3 3 6 4 1 3 3 3 3 3 3 3 3 3 3 9 5 2 3 3 3 3 3 6 3.5 4 3 3 30 12 3 3 3 3 3 3 9 4 5 3 6 3 4 4 3 3 3 3 3 30 7.5 6 3 6 6 5 5 3 3 3 3 6 3 3.5 7 3 6 9 6 6 3 3 3 3 6 6 4 8 3 6 30 13 7 3 3 3 3 6 9 4.5 9 3 9 3 5 8 3 3 3 3 6 30 8 10 3 9 6 6 9 3 3 3 3 9 3 4 Truncated 10 3 3 3 3 9 6 4.5 54 30 6 6 14 Truncated 55 30 6 9 15 4087 30 30 30 30 6 9 22.5 56 30 6 30 22 4088 30 30 30 30 6 30 26 57 30 9 3 14 4089 30 30 30 30 9 3 22 58 30 9 6 15 4090 30 30 30 30 9 6 22.5 59 30 9 9 16 4091 30 30 30 30 9 9 23 60 30 9 30 23 4092 30 30 30 30 9 30 26.5 61 30 30 3 21 4093 30 30 30 30 30 3 25.5 62 30 30 6 22 4094 30 30 30 30 30 6 26 63 30 30 9 23 4095 30 30 30 30 30 9 26.5 64 30 30 30 30 4096 30 30 30 30 30 30 30

Online access in http://ekja.org 145 Central limit theorem VOL. 70, NO. 2, April 2017

A B

20 800

15 600

10 400

Frequency Frequency

5 200

0 0 0510 15 20 25 30 0510 15 20 25 30 Mean of samples Mean of samples C D

60,000

1,500,000 40,000

Frequency 20,000 Frequency 500,000

0 0 0510 15 20 25 30 0510 15 20 25 30 Mean of samples Mean of samples

Fig. 1. Histogram representing the means for samples of sizes of 3 (A), 6 (B), 9 (C), and 12 (D).

σ2 1 mean, μ, and the variance, n , regardless of the population dis- rolling the number 3 in each trial, 1 − 6 : the probability of roll- tribution. Refer to the appendix for a near-complete proof of the ing a number other than 3 in each trial. central limit theorem, as well as the basic mathematical concepts required for its proof. The mathematical expectation of the random variable, X, (i.e., the number of times that the number 3 is rolled in each trial), Central Limit Theorem in the Real World which is also referred to as the mean of the distribution, is:

An unbiased, symmetric, 6-sided dice is rolled at random n 1 times. The probability of rolling the number 3 x times in n suc- And the variance is: 𝐸𝐸(𝑋𝑋) = 𝑛𝑛 × 6 cessive independent trials has the following probability density distribution, which is called the binomial distribution: 1 1 𝑉𝑉𝑉𝑉𝑉𝑉(𝑋𝑋) = 𝑛𝑛 × × (1 − ) When n = 10, the probability has6 a skewed6 distribution (Fig. 𝑥𝑥 𝑛𝑛−𝑥𝑥 𝑛𝑛 1 1 2A); however, as n increases, the distribution becomes sym- 𝑓𝑓(𝑥𝑥) = ( ) ( ) (1 − ) n: number of independent𝑥𝑥 trials6 (rolling6 a dice), x: number of metric with respect to its mean (Figs. 2B–2D). As n approaches 1 times the number 3 is rolled in each trial, 6 : the probability of infinity, the binomial distribution approximates the normal dis-

146 Online access in http://ekja.org KOREAN J ANESTHESIOL Kwak and Kim

A 0.35 0.30 0.25 0.20

Probability 0.15 0.10 0.05 0 123456789 10 Number of cases to obtain number3upon 10 successive trials of rolling an unbiased 6-sided dice B 0.25

0.20

0.15

Probability 0.10

0.05

0 123456789 10 11 12 13 14 15 16 17 18 19 20 Number of cases to obtain number3upon 20 successive trials of rolling an unbiased 6-sided dice

C 0.25

0.20

0.15

0.10

Probability 0.05

0 123456789 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Number of cases to obtain number3upon 30 successive trials of rolling an unbiased 6-sided dice

D 0.12

0.10

0.08

0.06

Probability 0.04

0.02 Fig. 2. The probability density function of 0 a binomial distribution with a probability 159131721252933 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 1 parameter of 6 (i.e., the probability of Number of cases to obtain number3upon 100 successive trials of rolling the number 3 in each trial), based rolling an unbiased 6-sided dice on to the number of trials.

Online access in http://ekja.org 147 Central limit theorem VOL. 70, NO. 2, April 2017 tribution with a mean, np, and a variance, np(1 − p), where p is As the degree of freedom increases, the Student’s t-distribution the probability constant for the occurrence of the specific event approaches the normal distribution. At a degree of freedom of during each trial. 30, the Student’s t-distribution is regarded as equaling the normal distribution [1]. The underlying assumption for the Stu- Central Limit Theorem in the Student’s t-test dent’s t-test is that samples should be obtained from a normally distributed population. However, since the distribution of popu- Since the central limit theorem determines the sampling dis- lation is not known, it should be determined whether the sample − tribution of the means with a sufficient size, a specific mean (X) is normally distributed. This is particularly true for small sample sizes. If small sample sizes are normally distributed, the studen- can be standardized and subsequently identified (𝑋𝑋̅ − 𝜇𝜇) tized distribution of the sample means is equal to the Student’s t- (𝑍𝑍 = 𝜎𝜎 ) against the normal distribution√𝑛𝑛 with mean of 0 and variance of distribution with a degree of freedom corresponding to the sam-

12. In reality, however, the lack of a known population variance ple size. If the small samples are not normally distributed, non- (σ2) prevents a determination of the probability density distribu- parametric tests should be performed instead of the Student’s t- tion. test since they do not require assumptions about the distribution of population. If the sample size is 30, the studentized sampling distribution approximates the standard normal distribution and 2 2 2 2 (𝑋𝑋1 − 𝜇𝜇) + (𝑋𝑋2 − 𝜇𝜇) + ⋯ + (𝑋𝑋𝑁𝑁 − 𝜇𝜇) assumptions about the population distribution are meaningless 𝜎𝜎 = X (i = 1, 2, …, n): a sample from𝑁𝑁 the population, N: the size since the sampling distribution is considered normal, according i of the population, μ: the mean of the population. to the central limit theorem. Therefore, even if the mean of a sample of size > 30 is studentized using the variance, a normal Notably, the Student’s t-distribution was developed to use a distribution can be used for the probability distribution. sample variance (S) instead of a population variance (σ2). Conclusions

2 2 2 2 (𝑥𝑥1 − 𝑋𝑋̅) + (𝑥𝑥2 − 𝑋𝑋̅) + ⋯ + (𝑥𝑥𝑛𝑛 − 𝑋𝑋̅) A comprehensive understanding of the central limit theorem 𝑆𝑆 = xi (i = 1, 2, …, n): a random 𝑛𝑛sample− 1 from the population, will assist medical investigators when performing parametric − n: sample size, XX : the mean of the samples. tests to analyze their data with high statistical powers. The use − of this theorem will also aid in the design of study protocols that The specific mean (X) is studentized and its loca- (𝑋𝑋̅ − 𝜇𝜇) are based on best-fit statistics. (𝑡𝑡 = ) 𝑆𝑆 tion is evaluated on the Student’s t-distribution,√ 𝑛𝑛based on the degree of freedom (n − 1). The shape of the Student’s t-distribu- ORCID tion is dependent on the degree of freedom. A low degree of freedom renders the peak of the Student’s t-distribution lower Sang Gyu Kwak, http://orcid.org/0000-0003-0398-5514 than that of a normal distribution, although at some points, the Jong Hae Kim, http://orcid.org/0000-0003-1222-0054 tails have higher values than those of the normal distribution.

Reference

1. Kim TK. T test as a parametric statistic. Korean J Anesthesiol 2015; 68: 540-6.

148 Online access in http://ekja.org KOREAN J ANESTHESIOL Kwak and Kim

Appendix

Moment-generating function

Since the central limit theorem is proven using a moment-generating function for a random variable, regardless of its distribution, it is necessary to understand the moment-generating function. Prior to that, however, it is necessary to understand the probability density function and mathematical expectation. When a fair coin is tossed at random on two independent and successive trials, the following probability distribution describes the expected number of head(s) (Table 1). Here, we derive a function, f(x), detailing the probability according to the number of the head(s).

𝑥𝑥 2−𝑥𝑥 2 1 1 𝑓𝑓(𝑥𝑥) = ( ) ( ) (1 − ) 2: number of independent trials𝑥𝑥 (coin2 tosses),2 x: number of head(s), 1 1 2 : the probability of having a head on each trial, 1 − 2 : the probability of having a tail on each trial.

The f(x) is called the probability density function. Based on Table 1, the mathematical expectation (or expected value) of the random variable, X, E(X) is:

1 1 1 𝐸𝐸(𝑋𝑋) = 0 × + 1 × + 2 × = 1 4 2 4 Table 1. Probability Distribution x 0 1 2 1 1 1 P (X = x) 4 2 4 x : number of head(s), P (X = x): probability to have x head(s).

Generally, it is expressed as:

𝑛𝑛

𝐸𝐸(𝑋𝑋) = ∑ 𝑥𝑥𝑥𝑥(𝑥𝑥) for a discrete variable, 𝑥𝑥=1

∞

𝐸𝐸(𝑋𝑋) = ∫−∞𝑥𝑥𝑥𝑥(𝑥𝑥)𝑑𝑑𝑑𝑑 for a continuous variable (1) Therefore, E(X) is equivalent to the mean, μ, of a random variable. E(X) = μ. The moment-generating function for a random continuous variable, X, is defined as the mathematical expectation of etX, E(etX), and denoted as MX(t). Thus,

∞ 𝑡𝑡𝑡𝑡 𝑡𝑡𝑡𝑡 𝑀𝑀𝑋𝑋(𝑡𝑡) = 𝐸𝐸(𝑒𝑒 ) = ∫ 𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 , −ℎ < 𝑡𝑡 < ℎ (ℎ > 0) Since the moment-generating function is a unique determinant−∞ for the distribution of a random sample, it is always true that two variables with a common moment-generating function have the same distribution. To get its first order derivative, tx is replaced by u. Then, du is obtained by: dt

𝑑𝑑𝑑𝑑 𝑢𝑢 = 𝑡𝑡𝑡𝑡, = 𝑥𝑥 𝑑𝑑𝑑𝑑

∞ 𝑡𝑡𝑡𝑡 ∞ 𝑢𝑢 ∞ 𝑢𝑢 ∞ 𝑢𝑢 ′ 𝑑𝑑𝑑𝑑(𝑡𝑡) ∫−∞ 𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 ∫−∞ 𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 ∫−∞ 𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 𝑑𝑑𝑑𝑑 ∫−∞ 𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 𝑑𝑑𝑑𝑑 𝑀𝑀𝑋𝑋 (𝑡𝑡) = = = = × = × 𝑑𝑑𝑑𝑑 𝑑𝑑𝑑𝑑 𝑑𝑑𝑑𝑑 𝑑𝑑𝑑𝑑 𝑑𝑑𝑑𝑑 𝑑𝑑𝑑𝑑 𝑑𝑑𝑑𝑑

Online access in http://ekja.org 149 Central limit theorem VOL. 70, NO. 2, April 2017

u Since e = eu, du = x, and u = tx, du dt

∞ ∞ ∞ ′ 𝑢𝑢 𝑢𝑢 𝑡𝑡𝑡𝑡 𝑋𝑋 𝑀𝑀 (𝑡𝑡) = ∫−∞𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 × 𝑥𝑥 = ∫−∞𝑥𝑥𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 = ∫−∞𝑥𝑥𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 (2) When t = 0,

∞ ∞ ′ 0 𝑋𝑋 𝑀𝑀 (0) = ∫−∞𝑥𝑥𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 = ∫−∞𝑥𝑥𝑥𝑥(𝑥𝑥)𝑑𝑑𝑑𝑑 = 𝐸𝐸(𝑋𝑋) = 𝜇𝜇 (∵ please, refer to equation (1). ) (3)

The second order derivative of MX (t) is,

∞ 𝑡𝑡𝑡𝑡 ∞ 𝑡𝑡𝑡𝑡 ′′ 𝑀𝑀′(𝑡𝑡) ∫−∞ 𝑥𝑥𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 ∫−∞ 𝑥𝑥 × 𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 𝑀𝑀𝑋𝑋 (𝑡𝑡) = = = 𝑑𝑑𝑑𝑑 𝑑𝑑𝑑𝑑 𝑑𝑑𝑑𝑑 According to the equation (2),

∞ 𝑡𝑡𝑡𝑡 ∞ ∫−∞ 𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 𝑡𝑡𝑡𝑡 = ∫ 𝑥𝑥𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 𝑑𝑑𝑑𝑑 −∞ Therefore,

∞ 𝑡𝑡𝑡𝑡 ∞ ∞ ′′ ∫−∞ 𝑥𝑥 × 𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 𝑡𝑡𝑡𝑡 2 𝑡𝑡𝑡𝑡 𝑀𝑀𝑋𝑋 (𝑡𝑡) = = ∫ 𝑥𝑥 × 𝑥𝑥𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 = ∫ 𝑥𝑥 𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 𝑑𝑑𝑑𝑑 −∞ −∞

∞ ∞ ′′ 2 0 2 2 𝑋𝑋 ∴ 𝑀𝑀 (0) = ∫−∞𝑥𝑥 𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 = ∫−∞𝑥𝑥 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 = 𝐸𝐸(𝑋𝑋 ) (4)

Moment-generating function of a normal distribution

The derivation of the probability density function for a normal distribution is beyond the scope of this review. However, the probability density function for a normal distribution with mean, μ, and variance, σ2, is:

2 1 (𝑥𝑥 − 𝜇𝜇) 𝑓𝑓(𝑥𝑥) = exp [− 2 ] , −∞ < 𝑥𝑥 < ∞ 𝜎𝜎√2𝜋𝜋 2𝜎𝜎 The sum of the probabilities under the normal distribution should be 1.

∞ 2 1 (𝑥𝑥 − 𝜇𝜇) ∫ exp [− 2 ] 𝑑𝑑𝑑𝑑 = 1 −∞ 𝜎𝜎√2𝜋𝜋 2𝜎𝜎

150 Online access in http://ekja.org KOREAN J ANESTHESIOL Kwak and Kim

As a reference, exp(x) = ex. According to the definition of the moment-generating function, , , 𝑡𝑡𝑡𝑡 ∞ 𝑡𝑡𝑡𝑡 2 , the moment-generating function for a normal distribution with mean, μ, and variance,𝑀𝑀𝑋𝑋(𝑡𝑡) = σ 𝐸𝐸, is:(𝑒𝑒 ) = ∫−∞ 𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 , −ℎ < 𝑡𝑡 < ℎ (ℎ > 0) 𝑡𝑡𝑡𝑡 ∞ 𝑡𝑡𝑡𝑡 𝑀𝑀𝑋𝑋(𝑡𝑡) = 𝐸𝐸(𝑒𝑒 ) = ∫−∞ 𝑒𝑒 𝑓𝑓(𝑥𝑥)𝑑𝑑𝑑𝑑 , −ℎ < 𝑡𝑡 < ℎ (ℎ > 0)

∞ 2 ∞ 2 𝑡𝑡𝑡𝑡 1 (𝑥𝑥 − 𝜇𝜇) 1 (𝑥𝑥 − 𝜇𝜇) 𝑀𝑀𝑋𝑋(𝑡𝑡) = ∫ 𝑒𝑒 exp [− 2 ] 𝑑𝑑𝑑𝑑 = ∫ exp(𝑡𝑡𝑡𝑡) exp [− 2 ] 𝑑𝑑𝑑𝑑 −∞ 𝜎𝜎√2𝜋𝜋 2𝜎𝜎 −∞ 𝜎𝜎√2𝜋𝜋 2𝜎𝜎 ∞ 2 ∞ 2 1 (𝑥𝑥 − 𝜇𝜇) 1 (𝑥𝑥 − 𝜇𝜇) = ∫ exp(𝑡𝑡𝑡𝑡) exp [− 2 ] 𝑑𝑑𝑑𝑑 = ∫ exp [𝑡𝑡𝑡𝑡 − 2 ] 𝑑𝑑𝑑𝑑 −∞ 𝜎𝜎√2𝜋𝜋 2𝜎𝜎 −∞ 𝜎𝜎√2𝜋𝜋 2𝜎𝜎 ∞ 2 1 2𝜎𝜎 𝑡𝑡𝑡𝑡 = ∫ exp [ 2 −∞ 𝜎𝜎√2𝜋𝜋 2𝜎𝜎 2 2 2 2 2 2 2 (𝑥𝑥 − 𝜇𝜇) + (𝜎𝜎 𝑡𝑡) − (𝜎𝜎 𝑡𝑡) − 2(𝑥𝑥 − 𝜇𝜇)𝜎𝜎 𝑡𝑡 + 2(𝑥𝑥 − 𝜇𝜇)𝜎𝜎 𝑡𝑡 − 2 ] 𝑑𝑑𝑑𝑑 2𝜎𝜎 ∞ 2 2 2 2 2 1 2𝜎𝜎 𝑡𝑡𝑡𝑡 (𝑥𝑥 − 𝜇𝜇) − 2(𝑥𝑥 − 𝜇𝜇)𝜎𝜎 𝑡𝑡 + (𝜎𝜎 𝑡𝑡) = ∫ exp [ 2 − 2 −∞ 𝜎𝜎√2𝜋𝜋 2𝜎𝜎 2𝜎𝜎 2 2 2 −(𝜎𝜎 𝑡𝑡) + 2(𝑥𝑥 − 𝜇𝜇)𝜎𝜎 𝑡𝑡 − 2 ] 𝑑𝑑𝑑𝑑 2𝜎𝜎 ∞ 2 2 2 2 1 −2𝜎𝜎 𝑡𝑡𝑡𝑡 − (𝜎𝜎 𝑡𝑡) + 2(𝑥𝑥 − 𝜇𝜇)𝜎𝜎 𝑡𝑡 = ∫ exp [− 2 −∞ 𝜎𝜎√2𝜋𝜋 2𝜎𝜎 2 2 2 2 (𝑥𝑥 − 𝜇𝜇) − 2(𝑥𝑥 − 𝜇𝜇)𝜎𝜎 𝑡𝑡 + (𝜎𝜎 𝑡𝑡) − 2 ] 𝑑𝑑𝑑𝑑 2𝜎𝜎 ∞ 2 2 2 2 2 1 −2𝜎𝜎 𝑡𝑡𝑡𝑡 − (𝜎𝜎 𝑡𝑡) + 2𝜎𝜎 𝑡𝑡𝑡𝑡 − 2𝜇𝜇𝜎𝜎 𝑡𝑡 = ∫ exp [− 2 −∞ 𝜎𝜎√2𝜋𝜋 2𝜎𝜎 2 2 2 2 (𝑥𝑥 − 𝜇𝜇) − 2(𝑥𝑥 − 𝜇𝜇)𝜎𝜎 𝑡𝑡 + (𝜎𝜎 𝑡𝑡) − 2 ] 𝑑𝑑𝑑𝑑 2𝜎𝜎 ∞ 2 2 2 2 2 1 −(𝜎𝜎 𝑡𝑡) − 2𝜇𝜇𝜎𝜎 𝑡𝑡 (𝑥𝑥 − 𝜇𝜇 − 𝜎𝜎 𝑡𝑡) = ∫ exp [− 2 − 2 ] 𝑑𝑑𝑑𝑑 −∞ 𝜎𝜎√2𝜋𝜋 2𝜎𝜎 2𝜎𝜎 2 2 2 ∞ 2 2 −(𝜎𝜎 𝑡𝑡) − 2𝜇𝜇𝜎𝜎 𝑡𝑡 1 (𝑥𝑥 − 𝜇𝜇 − 𝜎𝜎 𝑡𝑡) = exp [− 2 ] ∫ exp [− 2 ] 𝑑𝑑𝑑𝑑 2𝜎𝜎 −∞ 𝜎𝜎√2𝜋𝜋 2𝜎𝜎 2 2 2 ∞ 2 2 2𝜇𝜇𝜎𝜎 𝑡𝑡 + (𝜎𝜎 𝑡𝑡) 1 [𝑥𝑥 − (𝜇𝜇 − 𝜎𝜎 𝑡𝑡)] = exp [ 2 ] ∫ exp [− 2 ] 𝑑𝑑𝑑𝑑 2𝜎𝜎 −∞ 𝜎𝜎√2𝜋𝜋 2𝜎𝜎

2 The last integral, 2 2 , is the probability density function for the normal distribution with mean, μ − σ t, ∞ 1 [𝑥𝑥−(𝜇𝜇−𝜎𝜎 𝑡𝑡)] 2 2 and variance, σ . Since∫− ∞the𝜎𝜎√ sum2𝜋𝜋 exp of[ the− probabilities2𝜎𝜎 ] 𝑑𝑑𝑑𝑑 under the normal distribution should be 1,

∞ 2 2 1 [𝑥𝑥 − (𝜇𝜇 − 𝜎𝜎 𝑡𝑡)] ∫ exp [− 2 ] 𝑑𝑑𝑑𝑑 = 1 −∞ 𝜎𝜎√2𝜋𝜋 2𝜎𝜎

Online access in http://ekja.org 151 Central limit theorem VOL. 70, NO. 2, April 2017

Therefore, the moment-generating function for the normal distribution with mean, μ, and variance, σ2, is:

2 2 2 2 2 2 2 2 2𝜇𝜇𝜎𝜎 𝑡𝑡 + (𝜎𝜎 𝑡𝑡) 2𝜇𝜇𝜎𝜎 𝑡𝑡 (𝜎𝜎 𝑡𝑡) 𝜎𝜎 𝑡𝑡 𝑀𝑀𝑋𝑋(𝑡𝑡) = exp [ 2 ] = exp [ 2 + 2 ] = exp [𝜇𝜇𝜇𝜇 + ] (5) 2𝜎𝜎 2𝜎𝜎 2𝜎𝜎 2

Mathematical expectation of multiplication between stochastically independent random variables

Let the discrete random variables, X and Y, be stochastically independent.

𝑋𝑋 = {𝑋𝑋1, 𝑋𝑋2, ⋯ 𝑋𝑋𝑛𝑛}, 𝑌𝑌 = {𝑌𝑌1, 𝑌𝑌2, ⋯ 𝑌𝑌𝑚𝑚}

𝑋𝑋1𝑌𝑌1 + 𝑋𝑋1𝑌𝑌2 + ⋯ + 𝑋𝑋1𝑌𝑌𝑚𝑚 + 𝑋𝑋2𝑌𝑌1 + 𝑋𝑋2𝑌𝑌2 + ⋯ + 𝑋𝑋2𝑌𝑌𝑚𝑚 + ⋯ + 𝑋𝑋𝑛𝑛𝑌𝑌1 + 𝑋𝑋𝑛𝑛𝑌𝑌2 + ⋯ 𝑋𝑋𝑛𝑛𝑌𝑌𝑚𝑚 𝐸𝐸(𝑋𝑋𝑋𝑋) = 𝑛𝑛𝑛𝑛

𝑋𝑋1(𝑌𝑌1 + 𝑌𝑌2 + ⋯ + 𝑌𝑌𝑚𝑚) + 𝑋𝑋2(𝑌𝑌1 + 𝑌𝑌2 + ⋯ + 𝑌𝑌𝑚𝑚) + ⋯ + 𝑋𝑋𝑛𝑛(𝑌𝑌1 + 𝑌𝑌2 + ⋯ + 𝑌𝑌𝑚𝑚) = 𝑛𝑛𝑛𝑛

(𝑋𝑋1 + 𝑋𝑋2 + ⋯ + 𝑋𝑋𝑛𝑛)(1 + 𝑌𝑌1 + 𝑌𝑌2 + ⋯ + 𝑌𝑌𝑚𝑚) = = 𝐸𝐸(𝑋𝑋)𝐸𝐸(𝑌𝑌) (6) 𝑛𝑛𝑛𝑛 The above equation also applies to stochastically independent continuous random variables.

Taylor’s formula

Assume that the function, f, is continuous on the closed interval, [a, x], and differentiable on the open interval, (a, x).

𝑥𝑥 ′ 𝑓𝑓(𝑥𝑥) = 𝑓𝑓(𝑎𝑎) + [𝑓𝑓(𝑥𝑥) − 𝑓𝑓(𝑎𝑎)] = 𝑓𝑓(𝑎𝑎) + ∫𝑎𝑎 𝑓𝑓 (𝑡𝑡)𝑑𝑑𝑑𝑑 Let t be replaced by a + u(x − a).

𝑑𝑑𝑡𝑡 𝑡𝑡 = 𝑎𝑎 + 𝑢𝑢(𝑥𝑥 − 𝑎𝑎) ⟺ = 𝑥𝑥 − 𝑎𝑎 𝑑𝑑𝑑𝑑

∴ 𝑑𝑑𝑑𝑑 = 𝑑𝑑𝑑𝑑(𝑥𝑥 − 𝑎𝑎)

𝑎𝑎 − 𝑎𝑎 𝑥𝑥 − 𝑎𝑎 𝑎𝑎 < 𝑡𝑡 < 𝑥𝑥 ⟺ 𝑎𝑎 < 𝑎𝑎 + 𝑢𝑢(𝑥𝑥 − 𝑎𝑎) < 𝑥𝑥 ⟺ 𝑎𝑎 − 𝑎𝑎 < 𝑢𝑢(𝑥𝑥 − 𝑎𝑎) < 𝑥𝑥 − 𝑎𝑎 ⟺ < 𝑢𝑢 < 𝑥𝑥 − 𝑎𝑎 𝑥𝑥 − 𝑎𝑎

∴ 0 < 𝑢𝑢 < 1

Then,

𝑥𝑥 1 ′ ′ 𝑓𝑓(𝑥𝑥) = 𝑓𝑓(𝑎𝑎) + ∫𝑎𝑎 𝑓𝑓 (𝑡𝑡)𝑑𝑑𝑑𝑑 = 𝑓𝑓(𝑎𝑎) + ∫0 𝑓𝑓 (𝑎𝑎 + 𝑢𝑢(𝑥𝑥 − 𝑎𝑎))𝑑𝑑𝑑𝑑(𝑥𝑥 − 𝑎𝑎)

152 Online access in http://ekja.org KOREAN J ANESTHESIOL Kwak and Kim

As is known,

𝑏𝑏 𝑏𝑏 ′ 𝑏𝑏 𝑎𝑎 ∫𝑎𝑎 𝑓𝑓 (𝑥𝑥)𝑔𝑔(𝑥𝑥)𝑑𝑑𝑑𝑑 = [𝑓𝑓(𝑥𝑥)𝑔𝑔(𝑥𝑥)] − ∫𝑎𝑎 𝑓𝑓(𝑥𝑥)𝑔𝑔′(𝑥𝑥)𝑑𝑑𝑑𝑑

Using the above formula,

1 1 ′ ′ ∫0 𝑓𝑓 (𝑎𝑎 + 𝑢𝑢(𝑥𝑥 − 𝑎𝑎))𝑑𝑑𝑑𝑑 = ∫0 (−(1 − 𝑢𝑢))′𝑓𝑓 (𝑎𝑎 + 𝑢𝑢(𝑥𝑥 − 𝑎𝑎))𝑑𝑑𝑑𝑑

1 1 𝑑𝑑 ′ = [−(1 − 𝑢𝑢)𝑓𝑓′(𝑎𝑎 + 𝑢𝑢(𝑥𝑥 − 𝑎𝑎))]0 − ∫ −(1 − 𝑢𝑢) 𝑓𝑓 (𝑎𝑎 + 𝑢𝑢(𝑥𝑥 − 𝑎𝑎))𝑑𝑑𝑑𝑑 0 𝑑𝑑𝑑𝑑

= −(1 − 1)𝑓𝑓′(𝑎𝑎 + 1 × (𝑥𝑥 − 𝑎𝑎)) − [−(1 − 0)𝑓𝑓′(𝑎𝑎 + 0 × (𝑥𝑥 − 𝑎𝑎))] 1 ′′ + ∫0 (1 − 𝑢𝑢)𝑓𝑓 (𝑎𝑎 + 𝑢𝑢(𝑥𝑥 − 𝑎𝑎))(𝑎𝑎 + 𝑢𝑢(𝑥𝑥 − 𝑎𝑎))𝑑𝑑𝑑𝑑

1 ′′ = 𝑓𝑓′(𝑎𝑎) + ∫0 (1 − 𝑢𝑢)𝑓𝑓 (𝑎𝑎 + 𝑢𝑢(𝑥𝑥 − 𝑎𝑎))(𝑥𝑥 − 𝑎𝑎)𝑑𝑑𝑑𝑑 Therefore,

1 ′ 𝑓𝑓(𝑥𝑥) = 𝑓𝑓(𝑎𝑎) + ∫0 𝑓𝑓 (𝑎𝑎 + 𝑢𝑢(𝑥𝑥 − 𝑎𝑎))𝑑𝑑𝑑𝑑(𝑥𝑥 − 𝑎𝑎)

1 ′ ′′ = 𝑓𝑓(𝑎𝑎) + [𝑓𝑓 (𝑎𝑎) + ∫0 (1 − 𝑢𝑢)𝑓𝑓 (𝑎𝑎 + 𝑢𝑢(𝑥𝑥 − 𝑎𝑎))(𝑥𝑥 − 𝑎𝑎)𝑑𝑑𝑑𝑑] (𝑥𝑥 − 𝑎𝑎)

1 (0) (1) (2) 2 = 𝑓𝑓 (𝑎𝑎) + 𝑓𝑓 (𝑎𝑎)(𝑥𝑥 − 𝑎𝑎) + ∫0 (1 − 𝑢𝑢)𝑓𝑓 (𝑎𝑎 + 𝑢𝑢(𝑥𝑥 − 𝑎𝑎))(𝑥𝑥 − 𝑎𝑎) 𝑑𝑑𝑑𝑑 When the above process is generalized,

(1) (2) 2 (𝑛𝑛) 𝑛𝑛 (0) 𝑓𝑓 (𝑎𝑎)(𝑥𝑥 − 𝑎𝑎) 𝑓𝑓 (𝑎𝑎)(𝑥𝑥 − 𝑎𝑎) 𝑓𝑓 (𝑎𝑎)(𝑥𝑥 − 𝑎𝑎) 𝑓𝑓(𝑥𝑥) = 𝑓𝑓 (𝑎𝑎) + + + ⋯ + 1! 2! 𝑛𝑛! 1 𝑛𝑛 (1 − 𝑢𝑢) (𝑛𝑛+1) 𝑛𝑛+1 + ∫ 𝑓𝑓 (𝑎𝑎 + 𝑢𝑢(𝑥𝑥 − 𝑎𝑎))(𝑥𝑥 − 𝑎𝑎) 𝑑𝑑𝑑𝑑 0 𝑛𝑛!

The last term, 𝑛𝑛 , is called the remainder term and can be expressed as (𝑛𝑛+1) 𝑛𝑛+1 , . 1 (1−𝑢𝑢) 𝑓𝑓 (𝜉𝜉)(𝑥𝑥−𝑎𝑎) . (𝑛𝑛+1) 𝑛𝑛+1 (𝑛𝑛+1) 𝑛𝑛+1 . Finally,∫0 the𝑛𝑛 !Taylor𝑓𝑓 formula(𝑎𝑎 + 𝑢𝑢 is:(𝑥𝑥 − 𝑎𝑎))(𝑥𝑥 − 𝑎𝑎) 𝑑𝑑𝑑𝑑 𝑛𝑛+1! 𝑓𝑓 (𝜉𝜉)(𝑥𝑥−𝑎𝑎) , 𝑎𝑎 < 𝜉𝜉 < 𝑥𝑥

𝑛𝑛+1! , 𝑎𝑎 < 𝜉𝜉 < 𝑥𝑥

(1) (2) 2 (𝑛𝑛) 𝑛𝑛 (0) 𝑓𝑓 (𝑎𝑎)(𝑥𝑥 − 𝑎𝑎) 𝑓𝑓 (𝑎𝑎)(𝑥𝑥 − 𝑎𝑎) 𝑓𝑓 (𝑎𝑎)(𝑥𝑥 − 𝑎𝑎) 𝑓𝑓(𝑥𝑥) = 𝑓𝑓 (𝑎𝑎) + + + ⋯ + 1! 2! 𝑛𝑛! (𝑛𝑛+1) 𝑛𝑛+1 𝑓𝑓 (𝜉𝜉)(𝑥𝑥 − 𝑎𝑎) + (7) 𝑛𝑛 + 1!

Online access in http://ekja.org 153 Central limit theorem VOL. 70, NO. 2, April 2017

Central limit theorem

2 − Let X1, X2, …, Xn be a random sample from a distribution with mean, μ, and variance, σ . Here, X is defined as:

𝑋𝑋1 + 𝑋𝑋2 + ⋯ + 𝑋𝑋𝑛𝑛 𝑋𝑋̅ = 𝑛𝑛 tX Assuming the existence of the moment-generating function, MX(t) = E(e ), −h < t < h, for the distribution, the moment-generating function for is: (𝑋𝑋̅ − 𝜇𝜇) 𝜎𝜎 √𝑛𝑛

𝑋𝑋1 + 𝑋𝑋2 + ⋯ + 𝑋𝑋𝑛𝑛 (𝑋𝑋̅ − 𝜇𝜇) − 𝜇𝜇 𝑀𝑀𝑋𝑋𝑋𝑋(𝑡𝑡) = 𝐸𝐸 [exp (𝑡𝑡 ) ] = 𝐸𝐸 [exp (𝑡𝑡 𝑛𝑛 )] 𝜎𝜎⁄√𝑛𝑛 𝜎𝜎⁄√𝑛𝑛

𝑋𝑋1 + 𝑋𝑋2 + ⋯ + 𝑋𝑋𝑛𝑛 ( − 𝜇𝜇) × 𝑛𝑛 = 𝐸𝐸 [exp (𝑡𝑡 𝑛𝑛 )] 𝜎𝜎⁄√𝑛𝑛 × 𝑛𝑛

(𝑋𝑋1 + 𝑋𝑋2 + ⋯ + 𝑋𝑋𝑛𝑛) − 𝑛𝑛𝑛𝑛 = 𝐸𝐸 [exp (𝑡𝑡 )] 𝜎𝜎√𝑛𝑛

Xi is independently sampled at random from the population. So,

(𝑋𝑋1 + 𝑋𝑋2 + ⋯ + 𝑋𝑋𝑛𝑛) − (𝜇𝜇 + 𝜇𝜇 + ⋯ + 𝜇𝜇) = 𝐸𝐸 [exp (𝑡𝑡 )] 𝜎𝜎√𝑛𝑛

(𝑋𝑋1 − 𝜇𝜇) + ( 𝑋𝑋2 − 𝜇𝜇) + ⋯ + (𝑋𝑋𝑛𝑛 − 𝜇𝜇) = 𝐸𝐸 [exp (𝑡𝑡 )] 𝜎𝜎√𝑛𝑛

(𝑋𝑋1 − 𝜇𝜇) ( 𝑋𝑋2 − 𝜇𝜇) (𝑋𝑋𝑛𝑛 − 𝜇𝜇) = 𝐸𝐸 [exp (𝑡𝑡 { + + ⋯ + })] 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛

(𝑋𝑋1 − 𝜇𝜇) ( 𝑋𝑋2 − 𝜇𝜇) (𝑋𝑋𝑛𝑛 − 𝜇𝜇) = 𝐸𝐸 [exp (𝑡𝑡 + 𝑡𝑡 + ⋯ + 𝑡𝑡 )] 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛

𝑋𝑋1 − 𝜇𝜇 𝑋𝑋2 − 𝜇𝜇 𝑋𝑋𝑛𝑛 − 𝜇𝜇 = E [exp (𝑡𝑡 ) exp (𝑡𝑡 ) … exp (𝑡𝑡 )] 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛

2 Since X1, X2, …, Xn are from the same population with mean, μ, and variance, σ , they have a common moment-generating function.

𝑋𝑋 − 𝜇𝜇 𝑋𝑋 − 𝜇𝜇 𝑋𝑋 − 𝜇𝜇 = E [exp (𝑡𝑡 ) exp (𝑡𝑡 ) … exp (𝑡𝑡 )] 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛 𝑛𝑛 𝑋𝑋 − 𝜇𝜇 𝑋𝑋 − 𝜇𝜇 𝑋𝑋 − 𝜇𝜇 𝑋𝑋 − 𝜇𝜇 = E [exp (𝑡𝑡 )] E [exp (𝑡𝑡 )] … E [exp (𝑡𝑡 )] = {𝐸𝐸 [exp (𝑡𝑡 )]} 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛

∵ 𝐸𝐸(𝑋𝑋𝑋𝑋) = 𝐸𝐸(𝑋𝑋)𝐸𝐸(𝑌𝑌), Please refer to the equation (6).

154 Online access in http://ekja.org KOREAN J ANESTHESIOL Kwak and Kim

When we define the moment-generating function for X − μ,

𝑡𝑡(𝑋𝑋−𝜇𝜇) 𝑡𝑡𝑡𝑡−𝑡𝑡𝑡𝑡 𝑋𝑋𝑋𝑋 −𝜇𝜇𝜇𝜇 −𝜇𝜇𝜇𝜇 𝑋𝑋𝑋𝑋 −𝜇𝜇𝜇𝜇 𝑚𝑚𝑋𝑋(𝑡𝑡) = 𝐸𝐸[𝑒𝑒 ] = 𝐸𝐸[𝑒𝑒 ] = 𝐸𝐸[𝑒𝑒 × 𝑒𝑒 ] = 𝑒𝑒 𝐸𝐸(𝑒𝑒 ) = 𝑒𝑒 𝑀𝑀𝑋𝑋(𝑡𝑡), − ℎ < 𝑡𝑡 < ℎ Then,

𝑛𝑛 𝑛𝑛 𝑛𝑛 𝑋𝑋 − 𝜇𝜇 𝑡𝑡 𝑡𝑡 𝑡𝑡 {𝐸𝐸 [exp (𝑡𝑡 )]} = {𝐸𝐸 [exp ( (𝑋𝑋 − 𝜇𝜇))]} = {𝐸𝐸 [exp (−𝜇𝜇 + 𝑋𝑋 )]} 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛 𝑛𝑛 𝑛𝑛 𝑡𝑡 𝑡𝑡 𝑡𝑡 𝑡𝑡 = {𝐸𝐸 [exp (−𝜇𝜇 ) exp (𝑋𝑋 )]} = {exp (−𝜇𝜇 ) 𝐸𝐸 [exp (𝑋𝑋 )]} 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛 𝑛𝑛 𝑛𝑛 𝑡𝑡 𝑡𝑡 𝑡𝑡 𝑡𝑡 = {exp (−𝜇𝜇 ) 𝑀𝑀𝑋𝑋 ( )} = [𝑚𝑚𝑋𝑋 ( )] , − ℎ < < ℎ 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛

Using Taylor’s formula (equation (7)), the moment-generating function for X − μ, mX(t) is:

′ 1 2 𝑚𝑚𝑋𝑋 (0)(𝑡𝑡 − 0) 𝑚𝑚𝑋𝑋′′(𝜉𝜉)(𝑡𝑡 − 0) 𝑚𝑚𝑋𝑋(𝑡𝑡) = 𝑚𝑚𝑋𝑋(0) + + , − ℎ < 𝑡𝑡 < ℎ, 0 < 𝜉𝜉 < 𝑡𝑡 1! 2!

0×(X − μ) Since mX(0) = E[e ] = 1 and mX'(0) = E(X − μ) = E(X) − μ = μ − μ = 0, according to equation (3),

2 𝑚𝑚𝑋𝑋′′(𝜉𝜉)𝑡𝑡 𝑚𝑚𝑋𝑋(𝑡𝑡) = 1 + 2 2! By adding and subtracting σ2t /2,

2 2 2 2 2 2 2 ′′ 2 2 2 𝑚𝑚𝑋𝑋′′(𝜉𝜉)𝑡𝑡 𝜎𝜎 𝑡𝑡 𝜎𝜎 𝑡𝑡 𝜎𝜎 𝑡𝑡 𝑚𝑚𝑋𝑋 (𝜉𝜉)𝑡𝑡 𝜎𝜎 𝑡𝑡 𝑚𝑚𝑋𝑋(𝑡𝑡) = 1 + + − = 1 + + − 2! 2 2 2 2 2 2 2 ′′ 2 2 𝜎𝜎 𝑡𝑡 [𝑚𝑚𝑋𝑋 (𝜉𝜉) − 𝜎𝜎 ]𝑡𝑡 = 1 + + 2 2

When t is replaced by ,, 𝑡𝑡 𝜎𝜎√𝑛𝑛

2 2 2 2 2 𝑡𝑡 ′′ 2 𝑡𝑡 2 ′′ 2 𝜎𝜎 ( ) [𝑚𝑚𝑋𝑋 (𝜉𝜉) − 𝜎𝜎 ] ( ) 𝑡𝑡2 𝑋𝑋 𝑡𝑡2 𝑡𝑡 𝜎𝜎 𝑛𝑛 𝜎𝜎 𝑛𝑛 𝜎𝜎 × [𝑚𝑚 (𝜉𝜉) − 𝜎𝜎 ] 𝑚𝑚𝑋𝑋 ( ) = 1 + √ + √ = 1 + 𝜎𝜎 𝑛𝑛 + 𝜎𝜎 𝑛𝑛 𝜎𝜎√𝑛𝑛 2 2 2 2 2 ′′ 2 2 𝑡𝑡 [𝑚𝑚𝑋𝑋 (𝜉𝜉) − 𝜎𝜎 ]𝑡𝑡 𝑡𝑡 𝑡𝑡 = 1 + + 2 , −ℎ < < ℎ, 0 < 𝜉𝜉 < 2𝑛𝑛 2𝑛𝑛𝜎𝜎 𝜎𝜎√𝑛𝑛 𝜎𝜎√𝑛𝑛 Therefore,

𝑛𝑛 2 ′′ 2 2 𝑛𝑛 𝑡𝑡 𝑡𝑡 [𝑚𝑚𝑋𝑋 (𝜉𝜉) − 𝜎𝜎 ]𝑡𝑡 𝑀𝑀𝑋𝑋𝑋𝑋(𝑡𝑡) = [𝑚𝑚𝑋𝑋 ( )] = [1 + + 2 ] , − ℎ𝜎𝜎√𝑛𝑛 < 𝑡𝑡 < ℎ𝜎𝜎√𝑛𝑛, 𝜎𝜎√𝑛𝑛 2𝑛𝑛 2𝑛𝑛𝜎𝜎

𝑡𝑡 0 < 𝜉𝜉 < 𝜎𝜎√𝑛𝑛

Online access in http://ekja.org 155 Central limit theorem VOL. 70, NO. 2, April 2017

2 2 In ,, as n → ∞, ξ → 0. Since mX"(0) = E[(X − μ) ] = σ , based on equation (4) and the definition of the variance of X (the 𝑡𝑡 mean of the square of the deviation of a random variable, X, from its mean, μ), . 0 < 𝜉𝜉 < 𝜎𝜎√𝑛𝑛 ′′ 2 ′′ 2 𝑋𝑋 𝑋𝑋 𝑛𝑛 lim→∞ 𝑚𝑚 (𝜉𝜉) = σ ⟺ 𝑛𝑛lim→∞[𝑚𝑚 (𝜉𝜉) − 𝜎𝜎 ] = 0 As n becomes infinite, the moment-generating function for is defined as MXn(t). Therefore, (𝑥𝑥̅−𝜇𝜇) 𝜎𝜎 √𝑛𝑛

2 ′′ 2 2 𝑛𝑛 2 𝑛𝑛 𝑋𝑋 ′′ 2 𝑋𝑋𝑋𝑋 𝑡𝑡 [𝑚𝑚 (𝜉𝜉) − 𝜎𝜎 ]𝑡𝑡 𝑡𝑡 𝑋𝑋 𝑛𝑛lim→∞ 𝑀𝑀 (𝑡𝑡) = 𝑛𝑛lim→∞ [1 + + 2 ] = 𝑛𝑛lim→∞ [1 + ] ∵ nlim→∞[𝑚𝑚 (𝜉𝜉) − 𝜎𝜎 ] = 0 2𝑛𝑛 2𝑛𝑛𝜎𝜎 2𝑛𝑛

As is generally known, 𝑛𝑛 𝑛𝑛 . Thus, 1 1 1 𝑛𝑛lim→ ∞ (1 + ) = 𝑛𝑛lim→∞ (1 + 1 × ) = 𝑒𝑒 𝑛𝑛 𝑛𝑛 𝑛𝑛 𝑛𝑛 2 2 2 2 2 𝑡𝑡 ⁄2 𝑋𝑋𝑋𝑋 𝑡𝑡 𝑡𝑡 1 1 × 𝑡𝑡 𝑛𝑛lim→∞ 𝑀𝑀 (𝑡𝑡) = 𝑛𝑛lim→∞ [1 + ] = 𝑛𝑛lim→∞ [1 + × ] = 𝑒𝑒 = exp (0 × 𝑡𝑡 + ), − ∞ < 𝑡𝑡 < ∞ 2𝑛𝑛 2 𝑛𝑛 2

According to equation (5), the moment-generating function for the normal distribution with mean, μ, and variance, σ2, is:

2 2 𝜎𝜎 𝑡𝑡 𝑀𝑀𝑋𝑋(𝑡𝑡) = exp (𝜇𝜇𝜇𝜇 + ) 2 We conclude that the distribution of , with a fixed natural number, n, approximates the normal distribution with mean 0 and (𝑋𝑋̅−𝜇𝜇) 𝜎𝜎 − 2 variance 1. Thus, if the size (n) of X (the mean√𝑛𝑛 of a random sample from a distribution with mean, μ, and variance, σ ) is sufficiently − σ2 large, the distribution of X follows the normal distribution with mean, μ, and variance, n , regardless of the population distribution.

156 Online access in http://ekja.org