Basic Terminology, Distributions and the Central Limit Theorem

4.12.2020

Basic terminology, distributions Outline and the central limit theorem

• Basic concepts Terminology Descriptive statistics

• Distributions and probability Binomial and Poisson distributions Normal distribution Statistical distributions

• Central limit theorem Lecture 1 Biological Statistics III Sum of independent effects approaches normal Ayco Tack 1 2

Statistics: general methods to analyze and evaluate Statistics: general methods to analyze and evaluate empirical observations empirical observations

Descriptive methods: Unit: • Summarize data in some way • Smallest independent entity from which data is collected: individual, species, – Frequency distributions – Measures of location locality, etc – Measures of spread Population: Parameter estimation: • Use a statistic to estimate the ``true value´´ of a population parameter • The set of all units: real population (field study) or imagined population • Evaluate the accuracy of the estimate (experiment) – Standard error – Confidence interval Sample: Hypothesis testing: • Data from a number of units. Each unit should be randomly selected from the • Accept or reject a null hypothesis based on the value of a test statistic population • The null hypothesis usually has the form: – No difference between groups – No relationship between variables Basic idea of statistics: – The variable has some specified distribution • By performing the same kind of observation on several independent units, one can draw more solid and general conclusions

3 4

A variable is a property of a unit Descriptive statistics Pupal weight and sex in a sample of 102 Pieris napi butterfly individuals

Measures of location Measures of spread Quantitative variable (continuous): Qualitative variable: (= central tendency) Pieris napi (green-veined white) pupal • weight Sex Frequency • Arithmetic mean Standard deviation (SD) • • Ordinary average Standard deviation is the square root of Male 65 mean squared deviation 1 • 푦ത = σ푛 푦 Female 37 푛 푖=1 푖 • Deviation: 푦푖 − 푦ത 푛 2 102 • Geometric mean • Sum of squares: 푆푆 = σ푖=1(푦푖 −푦ത) 1 • Sample variance (mean squared deviation): • exp( σ푛 ln 푦 )) 푛 푖=1 푖 푆푆 푠2 = • Useful for the average of rates 푛 − 1 • Harmonic mean • Sample SD: 푠 = 푠2 −1 σ푛 푦−1 • Range: • A variable has a ´true´ distribution • 푖=1 푖 푛 • Max - min (population) • Less sensitive to outliers • A sample gives an estimated distribution • Median • Uncertainty • Middle value • Mode • Most common value

5 6

1 4.12.2020

Arithmetic mean Harmonic mean

• The data are not overly skewed (no extreme outliers) • The harmonic mean is less sensitive to outliers • The individual data points are not dependent on each other (like for the geometric mean)

7 8

Geometric mean Why is the level of spread so important

4 r = 1 4 r = 4 16

4 r = 2.5 10 r = 2.5 25

• Sometimes the geometric mean can be more informative

9 10

Description of the shape of a distribution Skewness and kurtosis

Skewness & kurtosis • Skewness • Deviation from symmetry • Skewed to the right (left) means that the right (left) tail is drawn out • Kurtosis • Deviation from normal distribution shape • Platykurtic: squarish shape • Leptokurtic: Narrow with long tails

Quantiles Quartile: 2nd quartile = median Decile: 5th decile = median Percentile: 50th percentile is median 11 12

2 4.12.2020

Binomial distribution

Binomial distribution Binomial distribution • A binomial distribution gives the number (y) of ´successes` in k independent trials, MORE OR LESS PODCAST with p the probability of success in each trial • Distribution function: A short history on probability • Pr 푦 = 푘 푝푦 1 − 푝 푘−푦 푦 https://www.bbc.co.uk/programmes/p08tw9pl • Mean: • 휇 = 푘푝 • Variance: Statistically savvy parrots 2 • 휎 = 푘푝(1 − 푝) https://www.bbc.co.uk/programmes/p08dx986 • Standard deviation: • 휎 = 푘푝(1 − 푝)

13 14

Poisson distribution Poisson distribution

Poisson distribution Related to the binomial distribution (one extreme case) • Distribution function: −μ 푦 • Pr 푦 = 푒 휇 푦! • Mean: • 휇 • Variance: • 휎2 = 휇 • Standard deviation: • 휎 = 휇 휇

Two kinds of deviations from the Poisson distribution • Clumped (overdispersed): • 휎2 > 휇 • Repulsed (underdispersed): • 휎2 < 휇

http://www.umass.edu/wsp/resources/poisson/ 15 16 Bortkewitsch, 1898, Das Gesetz der kleinen Zahlen, pages 23-25

Opinions about probability Poisson distribution – a classic case (from the book "The challenge of chance” by Hardy et al (1975))

The statistics of the New York Department of Fitting a Poisson distribution to data How do the dogs of New York know when to stop Health show that in 1955 the average number of biting, and when to make up the daily quota? How dogs biting people reported per day was 75.3; in do the murderers of England and Wales know to 1956, 73.6; in 1957, 73.5; in 1958, 74.5; in 1959, stop at four victims per million? 72.4. A similar statistical reliability was shown by cavalry horses administering fatal kicks to soldiers in the German army of the last century; they were apparently guided by the so-called Poisson Bortkiewz equation of probability theory. Murderers in England and Wales, however different in character and motives, displayed the same respect for the law of statistics: since the end of the first World They end with the following quote: • Data published in 1898 on the number of men killed War, the average number of murders over by being kicked by a horse in 10 Prussian army corps successive decades was: 1920-29, 3.84 per million I understand that I don’t really understand what I in the course of 20 years. of the population; 1930-39, 3.27 per million; have the illusion of understanding, since all • From the total of 200 cases (army corps per year), 1940-49, 3.92 per million; 1950-59, 3.3 per language is no more than a mirage of the total number of deaths from horse kicks were million; 1960-69, approximately 3.5 per million. comprehensibility above a sea of unknowing. 122, and the sample average is thus 122/200 = 0.610 men killed per corps and year, and the sample These bizarre examples may sharpen our variance is 0.611. appreciation of the mysterious nature of probability. • The fit to Poisson is very good (almost too good)

17 18

3 4.12.2020

Normal distribution Normal distribution

Normal distribution Normal distribution Normal distribution When a variable is the sum of many When a variable is the sum of many independent but similar contributions, its independent but similar contributions, its distribution will be approximately normal distribution will be approximately normal • Distribution function:

• 푥−휇 2 Distribution function: 1 − • Pr 푥 = 푒 2휎2 푥−휇 2 휎 2휋 1 − • Pr 푥 = 푒 2휎2 휎 2휋 • Mean: • Mean: • 휇 • 휇 • Variance: • 휎2 • Variance: • Standard deviation: • 휎2 • 휎 • Standard deviation: • 휎 • 휇 ± 휎 contains 68.7% • 휇 ± 2휎 contains 95.45% • 휇 ± 3휎 contains 99.73% x

19 20

Normal distribution Normal distribution

Z-distribution

21 22

...some other distributions... Testing for normality

Pieris napi pupal weight F-distribution Student’s t-distribution There are several methods of checking whether a variable is normally distributed: • Graphical methods • Test for skew or kurtosis • Tests that check for any kind of deviation from normality: • Shapiro-Wilks – Supposed to be the best • Kolmogorov-Smirnov – Requires that 휇 and 휎 are known 2 • Lilliefors Χ -distribution – The Shapiro-Wilks test is better

People often worry too much about testing for normality

23 24 Shapiro.test pupal weights: W = 0.986, p = 0.39

4 4.12.2020

From the distribution of the sample to the distribution Central limit theorem of the sample mean In probability theory, the central limit theorem (CLT) states that, given certain conditions, the arithmetic mean and sum of a sufficiently large number of iterates of independent random variables, each with a well- defined expected value and well-defined variance, will be approximately normally distributed, regardless of the The distribution of the sample underlying distribution. means = That is, suppose that a sample is obtained containing a large number of observations, each observation being • If you draw samples from a normal randomly generated in a way that does not depend on the values of the other observations, and that the distribution, then the distribution of arithmetic average of the observed values is computed. If this procedure is performed many times, the central limit theorem says that the computed values of the average will be distributed according to the normal these sample means is also normal. distribution (commonly known as a "bell curve"). • The mean of the distribution of Adapted from Wikipedia sample means is identical to the mean of the "parent population," the • http://www.youtube.com/watch?v=JNm3M9cqWyc population from which the samples are drawn. “I know of scarcely anything so apt to impress the imagination as the wonderful form of cosmic order expressed by • The higher the sample size that is the ‘Law of Frequency of Error’. The law would have been personified by the Greeks and deified, if they had known drawn, the "narrower" will be the of it. It reigns with serenity and in complete self-effacement, amidst the wildest confusion. The huger the mob, and the greater the apparent anarchy, the more perfect is its sway. It is the supreme law of Unreason. Whenever a spread of the distribution of sample large sample of chaotic elements are taken in hand and marshaled in the order of their magnitude, an means. unsuspected and most beautiful form of regularity proves to have been latent all along.” Sir Francis Galton (Natural Inheritance, 1889)

http://msenux.redwoods.edu/math/R/CentralLimit.php25 26

Central limit theorem Central limit theorem Example from the symmetric binomial distribution Example from the symmetric binomial distribution

In probability theory, the central limit theorem (CLT) states that, given certain conditions, the arithmetic mean and sum of a sufficiently In probability theory, the central limit theorem (CLT) states that, given certain conditions, the arithmetic mean of a sufficiently large large number of iterates of independent random variables, each with a well-defined expected value and well-defined variance, will be number of iterates of independent random variables, each with a well-defined expected value and well-defined variance, will be approximately normally distributed, regardless of the underlying distribution. approximately normally distributed, regardless of the underlying distribution. = That is, suppose that a sample is obtained containing a large number of observations, each observation being randomly generated in a That is, suppose that a sample is obtained containing a large number of observations, each observation being randomly generated in a way that does not depend on the values of the other observations, and that the arithmetic average of the observed values is computed. way that does not depend on the values of the other observations, and that the arithmetic average of the observed values is computed. If this procedure is performed many times, the central limit theorem says that the computed values of the average will If this procedure is performed many times, the central limit theorem says that the computed values of the average will be distributed according to the normal distribution (commonly known as a "bell curve"). be distributed according to the normal distribution (commonly known as a "bell curve"). Adapted from Wikipedia [Wikipedia]

27 28

Central limit theorem Central limit theorem Example from a skewed binomial distribution Example from a skewed binomial distribution

29 30

5 4.12.2020