MATH1024: Introduction to Probability and Statistics Prof Sujit Sahu Autumn 2020 2 Contents 1 Introduction to Statistics 9 1.1 Lecture 1: What is statistics? . .9 1.1.1 Early and modern definitions . .9 1.1.2 Uncertainty: the main obstacle to decision making . 10 1.1.3 Statistics tames uncertainty . 10 1.1.4 Why should I study statistics as part of my degree? . 10 1.1.5 Lie, Damn Lie and Statistics? . 11 1.1.6 What's in this module? . 11 1.1.7 Take home points: . 12 1.2 Lecture 2: Basic statistics . 12 1.2.1 Lecture mission . 12 1.2.2 How do I obtain data? . 12 1.2.3 Summarising data . 13 1.2.4 Take home points . 16 1.3 Lecture 3: Data visualisation with R .......................... 16 1.3.1 Lecture mission . 16 1.3.2 Get into R ...................................... 17 1.3.3 Working directory in R ............................... 18 1.3.4 Keeping and saving commands in a script file . 19 1.3.5 How do I get my data into R?........................... 19 1.3.6 Working with data in R .............................. 20 1.3.7 Summary statistics from R ............................. 21 1.3.8 Graphical exploration using R ........................... 21 1.3.9 Take home points . 22 2 Introduction to Probability 23 2.1 Lecture 4: Definitions of probability . 23 2.1.1 Why should we study probability? . 23 2.1.2 Two types of probabilities: subjective and objective . 23 2.1.3 Union, intersection, mutually exclusive and complementary events . 24 2.1.4 Axioms of probability . 26 2.1.5 Application to an experiment with equally likely outcomes . 27 2.1.6 Take home points . 27 2.2 Lecture 5: Using combinatorics to find probability . 27 3 CONTENTS 4 2.2.1 Lecture mission . 27 2.2.2 Multiplication rule of counting . 28 2.2.3 Calculation of probabilities of events under sampling `at random' . 29 2.2.4 A general `urn problem' . 29 2.2.5 Take home points . 31 2.3 Lecture 6: Conditional probability and the Bayes Theorem . 31 2.3.1 Lecture mission . 31 2.3.2 Definition of conditional probability . 32 2.3.3 Multiplication rule of conditional probability . 32 2.3.4 Total probability formula . 33 2.3.5 The Bayes theorem . 35 2.3.6 Take home points . 35 2.4 Lecture 7: Independent events . 36 2.4.1 Lecture mission . 36 2.4.2 Definition . 36 2.4.3 Take home points . 38 2.5 Lecture 8: Fun probability calculation for independent events . 39 2.5.1 Lecture mission . 39 2.5.2 System reliability . 39 2.5.3 The randomised response technique . 40 2.5.4 Take home points . 41 3 Random Variables and Their Probability Distributions 43 3.1 Lecture 9: Definition of a random variable . 43 3.1.1 Lecture mission . 43 3.1.2 Introduction . 43 3.1.3 Discrete or continuous random variable . 44 3.1.4 Probability distribution of a random variable . 44 3.1.5 Cumulative distribution function (cdf) . 46 3.1.6 Take home points . 47 3.2 Lecture 10: Expectation and variance of a random variable . 48 3.2.1 Lecture mission . 48 3.2.2 Mean or expectation . 48 3.2.3 Take home points . 50 3.3 Lecture 11: Standard discrete distributions . 50 3.3.1 Lecture mission . 50 3.3.2 Bernoulli distribution . 50 3.3.3 Binomial distribution . 51 3.3.4 Geometric distribution . 53 3.3.5 Hypergeometric distribution . 54 3.3.6 Take home points . 55 3.4 Lecture 12: Further standard discrete distributions . 55 3.4.1 Lecture mission . 55 3.4.2 Negative binomial distribution . 55 3.4.3 Poisson distribution . 56 CONTENTS 5 3.4.4 Take home points . 57 3.5 Lecture 13: Standard continuous distributions . 58 3.5.1 Lecture mission . 58 3.5.2 Exponential distribution . 58 3.5.3 Take home points . 61 3.6 Lecture 14: The normal distribution . 62 3.6.1 Lecture mission . 62 3.6.2 The pdf, mean and variance of the normal distribution . 62 3.6.3 Take home points . 64 3.7 Lecture 15: The standard normal distribution . 64 3.7.1 Lecture mission . 64 3.7.2 Standard normal distribution . 64 3.7.3 Take home points . 67 3.8 Lecture 16: Joint distributions . 67 3.8.1 Lecture mission . 67 3.8.2 Joint distribution of discrete random variables . 68 3.8.3 Covariance and correlation . 70 3.8.4 Independence . 71 3.8.5 Take home points . 72 3.9 Lecture 17: Properties of the sample sum and mean . 72 3.9.1 Lecture mission . 72 3.9.2 Introduction . 73 3.9.3 Take home points . 75 3.10 Lecture 18: The Central Limit Theorem . 76 3.10.1 Lecture mission . 76 3.10.2 Statement of the Central Limit Theorem (CLT) . 76 3.10.3 Application of CLT to binomial distribution . 77 3.10.4 Take home points . 79 4 Statistical Inference 81 4.1 Lecture 19: Foundations of statistical inference . 81 4.1.1 Statistical models . 82 4.1.2 A fully specified model . 82 4.1.3 A parametric statistical model . 83 4.1.4 A nonparametric statistical model . 83 4.1.5 Should we prefer parametric or nonparametric and why? . 84 4.1.6 Take home points . 84 4.2 Lecture 20: Estimation . 84 4.2.1 Lecture mission . ..