Lecture Notes in Applied Probability
Total Page：16
File Type：pdf, Size：1020Kb
Lecture Notes in Applied Probability 2005 Wayne F. Bialas Department of Industrial Engineering University at Buffalo i Copyright °c 2005 Wayne F. Bialas Department of Industrial Engineering University at Buffalo 301 Bell Hall Buffalo, New York 14260 USA E-mail: [email protected] Web: http://www.buffalo.edu/˜bialas All rights reserved. No part of this work may be reproduced by any means without the express written permission of the author. Date printed: May 12, 2005 ii CONTENTS Acknowledgements xi To the student xiii 1 Sets, Sample Spaces and Events 1 Mathematical Models :::::::::::::::::::::::::: 1 Deterministic models :::::::::::::::::::::::: 1 Nondeterministic models :::::::::::::::::::::: 1 Sets ::::::::::::::::::::::::::::::::::: 2 Basic definitions :::::::::::::::::::::::::: 2 Venn diagrams ::::::::::::::::::::::::::: 3 Set operations ::::::::::::::::::::::::::: 3 Set identities :::::::::::::::::::::::::::: 5 Proving statements about sets ::::::::::::::::::: 6 A proof of the distributive law for sets ::::::::::::::: 7 Elementary sets :::::::::::::::::::::::::: 8 Cardinality of sets ::::::::::::::::::::::::: 9 Sample spaces and events ::::::::::::::::::::: 10 Sets of events ::::::::::::::::::::::::::: 11 Self-Test Exercises for Chapter 1 :::::::::::::::::::: 12 Questions for Chapter 1 ::::::::::::::::::::::::: 14 iii iv CONTENTS 2 Enumeration 19 Fundamental Rule :::::::::::::::::::::::::::: 20 Sampling ::::::::::::::::::::::::::::::::: 20 Permutation of n items distinguished by groups :::::::::: 26 Occupancy :::::::::::::::::::::::::::::::: 26 Self-Test Exercises for Chapter 2 :::::::::::::::::::: 29 Questions for Chapter 2 ::::::::::::::::::::::::: 32 3 Probability 35 Probability Spaces :::::::::::::::::::::::::::: 35 Measuring sets ::::::::::::::::::::::::::: 35 Interpretation of probability :::::::::::::::::::: 36 Examples using logical probability ::::::::::::::::: 40 Relationships between English and set notation :::::::::: 41 Deductions from the axioms :::::::::::::::::::: 41 The Birthday Problem :::::::::::::::::::::::::: 45 The birthday problem on the planet Yak :::::::::::::: 48 Conditional Probability and Independence :::::::::::::::: 49 Conditional probability :::::::::::::::::::::: 49 Independence ::::::::::::::::::::::::::: 51 Bayes’ theorem :::::::::::::::::::::::::: 52 Independence of more than two events ::::::::::::::: 54 Self-Test Exercises for Chapter 3 :::::::::::::::::::: 57 Questions for Chapter 3 ::::::::::::::::::::::::: 67 CONTENTS v 4 Random Variables 81 Basic Concepts :::::::::::::::::::::::::::::: 81 Cumulative Distribution Function :::::::::::::::::::: 81 Properties of any cumulative distribution function ::::::::: 81 Discrete Random Variables ::::::::::::::::::::::: 82 Properties of any probability mass function :::::::::::: 82 Examples of discrete distributions ::::::::::::::::: 83 Continuous Random Variables :::::::::::::::::::::: 85 Properties of any probability density function ::::::::::: 86 Differential mass interpretation of the pdf ::::::::::::: 87 Examples of continuous distributions ::::::::::::::: 88 Expectation ::::::::::::::::::::::::::::::: 89 Finding E(X) for some specific distributions :::::::::::::: 90 Binomial distribution :::::::::::::::::::::::: 90 Poisson distribution :::::::::::::::::::::::: 90 Geometric distribution ::::::::::::::::::::::: 91 Exponential distribution :::::::::::::::::::::: 91 Other measures of central tendency ::::::::::::::::::: 92 Self-Test Exercises for Chapter 4 :::::::::::::::::::: 92 Questions for Chapter 4 ::::::::::::::::::::::::: 99 5 Functions of Random Variables 105 Basic Concepts :::::::::::::::::::::::::::::: 105 Some examples ::::::::::::::::::::::::::::: 109 Discrete case :::::::::::::::::::::::::::: 109 Continuous case :::::::::::::::::::::::::: 110 vi CONTENTS Invertible Functions of Continuous Random Variables :::::::::: 112 Expected values for functions of random variables :::::::::::: 115 The kth Moment of X:::::::::::::::::::::::::: 118 Variance ::::::::::::::::::::::::::::::::: 119 Approximating E(h(X)) and Var(h(X)) :::::::::::::::: 121 Chebychev’s Inequality ::::::::::::::::::::::::: 124 The Law of Large Numbers ::::::::::::::::::::::: 126 The Normal Distribution ::::::::::::::::::::::::: 126 Tools for using the standard normal tables ::::::::::::: 131 Self-Test Exercises for Chapter 5 :::::::::::::::::::: 133 Questions for Chapter 5 ::::::::::::::::::::::::: 142 6 Random Vectors 155 Basic Concepts :::::::::::::::::::::::::::::: 155 Definition :::::::::::::::::::::::::::::::: 155 Joint Distributions :::::::::::::::::::::::::::: 157 Discrete Distributions :::::::::::::::::::::::::: 158 Continuous Distributions ::::::::::::::::::::::::: 159 Marginal Distributions :::::::::::::::::::::::::: 162 Functions of Random Vectors :::::::::::::::::::::: 168 Independence of Random Variables ::::::::::::::::::: 171 Testing for independence :::::::::::::::::::::: 171 Expectation and random vectors ::::::::::::::::::::: 173 Random vectors and conditional probability :::::::::::: 178 Conditional distributions :::::::::::::::::::::: 179 Self-Test Exercises for Chapter 6 :::::::::::::::::::: 180 CONTENTS vii Questions for Chapter 6 ::::::::::::::::::::::::: 188 7 Transforms and Sums 199 Transforms :::::::::::::::::::::::::::::::: 199 Notation :::::::::::::::::::::::::::::: 199 Probability generating functions :::::::::::::::::: 199 Moment generating functions ::::::::::::::::::: 203 Other transforms of random variables ::::::::::::::: 209 Linear combinations of normal random variables :::::::::::: 209 The distribution of the sample mean :::::::::::::::: 211 The Central Limit Theorem ::::::::::::::::::::::: 211 The DeMoivre-Laplace Theorem ::::::::::::::::: 213 The distribution of the sample mean for large n:::::::::: 214 Self-Test Exercises for Chapter 7 :::::::::::::::::::: 215 Questions for Chapter 7 ::::::::::::::::::::::::: 221 8 Estimation 227 Basic Concepts :::::::::::::::::::::::::::::: 227 Random samples :::::::::::::::::::::::::: 227 Statistics :::::::::::::::::::::::::::::: 227 Point Estimation ::::::::::::::::::::::::::::: 229 The problem :::::::::::::::::::::::::::: 229 Unbiased estimators :::::::::::::::::::::::: 229 Finding Estimators :::::::::::::::::::::::::::: 229 Method of maximum likelihood :::::::::::::::::: 229 Self-Test Exercises for Chapter 8 :::::::::::::::::::: 230 Questions for Chapter 8 ::::::::::::::::::::::::: 231 viii CONTENTS 9 Probability Models 233 Coin Tossing (Binary) Models :::::::::::::::::::::: 233 The binomial distribution ::::::::::::::::::::: 233 The geometric distribution ::::::::::::::::::::: 234 The Poisson distribution :::::::::::::::::::::: 235 Random Particle Models ::::::::::::::::::::::::: 235 Self-Test Exercises for Chapter 9 :::::::::::::::::::: 237 Questions for Chapter 9 ::::::::::::::::::::::::: 238 10 Approximations and Computations 247 Approximation for the Normal Integral ::::::::::::::::: 247 Approximation for the Inverse Normal Integral ::::::::::::: 248 Questions for Chapter 10 ::::::::::::::::::::::::: 248 A Tables 249 Table I: Standard Normal Cumulative Distribution Function ::::::: 250 Table II: Some Common Probability Distributions :::::::::::: 251 B Answers to Self-Test Exercises 253 C Sample Examination 1 257 Part I: Multiple Choice Questions :::::::::::::::::::: 257 Part II :::::::::::::::::::::::::::::::::: 259 D Sample Examination 2 261 Part I: Multiple Choice Questions :::::::::::::::::::: 261 Part II :::::::::::::::::::::::::::::::::: 264 CONTENTS ix E Sample Final Examination 265 Part I: Multiple Choice Questions :::::::::::::::::::: 265 Part II :::::::::::::::::::::::::::::::::: 271 x CONTENTS ACKNOWLEDGEMENTS Contained herein is a bit of the wisdom of the greatest scholars in probability theory. Some of them were my professors. These notes have benefited the most from the many contributions of my students and teaching assistants. In particular, I am grateful to Mark Chew, Alfred Guiffrida and Nishant Mishra for several contributions to these notes. I must also thank Yu-Shuan Lee Brown, Rajarshi Chakraborty, Kevin Cleary, Shiju David, Satyaki Ghosh-Dastidar, Judah Eisenhandler, Harold T. Gerber, Michael Holender, Dawn Holmes, Derek Hoiem, Simin Huang, Mona Khanna, Steven Kieffer, Keith Klopfle, Lindsay Koslowski, Taneka Lawson, Andrew Lenkei, Nguk Ching Ling, Angelo Morgante, Xiaofeng Nie, James Pasquini, Nikola Petrovic, Amol Salunkhe, Lynne Makrina Tkaczevski, Elif Tokar and Min Zhang for finding numerous mistakes in previous editions. Any remaining errors and omissions are mine. This manual was typeset using Donald Knuth’s TEX with Leslie Lamport’s LaTEX enhancements as implemented by Eberhard Mattes. A LaTEX style file developed by H. Partel was modified and used to format this document. The Adobe AcrobatTM driver, dvipdfm, written by Mark A. Wicks, and AFPL Ghostscript were used to produce the printed output. I would not have dared to prepare these notes without these tools. WFB Buffalo, New York May 12, 2005 xi xii CONTENTS TO THE STUDENT These notes are for a course devoted to describing those processes around us that appear to produce random outcomes. Probability theory provides a way to mathematically represent chance. We will show you how sets can be used to express events, and then assign, in a systematic way, a measure of uncertainty to each of those sets. That measure will be called probability. For some problems, we can assign probabilities to events by simply counting equally likely outcomes. For that, we provide some counting techniques. When outcomes are not