Lecture 5: Basic Probability Theory Donglei Du ([email protected]) Faculty of Business Administration, University of New Brunswick, NB Canada Fredericton E3B 9Y2 Donglei Du (UNB) ADM 2623: Business Statistics 1 / 55 Table of contents 1 Probability Theory What is probability? Random Experiment (RE) Set/events operations How to Interpret Probabilities? Rules of Counting Donglei Du (UNB) ADM 2623: Business Statistics 2 / 55 Layout 1 Probability Theory What is probability? Random Experiment (RE) Set/events operations How to Interpret Probabilities? Rules of Counting Donglei Du (UNB) ADM 2623: Business Statistics 3 / 55 What is probability? Probability is a measure of the likelihood that an event in the future will happen. It can only assume a value between 0 and 1; A value near zero means the event is not likely to happen; A value near one means it is likely to happen. Donglei Du (UNB) ADM 2623: Business Statistics 4 / 55 Random Experiment (RE) RE is a process that satisfies the following propositions: the process can be repeated as many trials as you want the outcome of any trial is uncertain well-defined set of possible outcomes each outcome has a probability associated with it Donglei Du (UNB) ADM 2623: Business Statistics 5 / 55 An example Toss a coin once: This process can be repeated as many times as you want (repetitive nature) Nobody knows whether Head or Tail will appear in any particular tossing (uncertain outcome) However, we know either Head or Tail must appear (Well-defined sample space) Now what is the chance you will get a Head if you toss a coin now? Donglei Du (UNB) ADM 2623: Business Statistics 6 / 55 Some Terminologies Outcome: is a particular result of a random experiment Sample Space: is the collection or set of all the possible outcomes of a random experiment Event: is the collection of one or more outcomes of an experiment Donglei Du (UNB) ADM 2623: Business Statistics 7 / 55 Example Toss one coin in very trial The sample space is S = fH, Tg. An event is the occurrence of head: E = fhg. Flip two coins in very trial The sample space has two outcomes S = f(H,H), (H,T), (T,H), (T,T)g. An event is the occurrence of a head in the first coin: E = f(H; H); (H; T )g Roll one die in very trial The sample space contains 6 outcomes S = f1;:::; 6g. An event is the occurrence of an even number: E = f2; 4; 6g. Roll two dice in very trial The sample space contains the 36 outcomes S = f(1; 1);:::; (6; 6)g. An event is the occurrence of two dice with sum equal to 4: E = f(1; 3); (2; 2); (3; 1)g Donglei Du (UNB) ADM 2623: Business Statistics 8 / 55 Venn Diagram A Venn diagram or set diagram is a diagram that shows all possible logical relations between a finite collection of sets. The Venn diagram for the above die example. 1 35 246 Donglei Du (UNB) ADM 2623: Business Statistics 9 / 55 An example Toss two coins in very trial: This process can be repeated as many times as you want (repetitive nature) Every trial contains the outcome of two coins Origin First Second Flip Flip HH H H T HT H TH T T TT Donglei Du (UNB) ADM 2623: Business Statistics 10 / 55 2013/9/16 Donglei Du: Lecture 11 Set operations Union Intersection: Complement: Donglei Du (UNB) ADM 2623: Business Statistics 11 / 55 Union Sample Space A B A B Union: The union event of two events A and B is denoted as A [ B, which consists of all outcomes in either A or B or in both. Namely event A [ B occurs if either A or B occurs. Example: Roll a die: E1 = f1; 3; 5g and E2 = f1; 2; 3g. Then E1 [ E2 = f1; 2; 3; 5g Donglei Du (UNB) ADM 2623: Business Statistics 12 / 55 Intersection Sample Space A A B B Intersection: The intersection event of two events A and B is denoted as A \ B, which consists of all outcomes in both A and B or in both. Namely event A \ B occurs if both A and B occur. Example: Roll a die: E1 = f1; 3; 5g and E2 = f1; 2; 3g. Then E1 \ E2 = f1; 3g Two events A and B are mutually exclusive if A \ B = ;. Donglei Du (UNB) ADM 2623: Business Statistics 13 / 55 Complement Sample Space A A Complement: The complement event of an event A is denoted as A¯, which consists of all outcomes that are not in A. Namely event A¯ occurs if A does not occur. Example: Roll a die: E1 = f1; 3; 5g. Then E¯1 = f2; 4; 6g Donglei Du (UNB) ADM 2623: Business Statistics 14 / 55 Probabilities defined on events For any random experiment with sample space S, the probability of any event is P (E) satisfying (i) 0 ≤ P (E) ≤ 1. (ii) P (S) = 1. (iii) (special addition rule) For any pair-wise mutually exclusive events E1;E2;::: 1 ! 1 [ X P En = P (En): n=1 i=1 Donglei Du (UNB) ADM 2623: Business Statistics 15 / 55 Examples Toss one fair coin in very trial: If we assume a head is equally likely to appear as a tail, then 1 P (H) = P (T ) = 2 Toss one biased coin in very trial: If we assume a head is twice as likely to appear as a tail, then 2 1 P (H) = ;P (T ) = : 3 3 Toss one fair die in very trial: If we assume all sides are equally likely to appear, then 1 P (i) = ; i = 1;:::; 6: 6 From (iii) in the previous slide, we must have 1 P (f1; 3; 5g) = P (1) + P (2) + P (3) = : 2 Donglei Du (UNB) ADM 2623: Business Statistics 16 / 55 Probability Distributions defined on sample space A (discrete) probability distribution on a given sample space is a table of all disjoint outcomes and their associated probabilities. Example: Toss two fair coins in very trial: The table below is the probability distribution for the sample space S = fHH;HT;TH;TT g: Outcome Prob. HH 0.25 HT 0.25 TH 0.25 TT 0.25 Donglei Du (UNB) ADM 2623: Business Statistics 17 / 55 Rule of Probability Complement Rule Addition Rule Multiplication Rule Donglei Du (UNB) ADM 2623: Business Statistics 18 / 55 Rule of Complement For any event A: P (A¯) = 1 − P (A) Donglei Du (UNB) ADM 2623: Business Statistics 19 / 55 Rule of Addition For any two events A and B: P (A [ B) = P (A) + P (B) − P (A \ B): If events A and B are mutually exclusive: P (A [ B) = P (A) + P (B): The above can be extended to any number of mutually exclusive events A1;:::;An: P (A1 [ ::: [ An) = P (A1) + ::: + P (An): Donglei Du (UNB) ADM 2623: Business Statistics 20 / 55 Rule of Addition: Example Toss two fair coins in very trial and assume that the four possible outcomes S = fHH;HT;TH;TT g are equally likely to happen. Let A = fHH; HT g be the event that the first coin falls head, and B = fHH; T Hg be the event that the second coin falls head. Problem: Find the probability that either the first coin falls head or the second coin falls head? Solution: 1 1 1 3 P (A [ B) = P (A) + P (B) − P (A \ B) = + − = : 2 2 4 4 Donglei Du (UNB) ADM 2623: Business Statistics 21 / 55 Rule of Multiplication Joint probability Conditional probability Donglei Du (UNB) ADM 2623: Business Statistics 22 / 55 Joint probability Joint probability: For two events A and B defined on the same sample space, the joint probability of events A and B is P (A \ B). Example: Roll one die in very trial. Let A = f1; 2; 3g and B = f1; 3; 5g be two events. Then the joint probability of A and B is 2 1 P (A \ B) = P (f1; 3g) = = : 6 3 Donglei Du (UNB) ADM 2623: Business Statistics 23 / 55 Represent Joint probability as contingency table Joint probability distribution is given in a tabular form, called contingency table. The probability distribution for each variable is called the marginal distribution. Example: A survey of undergraduate students in the Faculty of Business Management at UNB revealed the following regarding the gender and majors of the students: Accoun. IB HR Marginal Male 150/750 150/750 50/750 350/750 Female 175/750 160/750 65/750 400/750 Marginal 325/750 310/750 115/750 1 Example: What is the Probability of selecting a Female Accounting student? Solution: P (F \ A) = 175=750 = 23:33% Donglei Du (UNB) ADM 2623: Business Statistics 24 / 55 Conditional probability Conditional probability: For any event A and B, the probability of event A, given the occurrence of event B: P (A \ B) P (AjB) = : P (B) Example: What is the probability of selecting a Female, given that the person selected is an International Business major? Solution: P (F jI) = 160=310 = 51:6% Compare to the unconditional probability: P (F ) = 400=750 = 50% Donglei Du (UNB) ADM 2623: Business Statistics 25 / 55 Independent events: two events Two events A and B are independent if and only if P (AjB) = P (A) or equivalently P (A \ B) = P (A) × P (B): Example: the events F = f(F; A); (F; IB); (F; HR)g and IB = f(F; IB); (M; IB)g are not independent, since P (F jIB) = 160=310 = 51:6% 6= 50% = 400=750 = P (F ) Donglei Du (UNB) ADM 2623: Business Statistics 26 / 55 Independent events: more than two events n events A1;:::;An are independent if and only if for any r ≤ n: P (A1 \ ::: \ Ar) = P (A1) × ::: × P (Ar): Example: (Pairwise independent events may not be independent overall): Roll a four-faced die once.