Probability Events, experiments,

• We refer to any observational or experimental experience as an experiment. The experiment will have an that is unknown before the experiment and will be known without uncertainty after the experiment • The possible outcomes of the experiment are denoted as 휔 • Their set, usually known in advance, is called sample space and denoted as Ω Examples:

• Tossing a coin 3 times: Ω = 푇푇푇, 푇푇퐻, 푇퐻푇, 푇퐻퐻, 퐻푇푇, 퐻푇퐻, 퐻퐻푇, 퐻퐻퐻 , Ω = 23 = 8 • Rolling a die: Ω = 1,2,3,4,5,6 , Ω = 6

• Sampling without replacement 3 balls from an urn of 5 balls labeled 1 - 5 Ω = 123, 124, 125, 134, 135, 145, 234, 235, 245, 345 ; Ω = 10 • Number of consecutive days before a certain company’s shares go down Ω = ℕ Events Ω

An event is any assertion on the experiment whose validity can be verified after its realization. Some outcomes will imply the event is false, the other will imply the event is true. A We can directly identify the event with the outcomes leading to its validity. So, an event is a subspace of the sample space Ω Examples with a die: 퐴 = {even outcome} = {2,4,6} 퐵 퐵 = {outcome greater than 3} = {4,5,6} Events Ω Symmetry between logical operators and set operators: 퐴={even outcome}, 퐵 ={outcome greater than 3} • Logical intersection {퐴 푎푛푑 퐵} ≡ 퐴 ∩ 퐵 = 4,6 ={even outcome greater than 3} A • Logical union (inclusive or) {퐴 표푟 퐵} ≡ 퐴 ∪ 퐵 = 2,4,5,6 ={any outcome even or > 3} • Negation (complementary event) 퐴ҧ = odd otcome 퐵ത = {smaller than 4} 퐴 ∩ 퐴ҧ = ∅ 퐴 ∪ 퐴ҧ = Ω 퐵 Events Ω 퐴 = {even outcome} = {2,4,6} 퐵 = {outcome greater than 3} = {4,5,6} C C = {outcome smaller than 2} = {1}

• Two events are said mutually exclusive (or disjoint) if A their intersection is empty (cannot both be true) 퐴 and C are disjoint, 퐵 and C are disjoint

• De Morgan’s laws: 퐴 ∩ 퐵 = 퐴ҧ ∪ 퐵ത = 1,2,3,5 퐵 퐴 ∪ 퐵 = 퐴ҧ ∩ 퐵ത = {1,3} Definition of Probability

• Classic definition • The probability of event 퐴 is the ratio between the number of outcomes where 퐴 occurs over the total number of possible outcomes. (Valid only if the possible outcomes are equally likely) • Frequentist definition • Limit of the relative frequency of 퐴 in repeated trials of the experiment (basis for all the classic frequentist ) • Subjective definition • Probability of 퐴 depends on the informational status of each subject, and measures the subjective degree of confidence towards 퐴 (basis for Bayesian statistics) Kolmogorov’s Axiomatic definition

Probability is a function 푃(⋅) defined over a family of subsets of Ω satisfying the following properties: 1. 푃 Ω = 1; 2. 푃 퐴 ≥ 0, for any event 퐴; 3. 푃 퐴1 ∪ 퐴2 ∪ ⋯ = 푃 퐴1 + 푃 퐴2 + ⋯, whenever the events are mutually exclusive.

Consequently, for any event 퐴: • If Ω is discrete, 푃 퐴 = σ휔∈퐴 푃(휔) • 푃 퐴ҧ = 1 − 푃 퐴 퐴 ∩ 퐴ҧ = ∅ , 퐴 ∪ 퐴ҧ = Ω • 푃 퐴 ≤ 1 • 푃 ∅ = 0 (Ωഥ = ∅) For any two events 퐴 and 퐵: Ω • 푃 퐴 ∪ 퐵 = 푃 퐴 + 푃 퐵 − 푃 퐴 ∩ 퐵 푃 퐴 ∪ 퐶 = 푃 퐴 + 푃 퐶 (퐴 and 퐶 are disjoint) 퐴

• 푃 퐴 ∩ 퐵 = 푃(퐴) + 푃(퐵) − 푃 퐴 ∪ 퐵 푃 퐴 ∩ 퐶 = 0 (퐴 and 퐶 are disjoint)

• 퐴 푥표푟 퐵 ("exclusive or" 퐴 or 퐵 but not both) 퐶 푃 퐴 푥표푟 퐵 = 푃 퐴 + 푃 퐵 − 2푃 퐴 ∩ 퐵 푃 퐴 푥표푟 퐶 = 푃 퐴 + 푃 퐶 (퐴 and 퐶 are disjoint)

• 퐶 ⊆ 퐵 ≡ "퐶 implies 퐵” (If 퐶 is true then 퐵 is true) 퐵 퐶 ⊆ 퐵 ⟹ 푃 퐶 ≤ 푃(퐵) Elements of combinatorics

You have 푛 objects: • The orderings or arrangements of the objects are called permutations, and the total number of possible distinct permutations is 푛! = 푛 ⋅ 푛 − 1 ⋯ 2 ⋅ 1

• We sample 푘 out of the 푛 objects without replacement. The number of possible distinct samples (if we are interested in the order) is the number of dispositions 푛! 퐷 = 푛 ⋅ 푛 − 1 ⋯ 푛 − 푘 + 1 = ς푘 푛 − 푖 + 1 = 푛,푘 푖=1 (푛−푘)!

• We sample 푘 out of the 푛 objects without replacement. The number of possible distinct samples (if we are NOT interested in the order) is the number of combinations 퐷 푛 ⋅ 푛 − 1 ⋯ 푛 − 푘 + 1 ς푘 푛 − 푖 + 1 푛! 푛 퐶 = 푛,푘 = = 푖=1 = = 푛,푘 푘! 푘! 푘! 푛 − 푘 ! 푘! 푘 Example

An urn contains 5 balls labeled 1:5. We sample at random 3 balls without replacement. What is the probability that the first and last balls are even?

All possible outcomes are equally likely, so we calculate the probability as number of favorable cases over number of possible cases. The number of possible cases is the number of dispositions: 5! Ω = 퐷 = = 5 ⋅ 4 ⋅ 3 = 60 5,3 (5 − 3)! Under 퐴 = {first and last even}, the first and last balls are either {2,4} or {4,2}, while the second ball can be any odd value. So, the favorable cases are: {{2,1,4}, {2,3,4},{2,5,4},{4,1,2},{4,3,2},{4,5,2}} the number of favorable cases is 6 and we have: 6 1 푃 퐴 = = 60 10 Example

An urn contains 5 balls labeled 1:5. We sample at random 3 balls without replacement. What is the probability that two balls are even and one is odd?

We are not interested in the order, then the number of possible outcomes is the number of combinations: 5 Ω = 퐶 = = 10 5,3 3

Under 퐴 = {two even one odd}, we have a sample consisting of 2,4, and any odd value. So, the number of favorable cases is 3 and we have: 3 푃 퐴 = 10

Let 퐴 and 퐵 be two events in the same sample space Ω, and let 푃 퐴 > 0, then the conditional probability of 퐵 given 퐴 is defined as:

푃(퐴 ∩ 퐵) 푃 퐵 퐴 = 푃(퐴) If 퐴 and 퐵 disjoint ⟹ 푃 퐵 퐴 = 푃 퐴 퐵 = 0

We can write the joint probability in terms of conditional probability in at least two ways:

푃 퐴∩퐵 • 푃 퐵 퐴 = ⟹ 푃 퐴 ∩ 퐵 = 푃 퐵 퐴 푃(퐴) 푃(퐴) 푃 퐴∩퐵 • 푃 퐴 퐵 = ⟹ 푃 퐴 ∩ 퐵 = 푃 퐴 퐵 푃(퐵) 푃(퐵) Conditional Probability

• Let 퐵1, 퐵2, … be mutually exclusive events, then:

푃[ 퐵 ∪ 퐵 ∪ ⋯ ∩ 퐴] 푃[ 퐵 ∩ 퐴 ∪ 퐵 ∩ 퐴 ⋯ ] 푃 퐵 ∪ 퐵 ∪ ⋯ 퐴 = 1 2 = 1 2 1 2 푃(퐴) 푃(퐴)

푃 퐵 ∩ 퐴 + 푃 퐵 ∩ 퐴 + ⋯ = 1 2 푃(퐴)

= 푃 퐵1 퐴 + 푃 퐵2 퐴 + ⋯ Example

We sample a card from a 52 cards deck. What is the probability of it not being a king, given that it is a face?

퐴 = {face} and 퐵 = {not king}

|퐴| 12 |퐵| 48 푃 퐴 = = 푃 퐵 = = |Ω| 52 |Ω| 52

|퐴 ∩ 퐵| 8 푃 퐴 ∩ 퐵 = = |Ω| 52

푃(퐴 ∩ 퐵) 8 푃 퐵 퐴 = = 푃(퐴) 12

You obtain the same result if you consider that, conditionally on 퐴, you either have a Jack, or a Queen or a King, then 푃 퐵 퐴 = 2/3

퐶1, 퐶2, … , 퐶푘 constitute a partition of Ω if they are mutually exclusive and exhaustive of Ω: Ω • 퐶푖 ∩ 퐶푗 = ∅ for any 푖 ≠ 푗 퐶 • 퐶1 ∪ 퐶2 ∪ ⋯ ∪ 퐶푘 = Ω 1 퐶… 퐶 퐴 We can write the marginal probability of 퐴 given … the joint : 푘 푘 퐶2 퐶… 푃 퐴 = ෍ 푃(퐴 ∩ 퐶푗) = ෍ 푃 퐴 퐶푗 푃(퐶푗) 푗=1 푗=1 퐶… 퐶푘 Example

A supermarket has 3 checkout lines. The probability of spending 10 minutes or more in the first is 0.2, at the second is 0.6 and at the third is 0.4. What is the probability of a random customer of spending 10+ minutes?

퐴 = 10+ minutes in line

푃 퐴 퐶1 = 0.2 푃 퐴 퐶2 = 0.6 푃 퐴 퐶3 = 0.4

푃 퐶1 = 푃 퐶2 = 푃 퐶3 = 1/3 1 1 1 푃 퐴 = 0.2 ⋅ + 0.6 ⋅ + 0.4 ⋅ = 0.4 3 3 3 Bayes’ theorem

Let 퐶1, 퐶2, … , 퐶푘 be a partition of |Ω| and 푃 퐴 > 0

푃(퐴 ∩ 퐶푖) 푃 퐴 퐶푖 푃(퐶푖) 푃 퐴 퐶푖 푃(퐶푖) 푃 퐶푖|퐴 = = = 푘 푃(퐴) 푃(퐴) σ푗=1 푃 퐴 퐶푗 푃(퐶푗)

In certain context, 퐶1, 퐶2, … , 퐶푘 are considered as possible causes of 퐴 and 푃(퐶푖) and 푃(퐶푖|퐴) are called respectively prior and posterior probability of the event 퐶푖 as a cause of 퐴. Example

A supermarket has 3 checkout lines. The probability of spending 10 minutes or more in the first is 0.2, at the second is 0.6, and at the third is 0.4. Knowing that a customer spent more than 10 minutes in line, what is the probability that it chose the second line? 퐴 = 10+ minutes in line

푃 퐴 퐶1 = 0.2 푃 퐴 퐶2 = 0.6 푃 퐴 퐶3 = 0.4

푃 퐶1 = 푃 퐶2 = 푃 퐶3 = 1/3 (퐩퐫퐢퐨퐫퐬)

푃 퐴 퐶 푃(퐶 ) 0.6 ⋅ 1/3 푃 퐶 퐴 = 2 2 = = 0.5 (퐩퐨퐬퐭퐞퐫퐢퐨퐫) 2 푃(퐴) 0.4 Independent events

• Event 퐵 is independent from 퐴 if the occurrence of 퐴 does not alter the probability of 퐵: 푃 퐵 퐴 = 푃 퐵 , (푃 퐴 > 0) • Independence is a symmetric relation, so we say that A and 퐵 are (mutually) independent if 푃 퐴 ∩ 퐵 = 푃 퐴 푃 퐵 This definition is equivalent to the previous, but is preferred as it holds when P 퐴 or P 퐵 are 0.

Note that disjoint events are not independent! 푃 퐴∩퐵 푃 ∅ 푃 퐵 퐴 = = = 0 even if 푃 퐵 ≠ 0 푃 퐴 푃 퐴 Example Roll a die. Are the two events 퐴 ={outcome is even} and 퐵 ={outcome > 2} independent?

퐴 = 2,4,6 퐵 = 3,4,5,6 퐴 ∩ 퐵 = {4,6}

3 1 4 2 2 1 푃 퐴 = = 푃 퐵 = = 푃 퐴 ∩ 퐵 = = 6 2 6 3 6 3

푃 퐴 ∩ 퐵 2 푃 퐵 퐴 = = = 푃(퐵) 푃 퐴 3

푃 퐴 ∩ 퐵 1 푃 퐴 퐵 = = = 푃(퐴) 푃 퐵 2