ECE 302: Lecture 2.3 Axioms of Probability

Prof Stanley Chan

School of Electrical and Computer Engineering Purdue University

©Stanley Chan 2020. All Rights Reserved. 1 / 15 Outline

2.1 Set theory 2.2 Probability space 2.3 Axioms of probability 2.3.1 The three axioms 2.3.2 Corollaries derived from the axioms 2.3.3 Examples 2.4 Conditional probability 2.5 Independence 2.6 Bayes theorem

©Stanley Chan 2020. All Rights Reserved. 2 / 15 Probability Law

Definition A probability law is a function P : F → [0, 1] that maps an event A to a real number in [0, 1]. The function must satisfy three axioms known as Probability Axioms.

I. Non-negativity:

II. Normalization:

©Stanley Chan 2020. All Rights Reserved. 3 / 15 Probability Law

III. Additivity:

For any disjoint subsets {A1, A2,...}, it holds that " ∞ # ∞ [ X P An = P[An]. n=1 n=1

©Stanley Chan 2020. All Rights Reserved. 4 / 15 Understanding additivity

If A and B are disjoint, then

P[A ∪ B] = P[A] + P[B], (1)

©Stanley Chan 2020. All Rights Reserved. 5 / 15 Outline

2.1 Set theory 2.2 Probability space 2.3 Axioms of probability 2.3.1 The three axioms 2.3.2 Corollaries derived from the axioms 2.3.3 Examples 2.4 Conditional probability 2.5 Independence 2.6 Bayes theorem

©Stanley Chan 2020. All Rights Reserved. 6 / 15 Properties of Probability

c 1 P[A ] = 1 − P[A].

2 For any A ⊆ Ω, P[A] ≤ 1.

3 P[∅] = 0.

©Stanley Chan 2020. All Rights Reserved. 7 / 15 Properties of Probability

4 For any A and B,

P[A ∪ B] = P[A] + P[B] − P[A ∩ B].

©Stanley Chan 2020. All Rights Reserved. 8 / 15 Properties of Probability

Proof.

©Stanley Chan 2020. All Rights Reserved. 9 / 15 Properties of Probability

5 (Union Bound) For any A and B, P[A ∪ B] ≤ P[A] + P[B].

©Stanley Chan 2020. All Rights Reserved. 10 / 15 Properties of Probability

6 If A ⊆ B, then P[A] ≤ P[B]

Example. A = {t ≤ 5}, and B = {t ≤ 10}, then P[A] ≤ P[B].

©Stanley Chan 2020. All Rights Reserved. 11 / 15 Outline

2.1 Set theory 2.2 Probability space 2.3 Axioms of probability 2.3.1 The three axioms 2.3.2 Corollaries derived from the axioms 2.3.3 Examples 2.4 Conditional probability 2.5 Independence 2.6 Bayes theorem

©Stanley Chan 2020. All Rights Reserved. 12 / 15 Example

Let the events A and B have P[A] = x, P[B] = y and P[A ∪ B] = z. Find the following probabilities. (a) P[A ∩ B]

c c (b) P[A ∩ B ]

©Stanley Chan 2020. All Rights Reserved. 13 / 15 Example

c c (c) P[A ∪ B ]

c (d) P[A ∩ B ]

©Stanley Chan 2020. All Rights Reserved. 14 / 15 Questions?

©Stanley Chan 2020. All Rights Reserved. 15 / 15