Basic Concepts of Probability Theory (Part I)

Home , Elementary event, Outcome (probability), Sample space

ECE270: Handout 1

Outline: 1. a random experiment, 2. an experiment outcome, 3. sample space and its three diﬀerent types, 4. events, 5. review of set theory, Venn diagrams, and DeMorgan’s laws.

F Random Experiment • Random experiment is an experiment in which the outcome varies in a unpredictable fashion, when the experiment is repeated under the same conditions. • A random experiment is denoted by E and is speciﬁed by i) stating an experimental proce- dure, ii) exactly what is (are) measured or observed.

EXAMPLE 1

– An information block of length 5 is transmitted repeatedly over a noisy channel until an error-free block arrives at the receiver. Count the number of transmission required. Count the number of errors in each transmitted block. – Measure the lifetime of a given computer memory chip in a speciﬁed environment. – Pick two numbers between 0 and 1. – Pick a number x between 0 and 1 randomly and then pick a number y between x and 1. – Flip a coin twice. Note the sequence of heads and tails. Note the number of tails.

F An Experiment Outcome • The result of a random experiment is called outcome. An outcome is denoted by lowercase letters x, y, t, ξ, .... When we perform a random experiment one and only one outcome occurs. So outcomes are mutually exclusive, in the sense that they cannot occur simultaneously. F Sample Space • Random experiments do not result in the same outcome. We should determine the set of all possible outcomes. The set of all possible outcomes is called sample space and denoted as S. An outcome ξ is an element of the set S, i.e., by set theory notation we have ξ ∈ S. • We can view the result of a random experiment as a random selection of a single outcome ξ from the set S. EXAMPLE 2

• A hospital administrator codes incoming patients according to whether they have insurance (coding 1 if they do and 0 if they don’t) and their condition (rated as good, fair, serious). Cod- ing the patients is a random experiment with sample space S = {(0, g), (0, f), (0, s), (1, g), (1, f), (1, s)}, ξ = (1, f) ∈ S is an outcome.

Exercise: What are the sample spaces S of the random experiments in Example 1?

F Diﬀerent Types of Sample Space

• A sample space S can be i) finite, ii) countably infinite, iii) uncountably infinite.

• A sample space S is either discrete or continuous.

• A discrete sample space S is either finite or countably infinite. A continuous sample space S is uncountably infinite.

EXAMPLE 3 (diﬀerent types of S)

– finite S, e.g., S = {1, 2, 3, 4, cos(π/6), sin(π/3)}, 6 ∈ S, S = {TT,TH,HT,HH},TH ∈ S – countably infinite S, e.g., S = {0, 1, 2, 3, ...}, 270 ∈ S, S = {T, HT, HHT, HHHT, ...},TH/∈ S, HHHHT ∈ S – uncountably infinite S, e.g., S = {θ : 0 6 θ 6 2π}, π ∈ S, S = {(x, y) : 0 6 x 6 y 6 2π}, (5.4, π) ∈ S (typical examples are: S is the real line, an interval, the plane, a region).

Exercise: Can you think of a random experiment corresponding to the sample spaces in Example 3?

Exercise: Consider the sample spaces of the random experiments in Example 1. Determine the type of each sample space (i.e., whether it is finite, countably infinite, or uncountably infinite).

Page 2 F Events

• An event is a set consisting of outcome(s) and is denoted by capital letters A, B, E, F, .... Any event E is a subset of the sample space S, i.e., E ⊂ S (Recall: in set theory for two sets A and B, we say A ⊂ B if every element of A also belongs to B).

• We say the event E happens if the outcome of the experiment ξ belongs to the set E.

• An event from a discrete sample space S (S is either ﬁnite or countably inﬁnite) may consist of a single outcome. Such an event is called elementary event.

• Two events of special interest are: i) certain event S, this event always occurs, ii) null or impossible event Φ, this event includes no outcome and hence never occurs. Exercise: Consider the the random experiments in Example 1. For each random experiment specify i) the null event, ii) the certain event, iii) an event that is neither certain nor impossible.

EXAMPLE 4 Let consider Example 2 again. Suppose A is the event that the patient is in serious condition ⇒ A = {(0, s), (1, s)}, we have A ⊂ S, (0, s) ∈ A, (0, f) ∈/ A. Suppose B is the event that the patient is uninsured ⇒ B = {(0, g), (0, f), (0, s)}, we have B ⊂ S, (0, f) ∈ B, (1, f) ∈/ B. Suppose C is the event that the patient is uninsured and is in serious condition ⇒ C = {(0, s)}, we have (0, s) ∈ C (C is an elementary event).

F Review of Set Theory

• In probability theory we are interested in determining the probabilities of events. Since each event is represented by a set we need to know the basics of set theory.

• For two events E and F the new event E ∪ F (so-called the union of E and F ) consists of all outcomes that are either in E or in F or in both.

• For two events E and F the new event E ∩ F or EF (so-called the intersection of E and F ) consists of all outcomes that are in both E and F . If E ∩ F = Φ then E and F are said to be mutually exclusive.

• For any event E we deﬁne the new event Ec (so-called the complement of E). The event Ec consists of all outcomes in S that are not in E, i.e., Ec occurs if and only if E does not occur.

• For any event E we have E ∪ Ec = S and E ∩ Ec = Φ.

• For any event E we have (Ec)c = E.

• If E = S then Ec = Φ. Also, if E = Φ then Ec = S.

• For two events E and F we write E ⊂ F if all outcomes in E are also in F .

Page 3 • For two events E and F suppose we have E ⊂ F . Then the occurrence of E necessarily implies the occurrence of F (why? suppose E occurs. This implies that the random experiment outcome belongs to E. Therefore, the outcome belongs to F as well. Hence, F occurs.).

• For two events E and F if E ⊂ F and F ⊂ E then we have E = F , i.e., the two events are equal.

EXAMPLE 5 Let consider Example 4 again. We have C = A ∩ B, {(1, s)} = A ∩ Bc, C ⊂ B, C ⊂ A, B = B ∪ C.

F Venn Diagrams

• When discussing sets, we use Venn diagram to show the relations among sets. S is represented as consisting of all outcomes in a large rectangular, the events F,E,G are represented as consisting of all the outcomes in given circles within the rectangular. Events of interest are indicated by shading appropriate regions of the diagram.

• The operation of forming unions, intersections, and complements of events obey certain rules (laws) similar to the rules of algebra. The rules are:

i) commutative laws E ∪ F = F ∪ E and E ∩ F = F ∩ E ii) associative laws (E ∪ F ) ∪ G = E ∪ (F ∪ G) and (E ∩ F ) ∩ G = E ∩ (F ∩ G) iii) distributive laws (E∪F )∩G = (E∩G)∪(F ∩G) and (E∩F )∪G = (E∪G)∩(F ∪G)

F DeMorgan’s Laws

• DeMorgan’s laws describes a useful relationship between the three basic operations of forming unions, intersections, and complements:

a)(E ∪ F )c = Ec ∩ F c b)(E ∩ F )c = Ec ∪ F c

Proof of a): we show 1) (E ∪ F )c ⊂ Ec ∩ F c and 2) Ec ∩ F c ⊂ (E ∪ F )c. Putting 1) and 2) together we conclude (E ∪ F )c = Ec ∩ F c.

– suppose ξ ∈ (E ∪ F )c ⇒ ξ∈ / E ∪ F ⇒ ξ∈ / E and ξ∈ / F ⇒ ξ ∈ Ec and ξ ∈ F c ⇒ ξ ∈ Ec ∩ F c ⇒ (E ∪ F )c ⊂ Ec ∩ F c – suppose ξ ∈ Ec ∩ F c ⇒ ξ ∈ Ec and ξ ∈ F c ⇒ ξ∈ / E and ξ∈ / F ⇒ ξ∈ / E ∪ F ⇒ ξ ∈ (E ∪ F )c ⇒ Ec ∩ F c ⊂ (E ∪ F )c.

Proof of b): we use rule a) (Ec ∪ F c)c = (Ec)c ∩ (F c)c = E ∩ F . By taking complement we ﬁnd (E ∩ F )c = Ec ∪ F c.

Page 4