RS – Chapter 2 – Random Variables 8/14/2019

Chapter 2 Random Variables

Random Variables

• A random variable is a convenient way to express the elements of Ω as numbers rather than abstract elements of sets.

Definition: Let AX; AY be nonempty families of subsets of X and -1 Y, respectively. A function f: X → Y is (AX;AY )-measurable if f A ∈ AX for all A ∈ AY.

Definition: Random Variable •A random variable X is a measurable function from the probability space (Ω, Σ ,P) into the probability space (χ, AX, PX), where χ in R is the range of X (which is a subset of the real line) AX is a Borel field of X, and PX is the probability measure on χ induced by X. Specifically, X: Ω→ χ.

1 RS – Chapter 2 – Random Variables 8/14/2019

Random Variables - Remarks

Remarks: - A random variable X is a function. - It is a numerical quantity whose value is determined by a random experiment. - It takes single elements in Ω and maps them to single points in R.

- P is the probability measure over the sample space and PX is the probability measure over the range of the random variable.

- The induced measure PX is just a way of relating measure on the real line –the range of X– back to the original probability measure over the abstract events in the σ-algebra of the sample space.

Random Variables - Interpretation

Interpretation

The induced measure PX allows us to relate a measure on the real line –the range of X– back to the original probability measure over the abstract events in the σ-algebra of the sample space: -1 PX[A] = P [X (A)] =P[{ω ∈ Ω: X(ω) ∈ A}.

That is, we take the probability weights associated with events and assign them to real numbers. Recall that when we deal with probabilities on some random variable X, we are really dealing with

the PX measure.

We measure the "size" of the set (using P as our measure) of ω's such that the random variable X returns values in A.

2 RS – Chapter 2 – Random Variables 8/14/2019

Random Variables - Interpretation

Interpretation

P is the probability measure over the sample space and PX is the probability measure over the range of the random variable.

Thus, we write P[A] (where A is a subset of the range of X) but we

mean PX[A], which is equivalent to P[{ω ∈ Ω: X(ω) ∈ A}].

Notational shortcut: We use P[A] instead of PX [A] (This notation can be misleading if there's confusion about whether A is in the sample space or in the range of X.)

Random Variables – Example 1

Example: Back to the previous example where two coins are tossed. We defined the sample space (Ω) as all possible outcomes and the sigma algebra of all possible subsets of the sample space. A simple probability measure (P) was applied to the events in the sigma algebra.

Let the random variable X be "number of heads." Recall that X

takes Ω into χ and induces PX from P. In this example, χ = {0; 1; 2} and A = {Φ; {0}; {1}; {2}; {0;1}; {0;2}; {1;2}; {0;1;2}}.

The induced probability measure PX from the measure defined above would look like:

3 RS – Chapter 2 – Random Variables 8/14/2019

Random Variables – Example 1

Example: Back to the previous example where two coins are tossed.

Prob. of 0 heads = PX[0] = P[{TT}] = 1/4

Prob. of 1 heads = PX[1] = P[{HT; TH}] = 1/2

Prob. of 2 heads = PX[2] = P[{HH}] = ¼

Prob. of 0 or 1 heads = PX[{0; 1}] = P[{TT; TH; HT}] = 3/4

Prob. of 0 or 2 heads = PX[{0; 2}] = P[{TT; HH}] = 1/2

Prob. of 1 or 2 heads = PX[{1; 2}] = P[{TH; HT; HH}] = 3/4

Prob. of 1, 2, or 3 heads = PX[{0; 1; 2}] = P[{HH; TH; HT; TT}] = 1

Prob. of "nothing" = PX[Φ] = P[Φ] = 0

The empty set is simply needed to complete the σ-algebra. Its interpretation is not important since P[Φ] = 0 for any reasonable P.

Random Variables – Example 2

Example: Probability Space One standard probability space is the Borel field over the unit interval of the real line under the Lebesgue measure λ. That is ([0, 1]; B; λ).

The Borel field over the unit interval gives us a set of all possible intervals taken from [0,1]. The Lebesgue measure measures the size of any given interval. For any interval [a; b] in [0,1] with b ≥ a, λ[[a, b]] = b – a.

This probability space is well known: “uniform distribution,” the probability of any interval of values is the size of the interval.

4 RS – Chapter 2 – Random Variables 8/14/2019

Random Variables – More Examples

1. Two dice are rolled and X is the sum of the two upward faces. 2. A coin is tossed n = 3 times and X is the number of times that a head occurs. 3. A point is selected at random from a square whose sides are of length 1. X is the distance of the point from the lower left hand corner. point

X

4.Xis the number of times the price of IBM increases during a time interval, say a day. 5. Today, the DJ Index is 19,504.17, X is the value of the index in thirty days.

Random Variables: Summary

Ω is the sample space –the set of possible outcomes from an experiment. – An event A is a set containing outcomes from the sample space.

• Σ is a σ -algebra of subsets of the sample space. Think of Σ as the collection of all possible events involving outcomes chosen from Ω.

• P is a probability measure over Σ. P assigns a number between [0,1] to each event in Σ.

• We have functions (random variables) that allow us to look at real numbers instead of abstract events in Σ.

5 RS – Chapter 2 – Random Variables 8/14/2019

Random Variables: Summary

• For each random variable X, there exists a new probability measure

PX: PX [A] where A ∈ R simply relates back to P[{ω ∈ Ω: X(ω) ∈A}.

We calculate PX[A], but we are really interested in the probability P[{ω ∈ Ω: X(ω) ∈ A}, where A simply represents {ω ∈ Ω: X(ω) ∈ A} through the inverse transformation X-1.

Random Variables: Probability Function & CDF Definition – The probability function, p(x), of a RV, X. For any random variable, X, and any real number, x, we define px PX x P X x where {X = x} = the set of all outcomes (event) with X = x.

Definition – The cumulative distribution function (CDF), F(x), of a RV, X. For any random variable, X, and any real number, x, we define

F xPXxPXx    where {X ≤ x} = the set of all outcomes (event) with X ≤ x.

6 RS – Chapter 2 – Random Variables 8/14/2019

Probability Function & CDF: Example I

• Two dice are rolled and X is the sum of the two upward faces. Sample space S = { 2:(1,1), 3:(1,2; 2,1), 4:(1,3; 3,1; 2,2), 5:(1,4; 2,3; 3,2; 4,1), 6, 7, 8, 9, 10, 11, 12}. Graph: Probability function:

0.18 p(x)

0.12

0.06

0.00 2 3 4 5 6 7 8 9 10 11 12 x

Probability Function & CDF: Example I

Probability function: 1 pPXP221,1  36 2 pPXP331,2,2,1  36 3 pPXP441,3,2,2,3,1  36 45654 ppppp5,6,7,8,9    36 36 36 36 36 321 ppp10 , 11 , 12 36 36 36 and p xx 0 for all other

Note: Xx for all other x

7 RS – Chapter 2 – Random Variables 8/14/2019

Probability Function & CDF: Example I

Graph: CDF

 0 x  2 1.2  1 23x  36 1 3  36 34x  6 0.8  36 45x  10 56x 0.6  36 15  36 67x 0.4 Fx   21 78x  36 0.2  26 36 89x  0 30 910x  36 0510  33 10x 11  36 35 F(x) is a step function  36 11x 12   1 12  x

Probability Function & CDF: Example II

• A point is selected at random from a square whose sides are of length 1. X is the distance of the point from the lower left-hand corner.

point

X

An event, E, is any subset of the square, S. S

P[E] = (area of E)/(Area of S) = area of E E

8 RS – Chapter 2 – Random Variables 8/14/2019

Probability Function & CDF: Example II

The probability function is given by:

set of all points a dist x px PX x P 0 from lower left corner S

Thus p(x) = 0 for all values of x. The probability function for this example is not very informative.

Probability Function & CDF: Example II

The Cumulative distribution function is given by:  set of all points within a   Fx PX x P     dist x from lower left corner 

S x x x

01x 12x 2  x

9 RS – Chapter 2 – Random Variables 8/14/2019

Probability Function & CDF: Example II

 0 x  0  2  x 01x  4 Fx PX x   Area Ax 1 2   1 2  x

S A x x x

01x 12x 2  x

Probability Function & CDF: Example II • Computation of area A: 12x x2 1  1    2 x 2 A 2 tan  x  1 x x2 1     tan12 x  1  1

11x2    A 21 xx22 x 2  22 2

222122   x 11tan1 xx   x   x 44   

10 RS – Chapter 2 – Random Variables 8/14/2019

Probability Function & CDF: Example II

 00x    x2  01 x   4 Fx PX x  2122   xxxx1tan1   12    4     12 x

1 F  x

0 -1 0 1 2

Random Variables: PDF for a Continuous RV

Definition: Suppose that X is a random variable. Let f(x) denote a function defined for -∞ < x < ∞ with the following properties:

1. f(x) ≥ 0  2.  fxdx  1.  b 3. PaXb fxdx . a

Then f(x) is called the probability density function of X.. The random variable X is called continuous.

11 RS – Chapter 2 – Random Variables 8/14/2019

Random Variables: PDF for a Continuous RV

  fxdx 1. 

b PaXb fxdx . a

Random Variables: PDF for a Continuous RV

x F xPXx ftdt . 

Fx 

12 RS – Chapter 2 – Random Variables 8/14/2019

CDF and PDF for a Continuous RV: Relation

Thus if X is a continuous random variable with probability density function, f(x), the cumulative distribution function of X is given by: x F xPXx ftdt . 

Also because of the FTC (fundamental theorem of calculus):

dF x Fxfx dx

CDF and PDF for a Continuous RV: Relation

Example: Deriving a pdf from a CDF A point is selected at random from a square whose sides are of length 1. X is the distance of the point from the lower left hand corner. point

X

 0  2 x  0   x  4 01x Fx PX x   2122  1x 2 xxx1tan1    4 2  x   1

13 RS – Chapter 2 – Random Variables 8/14/2019

CDF and PDF for a Continuous RV: Relation

Now

   00x  or 2 x   x fx Fx   01  x  2

 d 2122   xxxx1tan1   12   dx 4

CDF and PDF for a Continuous RV: Relation

Also

d 2122  x 1tan1 xx  dx 4   3  1 2 2  2 x 12xx  2 d 2tanxxx12 1 2 tan 12 x 1   dx   

 x  12 xxx3 2tan  1 2   2 x 2  1 d xx212 tan 1  dx   

14 RS – Chapter 2 – Random Variables 8/14/2019

CDF and PDF for a Continuous RV: Relation

d 1 Now tan  1 u  du1  u 2

 3 d 11 2 tan 12x  1 xx 2  1 2   2  dx 11x 2

212dx xxtan 1 3   2 dx x 2  1 and

d 2122  x 1tan1 xx  dx 4    xx2tan 12 x  1 2  

CDF and PDF for a Continuous RV: Relation

Finally

  00x  or 2 x    x fx Fx   01  x  2  xx2tan12 x 1 1 x 2  2  

15 RS – Chapter 2 – Random Variables 8/14/2019

CDF and PDF for a Continuous RV: Relation Graph of f(x)

2

1.5

1

0.5

0 -1 0 1 2

Discrete Random Variables

A random variable X is called discrete if  px px i  1 xi1 All the probability is accounted for by values, x, such that p(x) > 0.

For a discrete random variable X the probability distribution is described by the probability function p(x), which has the following properties: 1. 0 px   1  2. px px i  1 xi1 3. Pa  x b  px  axb

16 RS – Chapter 2 – Random Variables 8/14/2019

Discrete Random Variables: Graph

Pa  x b  px  axb p(x)

a b

Discrete Random Variables: Details

Recall p(x) = P[X = x] = the probability function of X. This can be defined for any random variable X.

For a continuous random variable p(x) = 0 for all values of X.

Let SX ={x| p(x) > 0}. This set is countable -i. e., it can be put into a 1-1 correspondence with the integers.

SX = {x| p(x) > 0} = {x1, x2, x3, x4, …}

 Thus, we can write px px i  xi1

17 RS – Chapter 2 – Random Variables 8/14/2019

Discrete Random Variables: Details

Thus the elements of SX = S1  S2  S3  S3  …

can be arranged {x1, x2, x3, x4, … }

by choosing the first elements to be the elements of S1 ,

the next elements to be the elements of S2 ,

the next elements to be the elements of S3 ,

the next elements to be the elements of S4 , etc

This allows us to write

 px for px i  xi1

Discrete & Continuous Random Variables

A Probability distribution is similar to a distribution of mass. A Discrete distribution is similar to a point distribution of mass.  Positive amounts of mass are put at discrete points.

p(x ) p(x ) p(x ) 1 2 p(x3) 4

x1 x2 x3 x4

18 RS – Chapter 2 – Random Variables 8/14/2019

Discrete & Continuous Random Variables

• A Continuous distribution is similar to a continuous distribution of mass.

The total mass of 1 is spread over a continuum. The mass assigned to any point is zero but has a non-zero density

f(x)

Distribution function F(x): Properties

• This is defined for any random variable, X: F(x) = P[X ≤ x]

Properties 1. F(-∞) = 0 and F(∞) = 1. Since {X ≤ - ∞}=  and {X ≤ ∞} =S  F(- ∞) = 0 and F(∞) = 1.

2. F(x) is non-decreasing (i. e., if x1 < x2  F(x1) ≤ F(x2) )

If x1 < x2 then {X ≤ x2} = {X ≤ x1}  {x1 < X ≤ x2} Thus,

P[X ≤ x2] = P[X ≤ x1] + P[x1 < X ≤ x2]

or F(x2) = F(x1) + P[x1 < X ≤ x2]

Since P[x1 < X ≤ x2] ≥ 0  F(x2) ≥ F(x1).

19 RS – Chapter 2 – Random Variables 8/14/2019

Distribution function F(x): Properties

3. F(b) – F(a) = P[a < X ≤ b]. If a < b then using the argument above F(b) = F(a) + P[a < X ≤ b]  F(b) – F(a) = P[a < X ≤ b].

4. p(x) = P[X = x] =F(x) – F(x-) Here, F xFulim   ux 5. If p(x) = 0 for all x (i.e., X is continuous) then F(x) is continuous. A function F is continuous if F xFuFxFulim    lim   ux ux  One can show that p(x) = 0 implies F  xFxFx    

Distribution function F(x): Discrete RV

Fx PX x pu  ux F(x) is a non-decreasing step function with FF 0 and    1 px Fx  Fx  jump in Fx  at x .

F(x) 1.2 1

0.8

0.6

0.4

0.2 p(x)

0 -101234

20 RS – Chapter 2 – Random Variables 8/14/2019

Distribution function F(x): Continuous RV

x Fx PX x fudu  F(x) is a non-decreasing continuous function with FF 0 and    1 f xFx  . f(x) slope F(x)

1

0 -1 0 1x 2

Some Important Discrete Distributions

21 RS – Chapter 2 – Random Variables 8/14/2019

The Binomial distribution

Jacob Bernoulli (1654– 1705)

Bernouille Distribution Suppose that we have a Bernoulli trial (an experiment) that has 2 results: 1. Success (S) 2. Failure (F)

Suppose that p is the probability of success (S) and q = 1 – p is the probability of failure (F). Then, the probability distribution with probability function  qx 0 px PX x   px 1 is called the Bernoulli distribution.

• Now assume that the Bernoulli trial is repeated independently n times. Let X be the number of successes ocurring in the n trials. (The possible values of X are {0, 1, 2, …, n})

22 RS – Chapter 2 – Random Variables 8/14/2019

The Binomial Distribution Suppose we have n = 5 the outcomes together with the values of X and the probabilities of each outcome are given in the table below:

FFFFF SFFFF FSFFF FFSFF FFFSF FFFFS SSFFF SFSFF 0 1 1 1 1 1 2 2 q5 pq4 pq4 pq4 pq4 pq4 p2q3 p2q3 SFFSF SFFFS FSSFF FSFSF FSFFS FFSSF FFSFS FFFSS 2 2 2 2 2 2 2 2 p2q3 p2q3 p2q3 p2q3 p2q3 p2q3 p2q3 p2q3 SSSFF SSFSF SSFFS SFSSF SFSFS SFFSS FSSSF FSSFS 3 3 3 3 3 3 3 3 p3q2 p3q2 p3q2 p3q2 p3q2 p3q2 p3q2 p3q2 FSFSS FFSSS SSSSF SSSFS SSFSS SFSSS FSSSS SSSSS 3 3 4 4 4 4 4 5 p3q2 p3q2 p4q p4q p4q p4q p4q p5

The Binomial Distribution For n = 5 the following table gives the different possible values of X, x, and p(x) = P[X = x] x 012345 p(x) = P[X = x] q5 5pq4 10p3q2 10p2q3 5p4qp5

• For general n, the outcome of the sequence of n Bernoulli trials is a sequence of S’s and F’s of length n: SSFSFFSFFF…FSSSFFSFSFFS • The value of X for such a sequence is k = the number of S’s in the sequence. • The probability of such a sequence is pkqn – k ( a p for each S and a q for each F) n • There are  such sequences containing exactly k S’s k

23 RS – Chapter 2 – Random Variables 8/14/2019

The Binomial Distribution n  k is the number of ways of selecting the k positions for the S’s (the remaining n – k positions are for the F’s). Thus,

n knk pk PX k pq k  0,1,2,3,K , n  1, n k These are the terms in the expansion of (p + q)n using the Binomial Theorem

n nn011220nn   n n  n n pq pq   pq   pq K  pq 01   2 n For this reason the probability function

n xnx p xPXx pqx  0,1,2,K , n x is called the probability function for the Binomial distribution

The Binomial Distribution

Summary We observe a Bernoulli trial (S,F) n times. Let X denote the number of successes in the n trials. Then, X has a binomial distribution:

n xnx px PX x pq x  0,1,2,K , n x where 1. p = the probability of success (S), and 2. q = 1 – p = the probability of failure (F)

24 RS – Chapter 2 – Random Variables 8/14/2019

The Binomial Distribution

Example: If a firm announces profits and they are “surprising,” the chance of a stock price increase is 85%. Assume there are n=20 (independent) announcements. Let X denote the number of increases in the stock price following surprising announcements in the n = 20 trials. Then, X has a binomial distribution, with p = 0.85 and n = 20.

Thus,

n xnx p xPXx pqx  0,1,2,K⋯ , n x

20 xx20 .85 .15 x 0,1,2,K⋯ ,20 x

x 0 12345 p (x ) 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 x 6 7 8 9 10 11 p (x ) 0.0000 0.0000 0.0000 0.0000 0.0002 0.0011 x 12 13 14 15 16 17 p (x ) 0.0046 0.0160 0.0454 0.1028 0.1821 0.2428 x 18 19 20 p (x ) 0.2293 0.1368 0.0388 0.3000

0.2500 p(x) 0.2000

0.1500

0.1000

0.0500

- 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 x

25 RS – Chapter 2 – Random Variables 8/14/2019

The Poisson distribution

Siméon Denis Poisson (1781-1840)

The Poisson distribution

• Suppose events are occurring randomly and uniformly in time. • The events occur with a known average. • Let X be the number of events occurring (arrivals) in a fixed period of time (time-interval of given length). • Typical example: X = Number of crime cases coming before a criminal court per year (original Poisson’s application in 1838.)

• Then, X will have a Poisson distribution with parameter .

x    px e x 0,1, 2,3, 4,K⋯ x! • The parameter λ represents the expected number of occurrences in a fixed period of time. The parameter λ is a positive real number.

26 RS – Chapter 2 – Random Variables 8/14/2019

The Poisson distribution

• Example: On average, a trade occurs every 15 seconds. Suppose trades are independent. We are interested in the probability of observing 10 trades in a minute (X=10). A Poisson distribution can be used with λ = 4 (4 trades per minute).

• Poisson probability function

Properties:  x 1. px e  1 xx00x !

 x 234   ee1  K⋯ x0 x!2!3!4!  x 234  Thus  ee1  K⋯  ee1 x0 x!2!3!4! 234 u uuu using eu 1 K⋯ 2! 3! 4!

n x nx 2.If pxpnBin ,1 p p x is the probability function for the Binomial distribution with parameters n and p. Let n → ∞ and p → 0 such that np = a constant (=λ, say) then x    limpxpnpBin ,  Poisson  x e np,0  x !

27 RS – Chapter 2 – Random Variables 8/14/2019

n x nx Proof: pxpnBin ,1 p p x  Suppose np or p n

x nx n!   pxpnpxnBin,, Bin   1   xn!! x n n  x n  x n!  x 11 x!!nnx  n n x n  x nn 11L⋯  n x   11 x! nn L⋯ n n n x n  x 11x   11L⋯ 1   1  1  x! nnnn 

Now limp Bin xn , n 

 x n x 11x   lim 1L⋯ 1  1 1 xnnnn! n   n  x  lim 1 x ! n  n n u u Using the classic limit lim 1   e n  n

xxn    limpBinxn , lim 1 e p Poisson  x nnxnx!!  

Note: In many applications, when n is large and p is very small --and the expectation np is not big. Then, the binomial distribution may be approximated by the easier Poisson distribution. This is called the law of rare events, since each of the n individual Bernoulli events rarely occurs.

28 RS – Chapter 2 – Random Variables 8/14/2019

The Poisson distribution: Graphical Illustration

• Suppose a time interval is divided into n equal parts and that one event may or may not occur in each subinterval. n subintervals

time interval - Event occurs X = # of events is Bin(n,p) - Event does not occur As n → ∞ , events can occur over the continuous time interval.

X = # of events is Poisson()

The Poisson distribution: Comments

• The Poisson distribution arises in connection with Poisson processes - a stochastic process in which events occur continuously and independently of one another. • It occurs most easily for time-events; such as the number of calls passing through a call center per minute, or the number of visitors passing through a turnstile per hour. However, it can apply to any process in which the mean can be shown to be constant. • It is used in finance (number of jumps in an asset price in a given interval); market microstructure (number of trades per unit of time in a stock market); sports economics (number of goals in sports involving two competing teams); insurance (number of a given disaster -volcano eruptions/hurricanes/floods- per year); etc.

29 RS – Chapter 2 – Random Variables 8/14/2019

Poisson Distribution - Example: Tropical Storms

The number of named storms over a period of a year in the Caribbean is known to have a Poisson distribution with  = 13.1

Determine the probability function of X. Compute the probability that X is at most 8. Compute the probability that X is at least 10. Given that at least 10 hurricanes occur, what is the probability that X is at most 15?

x   Solution: px e x 0,1,2,3,4,K⋯ x! x 13.1 13.1 ex 0,1,2,3, 4,K⋯ x!

Poisson Distribution - Example: Storms

Table of p(x)

x p (x ) x p (x ) 0 0.000002 10 0.083887 1 0.000027 11 0.099901 2 0.000175 12 0.109059 3 0.000766 13 0.109898 4 0.002510 14 0.102833 5 0.006575 15 0.089807 6 0.014356 16 0.073530 7 0.026866 17 0.056661 8 0.043994 18 0.041237 9 0.064036 19 0.028432

30 RS – Chapter 2 – Random Variables 8/14/2019

Poisson Distribution - Example: Storms PPXat most 8  8 pp0  1 L p 8 .09527

PPXPXat least 10  10 1 9 101 pp    L  p 9.8400 

PPXXat most 15 at least 10 15 10

PX 15  X 10 PX10 15  PX10 PX 10 pp10  11 L p 15 0.708 .8400

31