Stochastic Simulation APPM 7400
Lesson 5: Generating (Some) Continuous Random Variables
September 12, 2018
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 1 / 1 Furthermore, suppose that you can invert F .
If U ∼ unif (0, 1), the random variable
F −1(U)
has the same distribution as X !
Generating Continuous Random Variables
Suppose that X is a continuous random variable with cdf F (x).
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 2 / 1 If U ∼ unif (0, 1), the random variable
F −1(U)
has the same distribution as X !
Generating Continuous Random Variables
Suppose that X is a continuous random variable with cdf F (x).
Furthermore, suppose that you can invert F .
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 2 / 1 Generating Continuous Random Variables
Suppose that X is a continuous random variable with cdf F (x).
Furthermore, suppose that you can invert F .
If U ∼ unif (0, 1), the random variable
F −1(U)
has the same distribution as X !
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 2 / 1 = P(U ≤ F (x)) = F (x) = P(X ≤ x)
F non-decreasing ⇓ cdf of unif(0,1) definition of F F −1 non-decreasing
Generating Continuous Random Variables
Proof:
P(F −1(U) ≤ x)
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 3 / 1 = F (x) = P(X ≤ x)
cdf of unif(0,1) definition of F
Generating Continuous Random Variables
Proof:
P(F −1(U) ≤ x) = P(U ≤ F (x))
F non-decreasing ⇓
F −1 non-decreasing
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 3 / 1 = P(X ≤ x)
definition of F
Generating Continuous Random Variables
Proof:
P(F −1(U) ≤ x) = P(U ≤ F (x)) = F (x)
F non-decreasing ⇓ cdf of unif(0,1) F −1 non-decreasing
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 3 / 1 Generating Continuous Random Variables
Proof:
P(F −1(U) ≤ x) = P(U ≤ F (x)) = F (x) = P(X ≤ x)
F non-decreasing ⇓ cdf of unif(0,1) definition of F F −1 non-decreasing
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 3 / 1 ⇓
R x −λu −λx cdf: F (x) = 0 λe du = 1 − e , x ≥ 0
⇓
−1 1 Now plug in F (x) = − λ ln(1 − x) a uniform!
Generating Continuous Random Variables
Example: Generating exponential rate λ r.v.’s:
pdf: f (x) = λe−λx , x ≥ 0
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 4 / 1 ⇓
−1 1 Now plug in F (x) = − λ ln(1 − x) a uniform!
Generating Continuous Random Variables
Example: Generating exponential rate λ r.v.’s:
pdf: f (x) = λe−λx , x ≥ 0
⇓
R x −λu −λx cdf: F (x) = 0 λe du = 1 − e , x ≥ 0
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 4 / 1 Now plug in a uniform!
Generating Continuous Random Variables
Example: Generating exponential rate λ r.v.’s:
pdf: f (x) = λe−λx , x ≥ 0
⇓
R x −λu −λx cdf: F (x) = 0 λe du = 1 − e , x ≥ 0
⇓
−1 1 F (x) = − λ ln(1 − x)
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 4 / 1 Generating Continuous Random Variables
Example: Generating exponential rate λ r.v.’s:
pdf: f (x) = λe−λx , x ≥ 0
⇓
R x −λu −λx cdf: F (x) = 0 λe du = 1 − e , x ≥ 0
⇓
−1 1 Now plug in F (x) = − λ ln(1 − x) a uniform!
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 4 / 1 Generating Continuous Random Variables
Results: (100,000 values, λ = 3.5) Density 0.0 0.5 1.0 1.5 2.0 2.5 3.0
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5
sample
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 5 / 1 There is no closed form expression for the cdf!
Generating Continuous Random Variables
The Standard Normal Distribution:
1 − 1 x2 f (x) = √ e 2 , −∞ < x < ∞ 2π
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 6 / 1 Generating Continuous Random Variables
The Standard Normal Distribution:
1 − 1 x2 f (x) = √ e 2 , −∞ < x < ∞ 2π
There is no closed form expression for the cdf!
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 6 / 1 Then p X1 = −2 ln U1 cos(2πU2) is normally distributed with mean0 and variance1.
So is p X2 = −2 ln U1 sin(2πU2)
and X1 and X2 are independent!
Generating Continuous Random Variables
The Box-Muller Transformation:
Let U1 and U2 be independent unif (0, 1) r.v.’s.
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 7 / 1 So is p X2 = −2 ln U1 sin(2πU2)
and X1 and X2 are independent!
Generating Continuous Random Variables
The Box-Muller Transformation:
Let U1 and U2 be independent unif (0, 1) r.v.’s. Then p X1 = −2 ln U1 cos(2πU2) is normally distributed with mean0 and variance1.
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 7 / 1 and X1 and X2 are independent!
Generating Continuous Random Variables
The Box-Muller Transformation:
Let U1 and U2 be independent unif (0, 1) r.v.’s. Then p X1 = −2 ln U1 cos(2πU2) is normally distributed with mean0 and variance1.
So is p X2 = −2 ln U1 sin(2πU2)
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 7 / 1 Generating Continuous Random Variables
The Box-Muller Transformation:
Let U1 and U2 be independent unif (0, 1) r.v.’s. Then p X1 = −2 ln U1 cos(2πU2) is normally distributed with mean0 and variance1.
So is p X2 = −2 ln U1 sin(2πU2)
and X1 and X2 are independent!
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 7 / 1 If neither of these apply to your simulation problem, move on to the next algorithm. It requires that you be able to uniformly bound the “target pdf” by an integrable function. If you can’t do that, move on to the next algorithm... there is no “catch-all” procedure so we’d better build up a big arsenal of simulation techniques!
Generating Continuous Random Variables
So far, we have ways to simulate normal random variables random variables for which we have a closed-form invertible cdf
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 8 / 1 It requires that you be able to uniformly bound the “target pdf” by an integrable function. If you can’t do that, move on to the next algorithm... there is no “catch-all” procedure so we’d better build up a big arsenal of simulation techniques!
Generating Continuous Random Variables
So far, we have ways to simulate normal random variables random variables for which we have a closed-form invertible cdf
If neither of these apply to your simulation problem, move on to the next algorithm.
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 8 / 1 If you can’t do that, move on to the next algorithm... there is no “catch-all” procedure so we’d better build up a big arsenal of simulation techniques!
Generating Continuous Random Variables
So far, we have ways to simulate normal random variables random variables for which we have a closed-form invertible cdf
If neither of these apply to your simulation problem, move on to the next algorithm. It requires that you be able to uniformly bound the “target pdf” by an integrable function.
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 8 / 1 Generating Continuous Random Variables
So far, we have ways to simulate normal random variables random variables for which we have a closed-form invertible cdf
If neither of these apply to your simulation problem, move on to the next algorithm. It requires that you be able to uniformly bound the “target pdf” by an integrable function. If you can’t do that, move on to the next algorithm... there is no “catch-all” procedure so we’d better build up a big arsenal of simulation techniques!
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 8 / 1 want to simulate a r.v. X with pdf f need to find another function g so that
g(x) ≥ f (x) ∀x
normalize g to a pdf 1 h(x) = g(x) c where c = R g(x) dx
Generating Continuous Random Variables
Accept/Reject Algorithm: (due to von Neumann)
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 9 / 1 need to find another function g so that
g(x) ≥ f (x) ∀x
normalize g to a pdf 1 h(x) = g(x) c where c = R g(x) dx
Generating Continuous Random Variables
Accept/Reject Algorithm: (due to von Neumann) want to simulate a r.v. X with pdf f
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 9 / 1 normalize g to a pdf 1 h(x) = g(x) c where c = R g(x) dx
Generating Continuous Random Variables
Accept/Reject Algorithm: (due to von Neumann) want to simulate a r.v. X with pdf f need to find another function g so that
g(x) ≥ f (x) ∀x
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 9 / 1 Generating Continuous Random Variables
Accept/Reject Algorithm: (due to von Neumann) want to simulate a r.v. X with pdf f need to find another function g so that
g(x) ≥ f (x) ∀x
normalize g to a pdf 1 h(x) = g(x) c where c = R g(x) dx
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 9 / 1 generate Y with pdf h
generate U ∼ unif (0, 1) (indep of Y )
if U ≤ f (Y )/g(Y ), “accept” Y , set X = Y (Done!)
otherwise, “reject” Y , return to
Accept/Reject Algorithm
The Algorithm:
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 10 / 1 generate U ∼ unif (0, 1) (indep of Y )
if U ≤ f (Y )/g(Y ), “accept” Y , set X = Y (Done!)
otherwise, “reject” Y , return to
Accept/Reject Algorithm
The Algorithm:
generate Y with pdf h
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 10 / 1 if U ≤ f (Y )/g(Y ), “accept” Y , set X = Y (Done!)
otherwise, “reject” Y , return to
Accept/Reject Algorithm
The Algorithm:
generate Y with pdf h
generate U ∼ unif (0, 1) (indep of Y )
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 10 / 1 otherwise, “reject” Y , return to
Accept/Reject Algorithm
The Algorithm:
generate Y with pdf h
generate U ∼ unif (0, 1) (indep of Y )
if U ≤ f (Y )/g(Y ), “accept” Y , set X = Y (Done!)
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 10 / 1 Accept/Reject Algorithm
The Algorithm:
generate Y with pdf h
generate U ∼ unif (0, 1) (indep of Y )
if U ≤ f (Y )/g(Y ), “accept” Y , set X = Y (Done!)
otherwise, “reject” Y , return to
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 10 / 1 Let X be the value produced by this algorithm We will show that P(X = x) = f (x) where f is the target density.
Let N be the number of trials until a success? Q: Is N a geometric random variable? A: Yes! Success probability?
Accept/Reject Algorithm
Proof: (discrete case)
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 11 / 1 We will show that P(X = x) = f (x) where f is the target density.
Let N be the number of trials until a success? Q: Is N a geometric random variable? A: Yes! Success probability?
Accept/Reject Algorithm
Proof: (discrete case)
Let X be the value produced by this algorithm
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 11 / 1 Let N be the number of trials until a success? Q: Is N a geometric random variable? A: Yes! Success probability?
Accept/Reject Algorithm
Proof: (discrete case)
Let X be the value produced by this algorithm We will show that P(X = x) = f (x) where f is the target density.
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 11 / 1 Q: Is N a geometric random variable? A: Yes! Success probability?
Accept/Reject Algorithm
Proof: (discrete case)
Let X be the value produced by this algorithm We will show that P(X = x) = f (x) where f is the target density.
Let N be the number of trials until a success?
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 11 / 1 A: Yes! Success probability?
Accept/Reject Algorithm
Proof: (discrete case)
Let X be the value produced by this algorithm We will show that P(X = x) = f (x) where f is the target density.
Let N be the number of trials until a success? Q: Is N a geometric random variable?
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 11 / 1 Accept/Reject Algorithm
Proof: (discrete case)
Let X be the value produced by this algorithm We will show that P(X = x) = f (x) where f is the target density.
Let N be the number of trials until a success? Q: Is N a geometric random variable? A: Yes! Success probability?
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 11 / 1 Accept/Reject Algorithm
Proof: (discrete case)
Consider the joint probability
P(X = x, N = n)
We will write down an expression for this and then marginalize out the N.
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 12 / 1 indep f (x) = P(n − 1 failures) · P Yn = x, Un ≤ g(x)
ident f (x) = P(n − 1 failures) · P Y = x, U ≤ g(x)
Accept/Reject Algorithm
P(X = x, N = n) = P (n − 1 failures, f (yn−1) f (x) Un−1 > , Yn = x, Un ≤ g(yn−1) g(x)
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 13 / 1 ident f (x) = P(n − 1 failures) · P Y = x, U ≤ g(x)
Accept/Reject Algorithm
P(X = x, N = n) = P (n − 1 failures, f (yn−1) f (x) Un−1 > , Yn = x, Un ≤ g(yn−1) g(x)
indep f (x) = P(n − 1 failures) · P Yn = x, Un ≤ g(x)
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 13 / 1 Accept/Reject Algorithm
P(X = x, N = n) = P (n − 1 failures, f (yn−1) f (x) Un−1 > , Yn = x, Un ≤ g(yn−1) g(x)
indep f (x) = P(n − 1 failures) · P Yn = x, Un ≤ g(x)
ident f (x) = P(n − 1 failures) · P Y = x, U ≤ g(x)
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 13 / 1 f (x) P∞ = P Y = x, U ≤ g(x) · n=1 P(n − 1 failures)
f (x) = P Y = x, U ≤ g(x) · [blah (no x’s)]
indep f (x) = P(Y = x) · P U ≤ g(x) · [blah (no x’s)]
f (x) = h(x) · g(x) · [blah (no x’s)]
(since f (x) ≤ g(x))
Accept/Reject Algorithm
So, P∞ P(X = x) = n=1 P(X = x, N = n)
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 14 / 1 f (x) = P Y = x, U ≤ g(x) · [blah (no x’s)]
indep f (x) = P(Y = x) · P U ≤ g(x) · [blah (no x’s)]
f (x) = h(x) · g(x) · [blah (no x’s)]
(since f (x) ≤ g(x))
Accept/Reject Algorithm
So, P∞ P(X = x) = n=1 P(X = x, N = n)
f (x) P∞ = P Y = x, U ≤ g(x) · n=1 P(n − 1 failures)
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 14 / 1 indep f (x) = P(Y = x) · P U ≤ g(x) · [blah (no x’s)]
f (x) = h(x) · g(x) · [blah (no x’s)]
(since f (x) ≤ g(x))
Accept/Reject Algorithm
So, P∞ P(X = x) = n=1 P(X = x, N = n)
f (x) P∞ = P Y = x, U ≤ g(x) · n=1 P(n − 1 failures)
f (x) = P Y = x, U ≤ g(x) · [blah (no x’s)]
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 14 / 1 f (x) = h(x) · g(x) · [blah (no x’s)]
(since f (x) ≤ g(x))
Accept/Reject Algorithm
So, P∞ P(X = x) = n=1 P(X = x, N = n)
f (x) P∞ = P Y = x, U ≤ g(x) · n=1 P(n − 1 failures)
f (x) = P Y = x, U ≤ g(x) · [blah (no x’s)]
indep f (x) = P(Y = x) · P U ≤ g(x) · [blah (no x’s)]
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 14 / 1 Accept/Reject Algorithm
So, P∞ P(X = x) = n=1 P(X = x, N = n)
f (x) P∞ = P Y = x, U ≤ g(x) · n=1 P(n − 1 failures)
f (x) = P Y = x, U ≤ g(x) · [blah (no x’s)]
indep f (x) = P(Y = x) · P U ≤ g(x) · [blah (no x’s)]
f (x) = h(x) · g(x) · [blah (no x’s)]
(since f (x) ≤ g(x))
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 14 / 1 1 f (x) = c g(x) · g(x) · [blah (no x’s)]
1 = c f (x) · [blah (no x’s)]
Sum both sides over x:
X X 1 P(X = x) = f (x) · [blah (no x’s)] c x x
Accept/Reject Algorithm
So, f (x) P(X = x) = h(x) · g(x) · [blah (no x’s)]
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 15 / 1 1 = c f (x) · [blah (no x’s)]
Sum both sides over x:
X X 1 P(X = x) = f (x) · [blah (no x’s)] c x x
Accept/Reject Algorithm
So, f (x) P(X = x) = h(x) · g(x) · [blah (no x’s)]
1 f (x) = c g(x) · g(x) · [blah (no x’s)]
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 15 / 1 Sum both sides over x:
X X 1 P(X = x) = f (x) · [blah (no x’s)] c x x
Accept/Reject Algorithm
So, f (x) P(X = x) = h(x) · g(x) · [blah (no x’s)]
1 f (x) = c g(x) · g(x) · [blah (no x’s)]
1 = c f (x) · [blah (no x’s)]
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 15 / 1 Accept/Reject Algorithm
So, f (x) P(X = x) = h(x) · g(x) · [blah (no x’s)]
1 f (x) = c g(x) · g(x) · [blah (no x’s)]
1 = c f (x) · [blah (no x’s)]
Sum both sides over x:
X X 1 P(X = x) = f (x) · [blah (no x’s)] c x x
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 15 / 1 ⇓ 1 1 = 1 · · [blah (no x’s)] c
⇓ 1 · [blah (no x’s)] = 1 c √ ⇒ P(X = x) = f (x)
Accept/Reject Algorithm
X X 1 P(X = x) = f (x) · [blah (no x’s)] c x x
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 16 / 1 ⇓ 1 · [blah (no x’s)] = 1 c √ ⇒ P(X = x) = f (x)
Accept/Reject Algorithm
X X 1 P(X = x) = f (x) · [blah (no x’s)] c x x
⇓ 1 1 = 1 · · [blah (no x’s)] c
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 16 / 1 √ ⇒ P(X = x) = f (x)
Accept/Reject Algorithm
X X 1 P(X = x) = f (x) · [blah (no x’s)] c x x
⇓ 1 1 = 1 · · [blah (no x’s)] c
⇓ 1 · [blah (no x’s)] = 1 c
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 16 / 1 Accept/Reject Algorithm
X X 1 P(X = x) = f (x) · [blah (no x’s)] c x x
⇓ 1 1 = 1 · · [blah (no x’s)] c
⇓ 1 · [blah (no x’s)] = 1 c √ ⇒ P(X = x) = f (x)
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 16 / 1 Target Density: f (x) = 9xe−3x , x > 0 Will try to bound it by a function of the form g(x) = k · e−2x .
ke−2x − 9xe−3x = e−3x (kex − 9x)
Hmmm... k = 9 will work...... but we might try to do
better since the accept f1 probabilities are of the form f /g. 0.0 0.5 1.0 1.5 2.0 2.5 3.0
0 1 2 3 4
x
Accept/Reject Algorithm
Example: Simulate X ∼ Γ(2, 3)
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 17 / 1 Will try to bound it by a function of the form g(x) = k · e−2x .
ke−2x − 9xe−3x = e−3x (kex − 9x)
Hmmm... k = 9 will work...... but we might try to do
better since the accept f1 probabilities are of the form f /g. 0.0 0.5 1.0 1.5 2.0 2.5 3.0
0 1 2 3 4
x
Accept/Reject Algorithm
Example: Simulate X ∼ Γ(2, 3) Target Density: f (x) = 9xe−3x , x > 0
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 17 / 1 ke−2x − 9xe−3x = e−3x (kex − 9x)
Hmmm... k = 9 will work...... but we might try to do
better since the accept f1 probabilities are of the form f /g. 0.0 0.5 1.0 1.5 2.0 2.5 3.0
0 1 2 3 4
x
Accept/Reject Algorithm
Example: Simulate X ∼ Γ(2, 3) Target Density: f (x) = 9xe−3x , x > 0 Will try to bound it by a function of the form g(x) = k · e−2x .
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 17 / 1 Hmmm... k = 9 will work...... but we might try to do
better since the accept f1 probabilities are of the form f /g. 0.0 0.5 1.0 1.5 2.0 2.5 3.0
0 1 2 3 4
x
Accept/Reject Algorithm
Example: Simulate X ∼ Γ(2, 3) Target Density: f (x) = 9xe−3x , x > 0 Will try to bound it by a function of the form g(x) = k · e−2x .
ke−2x − 9xe−3x = e−3x (kex − 9x)
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 17 / 1 ... but we might try to do better since the accept probabilities are of the form f /g.
Accept/Reject Algorithm
Example: Simulate X ∼ Γ(2, 3) Target Density: f (x) = 9xe−3x , x > 0 Will try to bound it by a function of the form g(x) = k · e−2x .
ke−2x − 9xe−3x = e−3x (kex − 9x)
Hmmm... k = 9 will work... f1 0.0 0.5 1.0 1.5 2.0 2.5 3.0
0 1 2 3 4
x Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 17 / 1 Accept/Reject Algorithm
Example: Simulate X ∼ Γ(2, 3) Target Density: f (x) = 9xe−3x , x > 0 Will try to bound it by a function of the form g(x) = k · e−2x .
ke−2x − 9xe−3x = e−3x (kex − 9x)
Hmmm... k = 9 will work...... but we might try to do
better since the accept f1 probabilities are of the form f /g. 0.0 0.5 1.0 1.5 2.0 2.5 3.0
0 1 2 3 4
x Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 17 / 1 Target Density: f (x) = 9xe−3x , x > 0 Will try to bound it by a function of the form g(x) =9 k · e−2x .
9ke−2x − 9xe−3x = 9e−3x (kex − x)
k = e−1 will work... 0.0 0.5 1.0 1.5 2.0 2.5 3.0
0 1 2 3 4
x
Accept/Reject Algorithm
Example: Simulate X ∼ Γ(2, 3)
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 18 / 1 Will try to bound it by a function of the form g(x) =9 k · e−2x .
9ke−2x − 9xe−3x = 9e−3x (kex − x)
k = e−1 will work... 0.0 0.5 1.0 1.5 2.0 2.5 3.0
0 1 2 3 4
x
Accept/Reject Algorithm
Example: Simulate X ∼ Γ(2, 3) Target Density: f (x) = 9xe−3x , x > 0
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 18 / 1 9ke−2x − 9xe−3x = 9e−3x (kex − x)
k = e−1 will work... 0.0 0.5 1.0 1.5 2.0 2.5 3.0
0 1 2 3 4
x
Accept/Reject Algorithm
Example: Simulate X ∼ Γ(2, 3) Target Density: f (x) = 9xe−3x , x > 0 Will try to bound it by a function of the form g(x) =9 k · e−2x .
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 18 / 1 k = e−1 will work... 0.0 0.5 1.0 1.5 2.0 2.5 3.0
0 1 2 3 4
x
Accept/Reject Algorithm
Example: Simulate X ∼ Γ(2, 3) Target Density: f (x) = 9xe−3x , x > 0 Will try to bound it by a function of the form g(x) =9 k · e−2x .
9ke−2x − 9xe−3x = 9e−3x (kex − x)
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 18 / 1 Accept/Reject Algorithm
Example: Simulate X ∼ Γ(2, 3) Target Density: f (x) = 9xe−3x , x > 0 Will try to bound it by a function of the form g(x) =9 k · e−2x .
9ke−2x − 9xe−3x = 9e−3x (kex − x)
k = e−1 will work... 0.0 0.5 1.0 1.5 2.0 2.5 3.0
0 1 2 3 4
x Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 18 / 1 But, choosing something of the form g(x) = xc1 e−c2x that will be0 for x = 0 will result in us having to draw values (for possible acceptance) from a density that is probably at least as difficult to draw from as the original f !
Accept/Reject Algorithm
Example: Simulate X ∼ Γ(2, 3) In fact, it will be best (in terms of maximizing the acceptance rate) if we can, with our choice of g, closely mimic the target density f . 0.0 0.5 1.0 1.5 2.0 2.5 3.0
0 1 2 3 4
x
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 19 / 1 Accept/Reject Algorithm
Example: Simulate X ∼ Γ(2, 3) In fact, it will be best (in terms of maximizing the acceptance rate) if we can, with our choice of g, closely mimic the target density f .
But, choosing something of the form g(x) = xc1 e−c2x that will be0 for x = 0 will result in us having to draw values (for possible acceptance) from a density that is probably at least as difficult to draw from as the original f ! 0.0 0.5 1.0 1.5 2.0 2.5 3.0
0 1 2 3 4
x
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 19 / 1 ... but this is probably not worth the trouble. Let’s see how fast the algorithm is with
g(x) = 9e−1e−2x .
Accept/Reject Algorithm
Example: Simulate X ∼ Γ(2, 3) In fact, it will be best (in terms of maximizing the acceptance rate) if we can, with our choice of g, closely mimic the target density f .
We could use a piecewise uniform/exponential.... 0.0 0.5 1.0 1.5 2.0 2.5 3.0
0 1 2 3 4
x
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 20 / 1 Accept/Reject Algorithm
Example: Simulate X ∼ Γ(2, 3) In fact, it will be best (in terms of maximizing the acceptance rate) if we can, with our choice of g, closely mimic the target density f .
We could use a piecewise uniform/exponential...... but this is probably not worth the trouble. Let’s see how fast the algorithm is with
g(x) = 9e−1e−2x . 0.0 0.5 1.0 1.5 2.0 2.5 3.0
0 1 2 3 4
x
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 20 / 1 Accept/Reject Algorithm
Results: (100,000 values)
mean accept time = 1.65627 s.d. = 1.038629 Density Density 0.0 0.1 0.2 0.3 0.4 0.5 0.6
0 2 4 6 8 10 12 14 0.0 0.2 0.4 0.6 0.8 1.0
0 1 2 3 4 5
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 21 / 1 Odds and Ends...
Incidentally, how would one draw from this (unnormalized) density:
9e−2 , 0 < x < 0.5 f (x) ∝ 9e−1e−2x , x ≥ 0.5 0.0 0.2 0.4 0.6 0.8 1.0 1.2
0 1 2 3 4
Lesson 5: Generating (Some) Continuous Randomx VariablesStochastic Simulation September 12, 2018 22 / 1 could normalize with c, then w.p.9 e−1/c draw a unif (0, 1) w.p. 1 − 9e−1/c draw an exp(rate = 2) and shift it by adding0 .5
Odds and Ends...
Incidentally, how would one draw from this (unnormalized) density:
could normalize then compute and and invert the piecewise continuous cdf 0.0 0.2 0.4 0.6 0.8 1.0 1.2
0 1 2 3 4
x Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 23 / 1 Odds and Ends...
Incidentally, how would one draw from this (unnormalized) density:
could normalize then compute and and invert the piecewise continuous cdf could normalize with c, then w.p.9 e−1/c draw a unif (0, 1) w.p. 1 − 9e−1/c draw an exp(rate = 2) and shift it by adding0 .5 0.0 0.2 0.4 0.6 0.8 1.0 1.2
0 1 2 3 4
x Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 23 / 1 Odds and Ends...
Results: (second method candidate draws) Density 0.0 0.2 0.4 0.6 0.8 1.0
0 1 2 3 4 5 6
Lesson 5: Generating (Some) Continuous Random VariablesStochastic Simulation September 12, 2018 24 / 1