Macroeconomics Review Sheet: Laibson
Matthew Basilico Spring, 2013
Course Outline:
1. Discrete Time Methods
(a) Bellman Equation, Contraction Mapping Theorem, Blackwell's Su cient Conditions, Nu- merical Methods i. Applications to growth, search, consumption, asset pricing
2. Continuous Time Methods
(a) Bellman Equation, Brownian Motion, Ito Proccess, Ito's Lemma i. Application to search, consumption, price-setting, investment, I.O., asset-pricing
Lecture 1
Sequence Problem What is the sequence problem?
Find v(x0) such that ∞ X t v (x0) = sup δ F (xt, xt+1) ∞ {xt+1}t=0 t=0
subject to xt+1 ∈ Γ(x) with x0 given Variable de nitions
• xt is the state vector at date t
• F (xt, xt+1) is the ow payo at date t
function F is stationary a.k.a. the ow utility function
• δt is the expontential discount function
1 Discount Variable De nitions What are the de nitions of the dierent discount variables?
• δt is the exponential discount function • δ is referrred to as the exponential discount factor
• ρ is the discount rate
which is the rate of decline of the discount function, so
dδt dt ∗ ρ ≡ − ln δ = − δt · so e−ρ = δ
Bellman Equation What is the Bellman Equation? What are its components?
Bellman equation expresses the value function as a combination of a ow payo and a discounted continuation payo
v(x) = sup {F (x, x+1) + δv(x+1)} x+1∈Γ(x) Components:
• Flow payo is F (x, x+1)
• Current value function is v(x). Continuation value function is v(x+1) • Equation holds for all (feasible) values of x • v(·) is called the solution to the Bellman Equation
Any old function won't solve it
Policy Function What is a policy function? What is an optimal policy?
A policy is a mapping from x to the action space (which is equivalent to the choice variables) An optimal policy achieves payo v(x) for all feasible x
Relationship between Bellman and Sequence Problem
A solution to the Bellman will also be a solution to the sequence problem, and vice versa. 1. A solution to the Sequence Problem is a solution to the Bellman
2 2. A solution to the Bellman Equation is also a solution to the sequence problem
Notation with Sequence Problem and Bellman example: Optimal Growth with Cobb- Douglas Technology How do we write optimal growth using log utility and Cobb-Douglas technology? How can this be translated into sequence problem and Bellman Equation notation?
Optimal growth: ∞ X t sup δ ln(ct) ∞ {ct}t=0 t=0 α subject to the constraints c, k ≥ 0, k = c + k+1, and k0 given Optimal growth in Sequence Problem notation: [∞- horizon] ∞ X t α v(k0) = sup δ ln(kt − kt+1) ∞ {kt+1}t=0 t=0 α such that kt+1 ∈ [0, k ] ≡ Γ(k) and k0given Optimal growth in Bellman Equation notation: [2-period] α v(k) = sup {ln (k − k+1) + δv(k+1)} ∀k α k+1∈[0,k ]
Methods for Solving the Bellman Equation What are the 3 methods for solving the Bellman Equation?
1. Guess a solution 2. Iterate a functional operator analytically (This is really just for illustration) 3. Iterate a functional operator numerically (This is the way iterative methods are used in most cases)
3 FOC and Envelope What are the FOC and envelope conditions?
First order condtion (FOC): Dierentiate with respect to choice variable
∂F (x, x+1) 0 0 = + δv (x+1) ∂x+1 Envelope Theorem: Dierentiate with respect to state variable ∂F (x, x ) v0(x) = +1 ∂x
Optimal Stopping using G&C How do we approach optimal stopping using Guess and Check?
1. Write the Bellman Equation 2. Propose Threshold Rule as Policy Function 3. Use Policy Function to Write Value Function [Bellman Guess]
4. Re ne Value Function using continuity of v(·) 5. Find value of threshold x∗ so that Value Function solves Bellman (using indierence)
Expanded: Agent draws an oer x from a uniform distribution with support in the unit interval 1. Write the Bellman Equation
v(x) = max {x, δEv(x+1)} 2. Propose Threshold Rule as Policy Function
(a) Stationary threshold rule, with threshold x∗:
Accept if x ≥ x∗ Reject if x < x∗ 3. Use Policy Function to Write Value Function [Bellman Guess]
(a) Stationary threshold rule implies that there exists some constant v such that
x if x ≥ x∗ v(x) = v if x < x∗
4. Re ne Value Function using continuity of v(·)
(a) By continuity of the Value Function, it follows that v = x∗
x if x ≥ x∗ v(x) = x∗ if x < x∗
4 5. Find value of threshold x∗ so that Value Function solves Bellman
(a) Use indierence condition
∗ ∗ v(x ) = x = δEv(x+1)
∗ x=x x=1 1 1 so x∗ = δ x∗f(x)dx + δ xf(x)dx = δ (x∗)2 + δ ˆx=0 ˆx=x∗ 2 2 p Giving solution x∗ = δ−1 1 − 1 − δ2
Lecture 2
Bellman Operator What is the Bellman operator? What are its properties as a functional operator?
Bellman operator B, operating on a function w, is de ned (Bw)(x) ≡ sup {F (x, x+1) + δw(x+1)} ∀x x+1∈Γ(x) Note: The Bellman operator is a contraction mapping that can be used to iterate until convergence to a xed point, a function that is a solution to the Bellman Equation. Having Bw on the LHS frees the RHS continuation value function w from being the same function. Hence if sup Bw(x) 6= w(x), then keep getting a better (closer) function to the xed point where Bw(x) = w(x). Notation: When you iterate, RHS value function lags (one period). When you are not iterating, it doesn't lag.
Properties as a functional operator
• De nition is expressed pointwise - for one value of x - but applies to all values of x • Operator B maps function w to new function Bw
Hence operator B is a functional operator since it maps functions
Properties at solution to Bellman Equation
• If v is a solution to the Bellman Equation, then Bv = v (that is, (Bv)(x) = B(x) ∀x)
• Function v is a xed point of B (B maps v to v)
Contraction Mapping Why does Bnw converge as n → ∞? What is a contraction mapping? What is the contraction mapping theorem?
The reason Bnw converges as n → ∞ is that B is a contraction mapping.
De nition: Let (S, d) be a metric space and B : S → S be a function mapping S onto itself. B is a contraction mapping if for some δ ∈ (0, 1), d (Bf, Bg) ≤ δd(f, g) for any two functions f and g.
5 • Intuition: B is a contraction mapping if operating B on any two functions moves them strictly closer together. (Bf and Bg are strictly closer together than f and g) • A metric (distance function) is just a way of representing the distance between two functions (e.g. maximum pointwise gap between two functions)
Contraction Mapping Theorem: If (S, d) is a complete metric space and B : S → S is a contraction mapping, then: 1. B has exactly one xed point v ∈ S
n 2. for any v0 ∈ S, lim B v0 = v
n 3. B v0 has an exponential convergence rate at least as great as − ln δ
Blackwell's Theorem What is Blackwell's Theorem? How is it proved?
Blackwell's Su cient Conditions - these are su cienct (but not necessary) for an operator to be a contraction mapping Let X ⊆ Rl and let C(X) be a space of bounded functions f : X → R, with the sup-metric. Let B : C(X) → C(X) be an operator satisfying: 1. (Monotonicity) if f, g ∈ C(X) and f(x) ≤ g(x)∀x ∈ X, then (Bf)(x) ≤ (Bg)(x)∀x ∈ X 2. (Discounting) there exists some δ ∈ (0, 1) such that [B(f + a)] (x) ≤ (Bf)(x) + δa ∀f ∈ C(X), a ≥ 0, x ∈ X Then B is a contraction with modulus δ. [Note that a is a constant, and f + a is the function generated by adding a constant to the function f]
Proof: For any f, g ∈ C(X) we have f ≤ g + d(f, g) Properties 1 & 2 of the conditions imply
Bf ≤ B (g + d(f, g)) ≤ Bg + δd(f, g)
Bg ≤ B (f + d(f, g)) ≤ Bf + δd(f, g) Combining the rst and last terms:
Bf − Bg ≤ δd(f, g)
Bg − Bf ≤ δd(f, g)
|(Bf)(x) − (Bg)(x)| ≤ δd(f, g) ∀x
sup |(Bf)(x) − (Bg)(x)| ≤ δd(f, g) x
d(Bf, Bg) ≤ δd(f, g)
6 Checking Blackwell's Conditions Ex 4.1: Check Blackwell conditions for a Bellman operator in a consumption problem
Consumption problem with stochastic asset returns, stocastic labor income, and a liquidity constraint n o (Bf)(x) = sup u(c) + δEf R˜+1(x − c) +y ˜+1 ∀x c∈[0,x] 1. Monotonicity: Assume . Suppose ∗ is the optimal policy when the continuation value function is . f(x) ≤ g(x)∀x cf f
n ˜ o ∗ ˜ ∗ (Bf)(x) = sup u(c) + δEf R+1(x − c) +y ˜+1 = u(cf ) + δEf R+1(x − cf ) +y ˜+1 c∈[0,x]
∗ ˜ ∗ n ˜ o ≤ u(cf ) + δEg R+1(x − cf ) +y ˜+1 ≤ sup u(c) + δEg R+1(x − c) +y ˜+1 = (Bg)(x) c∈[0,x] [Note (in top line) elimination of sup by using optimal policy ∗ . In bottom line, can use any policy cf to add in sup]
2. Discounting Adding a constant (∆) to an optimization problem does not eect optimal choice, so: n h io [B(f + ∆)] (x) = sup u(c) + δE f R˜+1(x − c) +y ˜+1 + ∆ c∈[0,x] n o = sup u(c) + δEf R˜+1(x − c) +y ˜+1 + δ∆ = (Bf)(x) + δ∆ c∈[0,x]
Iteration Application: Search/Stopping What is the general notation? Taking lecture 1 example, can you iterate from an initial guess of v0(x) = 0 for 2 steps? How about proving convergence?
Notation: Iteration of the Bellman operator B:
n n−1 n−1 vn(x) = (B v0)(x) = B(B v0)(x) = max x, δE(B v0)(x) n−1 Let xn ≡ δE(B v0)(x+1), which is the continuation payo for vn(x). n xn is also the cuto threshold associated with vn(x) = (B v0)(x) n xn is su cient statistic for vn(x) = (B v0)(x)
Bellman Operator for the Problem: (Bw)(x) ≡ max {x, δEw(x+1)}. *Note there is no lag here in the w index since we are not iterating Iterating Bellman Operator once on v0(x) = 0:
v1(x) = (Bv0)(x) = max {x, δEv0(x+1)} = max {x, 0} = x
Iterating again on v1(x) = x:
2 v2(x) = (B v0)(x) = (Bv1)(x) = max {x, δEv1(x+1)} = max {x, δEx+1} δ x2 = δEx+1 = 2
7 x2 if x ≤ x2 v2 = x if x ≥ x2
Proof at limit: We have: n−1 xn−1 if x ≤ xn−1 (B v0)(x) = x if x ≥ xn−1
" x=xn−1 x=1 # n−1 δ 2 xn = δE(B v0)(x) = δ xn−1f(x)dx + xf(x)dx = xn−1 + 1 ˆx=0 ˆx=xn−1 2
We set xn = xn−1 to con rm that this sequence converges to −1 p 2 lim xn = δ 1 − 1 − δ n→∞
Lecture 3
Classical Consumption Model 1. Consumption; 2. Linearization of the Euler Equation; 3. Empirical tests without precautionary savings eects
Classical Consumption What is the classical consumption problem, in Sequence Problem and Bellman Equation notation?
1. Sequence Problem Representation
Find v(x) such that
∞ X t v(x0) = sup E0 δ u(ct) ∞ {ct}0 t=0
( a static budget constraint for consumption: c ∈ ΓC (x) ) subject to t X a dynamic budget constraint for assets: xt+1 ∈ Γ xt, ct, R˜t+1, y˜t+1,...
Variables: x is vector of assets, c is consumption, R is vector of nancial asset returns, y is vector of labor income
Common Example: The only asset is cash on hand, and consumption is constrained to lie between 0 and x:
C X ct ∈ Γ (xt) ≡ [0, xt]; xt+1 ∈ Γ xt, ct, R˜t+1, y˜t+1,... ≡ R˜t+1(xt − ct) +y ˜t+1; x0 = y0
0 Assumptions: y˜ is exogenous and iid; u is concave; limc↓0 u (c) = ∞ (so c > 0 as long as x > 0)
8 2. Bellman Equation Representation
[It is more convenient to think about c as the choice variable. The state variable, x, is stochastic, so it is not directly chosen (rather a distribution for xt+1 is chosen at time t)]
v(xt) = sup {u(ct) + δEtv(xt+1)} ∀x ct∈[0,xt]
xt+1 = R˜t+1 (xt − ct) +y ˜t+1
x0 = y0
Euler Equation 1: Optimization How is the Euler equation derived from the Bellman Equation of the consumption problem above using optimization?
n o v(xt) = sup u(ct) + δEtv R˜t+1(xt − ct) +y ˜t+1 ∀x ct∈[0,xt] 1. First Order Condition:
0 0 u (ct) = δEtR˜t+1v (xt+1) if 0 < ct < xt (interior) 0 0 u (ct) ≥ δEtR˜t+1v (xt+1) if ct = xt (boundary) h i 0 0 ˜ using chain rule FOCct : 0 = u (ct) + δEtv (xt+1) × (−Rt+1) 2. Envelope Theorem
0 0 v (xt) = u (ct) h i 0 0 ∂ct 0 Dierentiate with respect to state variable xt : v (x) = u (ct) = u (ct) ∂xt h xt+1+˜yt+1 i Note: xt+1 = R˜t+1 (xt − ct) +y ˜t+1 =⇒ ctR˜t+1 = R˜t+1xt − xt+1 +y ˜t+1 =⇒ ct = xt − R˜t+1 Putting the FOC and Envelope Condition together, and moving the Envelope Condition forward one period, we get: IEuler Equation:
0 0 u (ct) = δEtR˜t+1u (ct+1) if 0 < ct < xt (interior) 0 0 u (ct) ≥ δEtR˜t+1u (ct+1) if ct = xt (boundary)
Euler Equation 2: Perturbation How is the Euler equation derived from the Bellman Equation of the consumption problem above using perturbation? [*Prefers that we know it this way*]
1. General Intuition
• Cost of consuming one dollar less today?
9 0 Utility loss today = u (ct)
• Value of saving an extra dollar Discounted, expected utility gain of consuming R˜t+1 more dollars tomorrow:
0 Utility gain tomorrow = δEtR˜t+1u (ct+1)
2. Perturbation **Idea: at an optimum, there cannot be a perturbation that improves welfare of the agent. If there is such a feasible pertubation, then we've proven that whatever we wrote down is not an optimum.
Proof by contradiction: [Assume inequalities, and show they produce contradictions so cannot hold]
0 0 1. Suppose u (ct) < δEtR˜t+1u (ct+1) . 0 [If u (ct) is less than the discounted value of a dollar saved, then what should I do? Save more, consume less.]
(a) Then we can reduce ct by ε and raise ct+1 by R˜t+1 · ε to generate a net utility gain:
h 0 0 i 0 < δEtR˜t+1u (ct+1) − u (ct) · ε
=⇒ this perturbation raises welfare [Q? - de ning welfare here] 0 [Note the expression is from simply moving u (ct) to RHS from above and multiplying by ε] (a) If this perturbation is possible that raises welfare, then can't be true that we started this analysis at an optimum. i. This perturbation is always possible along the equilibrium path. ii. Hence, if this cannot be an optimum, we've shown that at an optimum, we must have:
0 0 u (ct) ≥ δEtR˜t+1u (ct+1)
0 0 2. Suppose u (ct) > δEtR˜t+1u (ct+1)
(a) Then we can raise ct by ε and reduce ct+1 by R˜t+1 · ε to generate a net utility gain: 0 i. [If u (ct) is greater than the value of a dollar saved, then what should I do? Save less, consume more.]
h 0 0 i ε · u (ct) − δEtR˜t+1u (ct+1) > 0
=⇒ this perturbation raises welfare [Q? - de ning welfare here] (b) If this perturbation is possible that raises welfare, then can't be true that we started this analysis at an optimum.
i. This perturbation is always possible along the equilibrium path, as long as ct < xt (liquidity constraint not binding). ii. Hence, if this cannot be an optimum, we've shown that at an optimum, we must have:
0 0 u (ct) ≤ δEtR˜t+1u (ct+1) as long as ct < xt
10 It follows that:
0 0 u (ct) = δEtR˜t+1u (xt+1) if 0 < ct < xt (interior) 0 0 u (ct) ≥ δEtR˜t+1u (xt+1) if ct = xt (boundary)
Linearizing the Euler Equation How do you linearize the Euler Equation?
Goal: Can we linearize this and make it operational without unrealistic assumptions?
Equation tells us what I should expect consumption growth ∆ ln ct+1 to do 1. Assume u is an isoelastic (i.e. CRRA) utility function
c1−γ − 1 u(c) = 1 − γ
Note: cγ−1−1 Important special case. limγ→1 1−γ =ln c.
2. Assume Rt+1 is known at time t 3. Rewrite the Euler Equation [using functional form]:
−γ −γ ct = EtδRt+1ct+1 4. [Algebra] Divide by −γ ct
−γ γ 1 = EtδRt+1ct+1ct 5. [Algebra] Rearrange, using exp[ln]:
−γ γ 1 = Et exp ln δRt+1ct+1ct 6. [Algebra] Distribute ln:
1 = Et exp [rt+1 − ρ + (−γ) ln ct+1/ct]
[Note: ln δ = −ρ; ln Rt+1 = rt+1] 7. [Algebra] Substitute notation for consumption growth:
1 = Et exp [rt+1 − ρ − γ∆ ln ct+1]
[Note: ln(ct+1/ct) = ln ct+1 − ln ct ≡ ∆ ln ct+1]
8. [Stats] Assume ∆ ln ct+1 is conditionally normally distributed. Apply expectation operator:
1 1 = exp E r − ρ − γ∆ ln c + γ2V ∆ ln c t t+1 t+1 2 t t+1
h a˜ Ea˜+ 1 V ara˜ i Note: Ee = e 2 where a˜is a random variable
2 Here we have: − γ∆ ln ct+1 ∼ N (−γ∆ ln ct+1, γ Vt∆ ln ct+1)
11 9. [Algebra] Take ln:
1 0 = E r − ρ − γ∆ ln c + γ2V ∆ ln c t t+1 t+1 2 t t+1 10. [Algebra] Divide by γ, rearrange:
1 1 (∗) ∆ ln c = (E r − ρ) + γV ∆ ln c t+1 γ t t+1 2 t t+1 Terms of linearized Euler Equation (∗): ∆ ln ct+1 = consumption growth Vt∆ ln ct+1 =conditional variance in consumption growth conditional on information known at time t. (i.e. conditional on being in year 49, what is the variance in consumption growth between 49 and 50) It is also known as the precautionary savings term as seen in the regression analysis
∆ ln ct+1 Why is ∆ ln ct+1 consumption growth?
ct+1 − ct ct+1 − ct ct+1 ct+1 ≈ ln 1 + = ln 1 + − 1 = ln ≡ ∆ ln ct+1 ct ct ct ct [the second step uses the log approximation, ∆x ≈ ln (1 + ∆x)]
Important Consumption Models What are the two important consumption models?
In 1970s, began recognizing that what is interesting is change in consumption, as opposed to just levels of consumption. Since then have stayed with this approach. 1. Life Cycle Hypothesis (Modigliani & Brumberg, 1954; Friedman) = Eat the Pie Problem
(a) Assumptions
i. R˜t = R ii. δR = 1 [δ < 1; R > 1] iii. Perfect capital markets, no moral hazard, so future labor income can be exchanged for current capital (b) Bellman Equation
v(x) = sup {u(c) + δEv(x+1)} ∀x c≤x
x+1 = R(x − c)
∞ X −t x0 = E R y˜t t=0
12 [At date 0, sells all his labor income for a lump sum, discounted at rate r. This sum is his only source of wealth now] [Eat the Pie, except every piece he leaves grows by a factor r] (c) Euler Equation
0 0 0 0 i. u (ct) = δRu (ct+1) = 1 · u (ct+1) = u (ct+1) ii. =⇒ consumption is constant in all periods (d) Budget Constraint
∞ ∞ X −t X −t R ct = E0 R y˜t t=0 t=0 [(Discounted sum of his consumption) must be equal to [the expectation at date 0 of (the discounted sum of his labor income)]] (e) Subsitute Euler Equation
∞ ∞ X −t X −t R c0 = E0 R y˜t t=0 t=0 h i i. To give nal form: [Euler Equation + Budget Constraint] Note: P∞ −t 1 t=0 R = 1 1− R
∞ 1 X c = 1 − E R−ty˜ ∀t 0 R 0 t t=0 Intuition: Consumption is an annuity. Consumption is equal to interest rate scaling my total net worth at date 0 1 is the annuity scaling term; approximately equal to the real net interest rate 1 − R 2. Certainty Equivalence Model (Hall, 1978)
(a) Assumptions
i. R˜t = R ii. δR = 1 [δ < 1; R > 1] iii. Can't sell claims to labor income iv. Quadratic utility: β 2 u(c) = αc − 2 c 0 A. Note: This admits negative consumption, and does not imply that limc↓0 u (c) = ∞ B. [Hence it is possible to consume a negative amount] (b) Bellman Equation
v(x) = sup {u(c) + δEv(x+1)} ∀x c
x+1 = R(x − c) +y ˜+1
x0 = y0 (c) Euler Equation
13 i. Euler becomes: ct = Etct+1 = Etct+n 0 0 0 0 u (c) = α − βc. u (ct) = δEtR˜t+1; u (ct+1) = Etu (ct+1) [Since δR = 1] =⇒ α − βct = Et [α − βct+1] = α − βEtct+1 =⇒ −βct = −βEtct+1 =⇒ ct = Etct+1
ii. So ct = Etct+1 = Etct+n implies consumption is a random walk:
ct+1 = ct + εt+1
iii. So ∆ct+1 can not be predicted by any information available at time t
A. If I know ct, no other economic variable should have predictive power at ct+1; any other information should be orthogonal (d) Budget Constraint
∞ ∞ X −t X −t R ct ≤ R y˜t t=0 t=0 [Sum from t to ∞ of discounted consumption is ≤the discounted value of stochastic labor income] i. Budget Constraint at time t:
∞ ∞ X −s X −s R ct+s = xt + R y˜t+s s=0 s=1 [Discounted sum of remaining consumption = cash on hand (at date t) and the dis- counted sum of remaining labor income] ii. Now applying expectation: [if it's true on every path, it's true in expectation]
∞ ∞ X −s X −s Et R ct+s = xt + Et R y˜t+s s=0 s=1
(e) Subsitute Euler Equation [ct = Etct+s]
∞ ∞ X −s X −s R ct = xt + Et R y˜t+s s=0 s=1 h i i. To give nal form: [Euler Equation + Budget Constraint] Note: P∞ −t 1 t=0 R = 1 1− R ∞ ! 1 X c = 1 − x + E R−sy˜ ∀t t R t t t+s s=1 Intuition: Just like in Modigliani, consumption is equal to interest rate scaling my total net worth at date t i. (but get there just by assuming a quadratic utility function) 1 is the annuity scaling term (approximately equal to the real net interest rate) 1 − R P∞ −s is total net worth at date (xt + Et s=1 R y˜t+s) t
14 Empirical Tests of Linearized Euler Equation: Without Precautionary Sav- ings Eects
Want to prove that consumption growth is unpredictable between t and t + 1 A. Testing Linearized Euler Equation: Without Precautionary Savings Eects 1. Start with: Linearized Euler Equation, in regression form:
1 1 ∆ ln c = (E r − ρ) + γV ∆ ln c + ε t+1 γ t t+1 2 t t+1 t+1
[where εt+1 is orthogonal to any information known at time t.] 2. Assume (counterfactually) precautionary savings is constant
(a) Euler equation reduces to: constant 1 ∆ ln ct+1 = + γ Etrt+1 + εt+1 [Note: when we replace precautionary term with a constant, we are eectively ignoring its eect (since it is no longer separately identi ed from the other constant term ρ )] γ B. Estimating the Linearized Euler Equation: 2 Goals and Approaches
1. Goal 1: Estimate 1 γ
(a) 1 is EIS, the elasticity of intertemporal substitution [for this model, EIS is the inverse of γ the CRRA; in other models EIS not 1 ] ∴ γ
∂∆ ln ct+1 1 (b) Estimate using , with equation ∆ ln ct+1 = constant + Etrt+1 + εt+1 ∂Etrt+1 γ
2. Goal 2: Test the orthagonality restriction Ωt⊥εt+1
(a) This means, test the restriction that information available at time t does not predict con- sumption growth in the following regression [new βXt term, for any variable]:
1 ∆ ln c = constant + E r + βX + ε t+1 γ t t+1 t t+1
(a) For example, does the date t expectation of income growth, Et∆ ln Yt+1 predict date t + 1 consumption growth [for Shea, etc test below]? i. [Model is trying to say that an expectation of rising incomei.e. being hired on job marketdoesn't translate into a rise in consumption. α should be 0.]
1 ∆ ln c = constant + E r + αE ∆ ln Y + ε t+1 γ t t+1 t t+1 t+1 C. Summary of Results:
1. 1 (Hall, 1988) γ ∈ [0, 0.2] (a) EIS is very smallpeople are fairly unresponsive to expected movements in the interest rate; this doesn't seem to be a big driver of consumption growth i. the parameter that is scaling the expected value of the interest rate is estimated to be close to 0
15 (b) But estimate may be messed up once you add in liquidity contraints. Someone who is very liquidity constrained → EIS of 0. (c) So what exactly are we learning when we see American consumers are not responsive to the interest rate? [Is it just about liquidity constraints, or does it mean that even in their absence, something deep about responsiveness to interest rate]
2. α ∈ [0.1, 0.8] (Campell and Mankiw 1989, Shea 1995)
(a) =⇒ expected income growth (predicted at date t) predicts consupmtion growth at date t + 1 (b) =⇒ the assumptions (1) the Euler Equation is true, (2) the utility function is CRRA, (3) the linearization is accurate, and (4) Vt∆ ln ct+1 is constant, are jointly rejected (c) Theories on why does expected income predict consumption growth: i. Leisure and consumption expenditure are substitutes ii. Work-based expenses iii. Households support lots of dependents in mid-life when income is highest iv. Houesholds are liquidity-constrained and impatient
v. Some consumers use rules of thumb: cit = Yit vi. Welfare costs of smoothing are second-order
Lecture 4
Precautionary Savings and Liquidity Contraints 1. Precautionary Savings Motives; 2. Liquidity Constraints; 3. Application: Numerical solution of problem with liquidity constraints; 4. Comparison to eat-the-pie problem
Precautionary Motives Why do people save more in response to increased uncertainty?
1. Uncertainty and the Euler Equation
1 γ E ∆ ln c = (E r − ρ) + V ∆ ln c t t+1 γ t t+1 2 t t+1
2 where Vt∆ct+1 = Et[∆ ln ct+1 − Et∆ ln ct+1]
(a) Increase in economic uncertainty raises Vt∆ct+1, raising Et∆ ln ct+1 i. Note: this is not a general result (i.e. doesn't apply to quadratic) 2. Key reason is the convexity of marginal utility
(a) An increase in uncertainty raises the expected value of marginal utility i. Convexity of marginal utility curve raises marginal utility in expectation in the next period 0 0 0 0 A. Euler equation (for CRRA): u (ct) 6= u (Et(ct+1)) but rather u (ct) = Etu (ct+1)
16 B. (Since by Jensen's inequality =⇒ Et(ct+1) > 1) ct (b) This increase the motive to save i. =⇒ referred to as the precautionary savings eect
3. De nition: Precautionary Savings
(a) Precautionary saving is the reduction in consumption due to the fact that future labor income is uncertain instead of being xed at its mean value.
Liquidity Constraints
Buer Stock Models What are the two key assumptions of the buer stock models? Qualitatively, what are some of their predictions
Since the 1990s, consumption models have emphasized the role of liquidity constraints (Zeldes, Caroll, Deaton).
1. Consumers face a borrowing limit: e.g. ct ≤ xt (a) This matters whether or not it actually binds in equilibrium (since desire to preserve buer stock will aect marginal utilty as you get close to full-borrowing level)
2. Consumers are impatient: ρ > r
• Predictions
Consumers accumulate a small stock of assets to buer transitory income shocks Consumption weakly tracks income at high frequencies (even predictable income) Consumption strongly tracks income at low frequencies (even predictable income)
17 Eat the Pie Problem
A model in which the consumer can securitize her income stream. In this model, labor can be trans- formed into a bond. 1. Assumptions
(a) If consumers have exogenous idiosyncratic labor income risk, then there is no risk premium and consumers can sell their labor income for
∞ X −t W0 = E0 R yt t=0 (b) Budget constraint
W+1 = R(W − c) 2. Bellman Equation
v(W ) = sup {u(c) + δEv(R(W − c))} ∀x c∈[0,W ] 3. Solve Bellman: Guess Solution Form
( 1−γ ) ψ W if γ ∈ [0, ∞], γ 6= 1 v(W ) = 1−γ φ + ψ ln W if γ = 1
4. Con rm that solution works (problem set)
5. Derive optimal policy rule (problem set)
− 1 c = ψ γ W
− 1 1−γ 1 ψ γ = 1 − (δR ) γ
Lecture 5
Non-stationary dynamic programming
18 Non-Stationary Dynamic Programming What type of problems does non-stationary dynamic programming solve? What is dierent about the approach generally?
Non-Stationary Problems:
• So far we have assumed problem is stationary - that is, value function does not depend on time, only on the state variable
• Now we apply to nite horizon problems
E.g. Born at t = 1, terminate at t = T = 40
Two changes for nite horizon problems: backwards induction 1. We index the value function, since each value function is now date-speci c
• vt(x) represents value function for period t E.g., PT s−t • vt(x) = Et s=t δ u(cs) 2. We don't use an arbitrary starting function for the Bellman operator
• Instead, iterate the Bellman operator on vT (x), where T is the last period in the problem In most cases, the last period is easy to calculate, (i.e. end of life with no bequest
motive: vT (x) = u(x))
Backward Induction How does backward induction proceed?
• Since you begin in the nal period, the RHS equation is one period ahead of the LHS side equation (opposite of previous notation with operator)
Each iteration takes on more step back in time
• Concretely, we have the Bellman Equation
vt−1(x) = sup {u(c) + δEtvt (R(x − c) +y ˜)} ∀x c∈[0,x]
• *To generate vt−1(x), apply the Bellman Operator
vt−1(x) = (Bvt)(x) = sup {u(c) + δEtvt (R(x − c) +y ˜)} ∀x c∈[0,x]
(As noted above, generally start at termination date T and work backwards)
• Generally we have:
n vT −n(x) = (B vT )(x)
19 Lecture 6
Hyperbolic Discounting
Context: 2 3 For β < 1, Ut = u(ct) + β δu(ct+1) + δ u(ct+2) + δ u(ct+3) + ...
Hyperbolic Bellman Equations
Bellman Equation in set up in the following way. De ne three equations where: V is the continuation value function W is the current value function C is the consumption function 1. V (x) = U(C(x)) + δE[V (R(x − C(x)) + y)] 2. W (x) = U(C(x)) + βδE[V (R(x − C(x)) + y)]
3. C(x) = arg maxc U(c) + βδE[V (R(x − c) + y)]
Three further identities:
• Identity linking V and W βV (x) = W (x) − (1 − β)U(C(x))
• Envelope Theorem
W 0(x) = U 0(C(x))
• FOC
U 0(C(x)) = RβδE [V 0(R(x − C(x)) + y)]
Hyperbolic Euler Equation How do we derive an Euler equation for the hyperbolic scenario? What is the intuition behind the equation?
1. We have the FOC
0 0 u (ct) = RβδEt [V (xt+1)]
2. Use the Identity linking V and W to subsitute for V 0
0 0 dCt+1 = RδEt W (xt+1) − (1 − β)u (ct+1) dxt+1 3. Use envelope theorem
20 0 0 (a) [substitute u (xt+1) for W (ct+1)]
0 0 dCt+1 = RδEt u (xt+1) − (1 − β)u (ct+1) dxt+1 4. Distribute
dCt+1 dCt+1 0 = REt βδ + δ 1 − u (ct+1)(∗) dxt+1 dxt+1
Interpretation of Hyperbolic Euler Equation (∗)
• First, note that dCt+1 is the marginal propensity to consume (MPC). dxt+1 • We then see that the main dierence in this Euler equation is that it is a weighted average of discount factors dCt+1 dCt+1 0 = REt βδ +δ 1 − u (ct+1) dxt+1 dxt+1 | {z } | {z } short run long run
• More cash on hand will shift to to the long-run term (less hyperbolic discounting)
Sophisticated vs. Naive Agents
[From Section Notes]
• Sophisticated Agents
The above analysis has assumed sophisticated agents A sophisticated agent knows that in the future, he will continue to be quasi-hyperbolic (that is, he knows that he will continue to choose future consumption in such a way that he discounts by βδ between the current and next period, but by δ between future periods.) This is re ected in the speci cation of the consumption policy function (as above)
• Naive Agents
Doesn't realize he will discount in a quasi-hyperbolic way in future periods He wrongly believes that, starting from the next period, he will discount by δ between all periods. ∗ (Another way of saying: he wrongly believes that he will be willing to commit to the path that he currently sets) He will decide today, believing that he will choose from tomorrow forward by the exponen- tial discount function. (e superscript); however, when he reaches the next date, he again reoptimizes:
21 1. De ne (a) Continuation value function, as given by standard exponential discounting Bellman equation:
e e e e V (x) = max u c + δEt+1V R(x − c ) +y ˜ t+1 e t+1 t+2 t+1 ct+1∈[0,x] (b) Current value function, as given by the hyperbolic equation:
n n e e W (x) = max u (c ) + δEtV (R(x − c ) +y ˜) t e t+1 t ct ∈[0,x] 2. Since he is not able to commit to the exponential continuation function, realized consump- tion path will be given: (a) Realized consumption path: n T {cτ (xτ )}τ=t (b) Solution at each time t is characterized by
0 e 0 e u (ct+1) = δREt+1u ct+2
0 n 0e 0 e u (ct ) = βδREtV (xt+1) = βδREtu ct+1
Lecture 7
Asset Pricing 1. Equity Premium Puzzle; 2. Calibration of Risk Aversion; 3. Resolutions to the Equity Premium Puzzle
Intuition
• Classical economics says that you should take an arbitrarily small bet with any positive re- turn...as long as the payos of the bet are uncorrelated with your consumption path (punchline)
• Don't want to make a bet where you will give up resources in a low state of consumption (when the marginal utility of consumption is higher) for resources in a high state of consumption (where marginal utility of consumption is lower) [all because of curvature of the utility function]
• Not variance-based risk, covariance based risk
Asset Pricing Equation What is the asset pricing equation? How is it derived?
Asset Pricing Equation General Form:
ij π = γσic − γσjc
22 where πij = ri − rj (the gap in rate of returns between assets iand j)
i i σic = Cov(σ ε , ∆ ln c) Risk-Free Asset: De nition: we denote the risk free return: f . Speci cally, we assume that at time , f is known. Rt t − 1 Rt When i =equities and j =risk free asset, we have the simple relationship
equity,f π = γσequity,c Intuition:
equity,f • The equity premium π , is equal to the amount of risk, σequity,c, multiplied by the price of risk, γ.
Asset Notation and Statistics:
1. Allowing for many dierent types of assets: 1 2 i I Rt+1,Rt+1,...,Rt+1,...,Rt+1 2. Assets have stochastic returns:
2 ri +σiεi − 1 σi i t+1 t+1 2 [ ] where i has unit variance Rt+1 = e εt+1 i has unit variance i is a normally distributed random variable: i εt+1 =⇒ εt+1 εt+1 ∼ N (0, 1) x ∼ N(µ, σ2) −→Ax + B ∼ N(Aµ + B,A2σ2) 1 2 2 −→ E [exp (Ax + B)] = exp Aµ + B + 2 A σ [Since 2 exp exp 1 ; so if standard normal: x ∼ N(µ, σ )−→ E( (x)) = (E(x) + 2 V ar(x)) 1 2 ] = exp(µ + 2 σ ) Hence: i i i 1 i 2 i i 1 i 2 i2 2 i 1 i 2 i2 σ εt+1 + rt+1 − 2 [σ ] ∼ N(σ ∗ 0 + rt+1 − 2 [σ ] , σ ∗ 1 ) = N(rt+1 − 2 [σ ] , σ ) t i i i 1 i 2 which as we've seen is now exp(a random variable) so E [Ri] = E exp(rt+1 + σ εt+1 − 2 [σ ] ) we can use 1 i 1 i 2 1 i2 i = exp(mean + 2 V ar) = exp(rt+1 − 2 [σ ] + 2 σ ) = exp(rt+1) Finally we have that since for small x: ln(1 + x) ≈ x =⇒ 1 + x ≈ ex i i . ∴ exp(rt) ≈ 1 + rt+1
ij Derivation of Asset Pricing Equation: π = γσic − γσjc How is the Asset Pricing Equation derived?
Use Euler, Substitute, Take Expectation (Random Variable), Dierence Two Equations, Apply Co- variance Formula 1. Start with the Euler Equation
0 i 0 u (ct) = Et δRt+1u (ct+1) 2. Assume u is isoelastic (CRRA):
c1−γ − 1 u(c) = 1 − γ
23 3. Substitute into the Euler equation
(a) δ = exp(−ρ)
2 ri +σiεi − 1 σi (b) i t+1 t+1 2 [ ] where i has unit variance Rt+1 = e εt+1 (c) u0(c) = c−γ (given CRRA)
−γ i −γ c = Et δRt+1ct+1
1 2 c−γ = E exp −ρ + ri + σiεi − σi c−γ t t+1 t+1 2 t+1
" −γ # i i i 1 i2 ct+1 1 = Et exp −ρ + rt+1 + σ εt+1 − σ 2 ct
4. Re-write in terms of exp(·), using exp(ln)
i i i 1 i2 ct+1 1 = Et exp −ρ + rt+1 + σ εt+1 − σ exp −γ ln 2 ct
1 2 1 = E exp −ρ + ri + σiεi − σi − γ∆ ln (c ) t t+1 t+1 2 t+1 (a) Rearrange, and note
i 1 i2 i i 1 = Et exp −ρ + r − σ + σ ε − γ∆ ln (ct+1) t+1 2 t+1 | {z } | {z } non-stochastic random variable 5. Take Expectation, applying the formula for the expectation of a Random Variable
(a) That is, use: 2 2 1 2 2 Sx ∼ N(Sµ, S σ ) −→ E [exp (Sx)] = exp Sµ + 2 S σ i. For the random variable i i σ εt+1 − γ∆ ln (ct+1) ii. That is, i i 1 i i E σ εt+1 − γ∆ ln (ct+1) = exp −γEt [∆ ln (ct+1)] + 2 V σ εt+1 − γ∆ ln (ct+1)
1 2 1 1 = exp −ρ + ri − σi − γE [∆ ln (c )] + V σiεi − γ∆ ln (c ) t+1 2 t t+1 2 t+1 t+1
6. Take Ln
1 2 1 0 = −ρ + ri − σi − γE [∆ ln (c )] + V σiεi − γ∆ ln (c ) t+1 2 t t+1 2 t+1 t+1 7. Dierence Euler Equation for assets i and j, and Re-write:
1 h 2 2i 1 h h ii 0 = ri − rj − σi − σj + V σiεi − γ∆ ln (c ) − V σjεj − γ∆ ln (c ) t+1 t+1 2 2 t+1 t+1 t+1 t+1
24 (a) Note that −ρ, γEt [∆ ln (ct+1)] drop out of the equation (b) Re-write with interest rate dierence on LHS:
1 h 2 2i 1 h h ii ri − rj = σi − σj − V σiεi − γ∆ ln (c ) − V σjεj − γ∆ ln (c ) t+1 t+1 2 2 t+1 t+1 t+1 t+1
8. Apply formula for V (A + B)
(a) [V (A + B) = V (A) + V (B) + 2Cov(A, B)] i. Also note 2Cov(A, −αB) = −2αCov(A, B)
Note, in our case: i i i i i i V σ εt+1 − γ∆ ln (ct+1) = V σ εt+1 +V (−γ∆ ln (ct+1))+2Cov σ εt+1, γ∆ ln (ct+1) i2 2 = σ + γ V ∆ ln (ct+1) − 2γσic
1 h 2 2 h 2 i h 2 ii ri −rj = σi − σj − σi + γ2V ∆ ln (c ) − 2γσ + σj + γ2V ∆ ln (c ) − 2γσ t+1 t+1 2 t+1 ic t+1 jc Just cancel terms: i j 1 i \2 i 2 j \2 j 2 2 2 rt−rt = 2 (σ ) − (σ ) + (σ ) − (σ ) + γ V ∆ ln (ct+1\) − γ V ∆ ln (ct+1) + 2γσic − 2γσjc To get out asset pricing equation:
ij i j π = rt+1 − rt+1 = γσic − γσjc
Equity Premium Puzzle What is the equity premium puzzle?
equity,f We can rearrange the asset pricing equation for the risk free asset π = γσequity,c to isolate γ:
πequity,f γ = σequity,c Empirical Date: In the US post-war period
• πequity,f ≈ .06
• σequity,c ≈ .0003 .06 γ = = 200 .0003
25 Lecture 8
Summary Equations
• Brownian Motion; Discrete to Continuous +h with prob p ∆x ≡ x(t + ∆t) − x(t) = −h with prob q = 1 − p t (*) E [x(t) − x(0)] = n ((p − q)h) = ∆t (p − q)h 2 t 2 (*) V [x(t) − x(0)] = n 4pqh = ∆t 4pqh • Calibration √ h = σ ∆t h √ i √ 1 α , α p = 2 1 + σ ∆t ⇒ (p − q) = σ ∆t √ √ E [x(t) − x(0)] = t (p − q)h = t α ∆t σ ∆t = αt ∆t ∆t σ |{z} linear drift V [x(t) − x(0)] = σ2t |{z} linear variance • Weiner √ √ 1. ∆z = ε ∆t where ε ∼ N(0, 1), [also stated ∆z ∼ N(0, ∆t)] Vertical movements ∝ ∆t 2. Non overlapping increments of ∆z are independent
• Ito: dx = a(x, t)dt + b(x, t)dz; dx = a(x, t)dt + b(x, t)dz | {z } | {z } drift standard deviation 1. Random Walk: dx = αdt + σdz [with drift α and variance σ2] 2. Geometric Random Walk: dx = αxdt + σxdz [with proportional drift α and proportional variance σ2] • Ito's Lemma: z(t) is Wiener, x(t) is Ito with dx = a(x, t)dt + b(x, t)dz. Let V = V (x, t), then ∂V ∂V 1 ∂2V 2 dV = ∂t dt + ∂x dx + 2 ∂x2 b(x, t) dt h ∂V ∂V 1 ∂2V 2i ∂V = ∂t + ∂x a(x, t) + 2 ∂x2 b(x, t) dt + ∂x b(x, t)dz • Value Function as Ito Process:
Value Function is Ito Process (V ) of an Ito Process (x) or a Weiner Process (z) (need to check this)
∂V ∂V 1 ∂2V ∂V dV =a ˆ(x, t)dt + ˆb(x, t)dz = + a(x, t) + b(x, t)2 dt + b(x, t) dz ∂t ∂x 2 ∂x2 ∂x | {z } | {z } aˆ ˆb • Proof of Ito's Lemma: ∂V 1 ∂2V 2 ∂V 1 ∂2V 2 ∂2V h.o.t.` dV = ∂t dt + 2 ∂t2 (dt) + ∂x dx + 2 ∂x2 (dx) + ∂x2 dxdt + (dx)2 = b(x, t)2(dz)2 + h.o.t. = b(x, t)2dt + h.o.t.
26 Motivation: Discrete to Continuous Time Brownian Motion Approximation with discrete intervals in continuous time
Have continuous time world, but use discrete time steps ∆t. Want to see what happens as the step length goes to zero, that is ∆t → 0.
Process with jumps at discrete intervals:
• At every ∆t intervals, a process x(t) either goes up or down:
+h with prob p ∆x ≡ x(t + ∆t) − x(t) = −h with prob q = 1 − p
• Expectation of this process: E(∆x) = ph + q(−h) = (p − q)h • Variance of this process De nition of Variance of a random variable: V (∆x) = E [∆x − E∆x]2 We have: V (∆x) = E [∆x − E∆x]2 = E (∆x)2 − [E∆x]2 = 4pqh2 (Since)E (∆x)2 = ph2 + q (−h)2 = h2
Analysis of x(t) − x(0) in this process Time span of length implies t steps in • t n = ∆t x(t) − x(0) =⇒ x(t) − x(0) is a binomial random variable
• General form for probability that x(t) − x(0) is a certain combination of h and −h: Probability that x(t) − x(0) = (k)(h) + (n − k)(−h) is n k n−k k p q (where n n! ) k = k!(n−k)! • Expectation and Variance:
t (*) E [x(t) − x(0)] = n ((p − q)h) = ∆t (p − q)h ∗ (n times the individual step ∆x. Since these are iid steps, the sum is n times each element in the sum) 2 t 2 (*) V [x(t) − x(0)] = n 4pqh = ∆t 4pqh ∗ (Since all steps are iid: when we look at sum of random variables, where random variables are all independent, the variance of the sum is the sum of the variances)
We have binomial random variable. As we chop into smaller and smaller time steps, will converge to normal density.
So far: we've taken the discrete time process, discussed the individual steps ∆x given a time step ∆t, and then we've aggregated across many time steps to an interval of length t, during which time there are steps t , and we've observed that this random variable, , is a binomial random n n = ∆t x(t) − x(0)
27 variable with mean t and variance t 2. Our next job is to move from discrete number ∆t (p − q)h ∆t 4pqh of steps to smaller and smaller steps.
Calibrate h and p to give us properites we want as we take ∆t → 0: [Linear drift and variance]
• Let: √ h = σ ∆t (*) h √ i 1 α (*) p = 2 1 + σ ∆t These Follow: √ 1 h α i q = 1 − p = 2 1 − σ ∆t √ α (p − q) = σ ∆t • Where did this come from? (linking ∆t to h, p, q) We are trying to get to continuous time random walk with drift. Random walk has variance that increases linearly with t forcasting horizon. Would like this property to emerge in continuous time.
Consider the variance (of the aggregate time random variable): t 2. ∆t 4pqh ∆t is going to zero; p and q are numbers that are around .5 so can think of them as constants in the limit, 4 is constant, t is the thing we'd like everything to be linear with respect to √ ∗ So given all this, h must be proportional to ∆t [then put in scaling parameter σ] · But then, what must (p−q) be for drift to be linear in t E [x(t) − x(0)] = t (p − q)h ? √ ∆t (p − q) must also be proportional to ∆t, which (subject to a ne transformations) pins down p, q and (p − q). ∗ No other way to calibrate this for it to make sense as a continuous time random walk
• Expectation and Variance
t t α√ √ E [x(t) − x(0)] = (p − q)h = ∆t σ ∆t = αt ∆t ∆t σ |{z} linear drift
t t 1 α2 α2 V [x(t) − x(0)] = 4pqh2 = 4 1 − ∆t σ2∆t = tσ2 1 − ∆t −→ ∆t ∆t 4 σ σ
σ2t (as ∆t → 0) |{z} linear variance
Intuition as ∆t → 0 • As take ∆t to 0, mathematically converging to a continuous time random walk. • At a point t,we have a binomial random variable. It is converging to a normal random variable with mean αt and var σ2t.
• [These are the four graphs simulating the process, which appear closer and closer to Brownian motion as ∆t gets smaller]
28 • Hence this Random Walk, a continuous time stochastic process, is the limit case of a family of stochastic processes
Properties of Limit Case (Random Walk): √ 1. Vertical movements proportional to ∆t (not ∆t) 2. [x(t) − x(0)] →D N(αt, σ2t), since Binomial →D Normal √ 3. Length of curve during 1 time period > nh = 1 σ ∆t = σ √1 → ∞ ∆t ∆t (a) (Since length will be greater than the number of step sizes, time the length of each step size) (b) Hence length of curve is in nity over any nite period of time
√ 4. ∆x ±σ ∆t ±σ ∆t = ∆t = ∆t → ±∞ (a) Time derivative, dx , doesn't exist E dt (b) So we can't talk about expectation of the derivative. However, we can talk about the following derivative-like objects (5,6):
√ √ α ∆t σ ∆t 5. E(∆x) (p−q)h ( σ )( ) ∆t = ∆t = ∆t = α (a) So we write E (dx) = αdt i. That is, in a period ∆t, you expect this process to increase by α
2 4( 1 ) 1−( α ) ∆t σ2∆t 6. V (∆x) 4 σ 2 ∆t = ∆t → σ (a) So we write V (dx) = σ2dt
• Finally we see what the process is everywhere continuous, nowhere dierentiable
When we let ∆t converge to zero, the limiting process is called a continuous time random walk with (instantaneous) drift α and (instantaneous) variance σ2. We generated this continuous-time stochastic process by building it up as a limit case. We could have also just de ned the process directly as opposed to converging to it:
Note that ”x” above becomes ”z” for following:
Wiener Process
Weiner processes: Family of processes described before with zero drift, and unit variance per unit time De nition
If a continuous time stoachastic process z(t) is a Wiener Process, then any change in z, ∆z, corre- sponding to a time interval ∆t, satis es the following conditions: √ 1. ∆z = ε ∆t where ε ∼ N(0, 1)
29 (a) [often also stated ∆z ∼ N(0, ∆t)] √ (b) ε is the gausian piece, and ∆t is the scaling piece
2. Non overlapping increments of ∆z are independent
(a) A.K.A. If t1 ≤ t2 ≤ t3 ≤ t4 then E [(z(t2) − z(t1)) (z(t4) − z(t3))] = 0
A process tting these two properties is a Weiner process. It is the same process converged to in the rst part of the lecture above.
Properties
• A Wiener process is a continuous time random walk with zero drift and unit variance • z(t) has the Markov property: the current value of the process is a su cient statistic for the distribution of future values
History doesn't matter. All you need to know to predict in y periods is where you are right now.
• z(t) ∼ N(z(0), t) so the variance of z(t) rises linearly with t (Zero drift case)
Note on terminology: Wiener processes are a subset of Brownian motion (processes), which are a subset of Ito Processes.
Ito Process
Now we generalize. Let drift (which we'll think of as a with arguments x and t, a(x, t)) depend on time t and state x. [Let E∆x equal to a function with those two arguments] lim∆t→0 ∆t a Similarly, generalize variance.
From Wiener to Ito Process
Let z(t) be a Wiener Process. Let x(t) be another continuous time stocastic process such that
E∆x i.e. "drift" lim∆t→0 ∆t = a (x, t) E (dx) = a (x, t) dt V ∆x 2 i.e. 2 "variance" lim∆t→0 ∆t = b (x, t) V (dx) = b (x, t) dt Note that the i.e. equations are formed by simply bringing the dt to the RHS. We summarize these properties by writing the expression for an Ito Process:
dx = a(x, t)dt + b(x, t)dz
• This is not technically an equation, but a way of saying • Drift and Standard Deviation terms: dx = a(x, t)dt + b(x, t)dz | {z } | {z } drift standard deviation
30 We think of dz as increments of our Brownian motion from Weiner process/limit case of discrete time process. What is telling us about variance is whatever term is multiplying dz. The process changes as z changes, and what you want to know is how does a little change in zthis background Weiner processcause a change in x. In this case, an instantaneous change in z causes a change in x with scaling factor b.
• Deterministic and Stochastic terms: dx = a(x, t)dt + b(x, t)dz | {z } | {z } deterministic stochastic
Ito Process: 2 Key Examples
• Random Walk
dx = αdt + σdz
Random Walk with drift α and variance σ2
• Geometric Random Walk
dx = αxdt + σxdz
Geometric Random Walk with proportional drift α and proportional variance σ2 ∗ Drift and variance are now dependent on the state variable x Can also think of as dx , so percent change in is a random walk ∗ x = αdt + σdz dx **Note on Geometric Random Walk: If it starts above zero, if will never become negative (or even become zero): even with strongly negative drift, given its proportionality to x the process will be driven asymptotically to zero at the lowest. Laibson: Asset returns are well approximated by Geometric Random Walk
Ito's Lemma Motivation is to work with functions that take Ito Process as an argument
Ito's Lemma
Basically just a taylor expansion
Theorem: Let z(t) be a Wiener Process. Let x(t) be an Ito Process with dx = a(x, t)dt + b(x, t)dz. Let V = V (x, t), then
∂V ∂V 1 ∂2V dV = dt + dx + b(x, t)2dt ∂t ∂x 2 ∂x2
31 ∂V ∂V 1 ∂2V ∂V = + a(x, t) + b(x, t)2 dt + b(x, t)dz ∂t ∂x 2 ∂x2 ∂x [the second line simply follows from expanding dx = a(x, t)dt + b(x, t)dz]
Motivation: Work with Value Functions (which will be Ito Processes) that take Ito Processes as Arguments
• We are going to want to work with functions (value functionsthe solutions of Bellman equations) that are going to take as an argument, an Ito Process
• Motivation is to know what these functions look like, that take x as an argument, where x is an Ito Process
More speci cally, we'd like to know what the Ito Process looks like that characterizes the value function V . ∗ Value function V is 2nd Ito Process, which takes an Ito Process (x) as an argument · Not only want to talk about Ito Process x, but also about Ito Process V
• Since we are usually looking for a solution to a question: i.e. Oil Well
Price of oil is given by x The Ito Process dx = a(x, t)dt + b(x, t)dz describes the path of x, the price of oil (not the oil well) ∗ e.g. price of oil is characterized by geometric Brownian motion: dx = αxdt + σxdz V (x, t) is value of Oil Well, which depends on the price of oil x and time ∗ Want to characterize the value function of the oil well, V (x, t), where x follows an Ito Process · Ito Process that characterizes value function over time: dV =a ˆ(V, x, t)dt+ˆb(V, x, t)dz · But we will see that the dependence of aˆ and ˆb on V will drop We want to write the stochastic process which describes the evolution of V :
dV =a ˆ(x, t)dt + ˆb(x, t)dz ∗ which is called the total dierential of V ∗ Note that the hatˆ terms (ˆa, ˆb) are what we are looking for in this stochastic process, which are dierent than the terms in the Ito Process (a, b)
• Ito's Lemma gives the solution to this problem in the following form:
∂V ∂V 1 ∂2V ∂V ∂V 1 ∂2V ∂V dV = dt+ dx+ b(x, t)2dt = + a(x, t) + b(x, t)2 dt+ b(x, t)dz where: ∂t ∂x 2 ∂x2 ∂t ∂x 2 ∂x2 ∂x
∂V ∂V 1 ∂2V ∂V dV = + a(x, t) + b(x, t)2 dt + b(x, t) dz ∂t ∂x 2 ∂x2 ∂x | {z } | {z } aˆ ˆb
32 Proof of Ito's Lemma
Proof: Using a Taylor Expansion
∂V 1 ∂2V ∂V 1 ∂2V ∂2V dV = dt + (dt)2 + dx + (dx)2 + dxdt + h.o.t. ∂t 2 ∂t2 ∂x 2 ∂x2 ∂x2
3 • Any term of order (dt) 2 or higher is small relative to terms of order dt.
2 3 [This is because (∆t) is in nitely less important than (∆t) 2 , which is in nitely less impor- tant than (∆t)] √ • Note that (dz)2 = dt. Since ∆z ∝ ∆t, recalling ∆z ≈ h [there's a Law of Large Numbers making this true, this is math PhD land and not something Laibson knows or expects us to know]
Hence we can eliminate: ∗ (dt)2 = h.o.t. ∗ dxdt = a(x, t)(dt)2 + b(x, t)dzdt = h.o.t. ∗ (dx)2 = b(x, t)2(dz)2 + h.o.t. = b(x, t)2dt + h.o.t. Combining these results we have:
∂V ∂V 1 ∂2V dt + dx + b(x, t)2dt ∂t ∂x 2 ∂x2 dz is a Wiener process that depends only on t through random motion dx is a Ito process that depends on x and t through random motion V (x, t) is a value function (and an Ito Process) that depends on a Ito process dx and directly on t
Intuition: Drift in V through Curvature of V : Even if we assume [no drift in Ito Process] and assume ∂V [holding xed, • a(x, t) = 0 ∂t = 0 x V doesn't depend on t]
We still have 1 ∂2V 2 • E(dV ) = 2 ∂x2 b(x, t) dt 6= 0 If V is concave, V is expected to fall due to variation in x [convex → rise] Ex: , hence 0 1 and 00 1 ∗ V (x) = ln x V = x V = − x2 · dx = αxdt + σxdz h ∂V ∂V 1 ∂2V 2i ∂V · dV = ∂t + ∂x a(x, t) + 2 ∂x2 b(x, t) dt + ∂x b(x, t)dz h 1 1 2i 1 · = 0 + x αx − 2x2 (σx) dt + x σxdz 1 2 · = α − 2 σ dt + σdz =⇒ Growth in V falls below α due to concavity of V
• Intuition for proof : for Ito Processes, (dx)2 behaves like b(x, t)2dt, so the eect of concavity is of order dt and can not be ignored when calculating the dierential of V
Because of Jensen's inequality, if you are moving on independent variable (x) axis randomly, expected value will go down
33 Lecture Summary Talked about a continuous time process (Brownian motion, Weiner process or its generalization as an Ito Process) that has these wonderful properties that is kind of like the continuous time analog of things in discrete time are thought of as random walks. But it is even more general that this: Has perverse property that it moves in nitely far over unit time, but when you look at how it changes over any discrete interval of time, it looks very natural. Also learned how to study functions that take Ito Processes as arguments: key tool in these processes is Ito's Lemma. It uses a 2nd order taylor expansion and throws out higher order terms. So we can characterize drift in that function V very simply and powerfully, which we put to use in next lectures.
Lecture 9
Continuous Time Bellman Equation & Boundary Conditions
Continuous Time Bellman Equation
Final Form:
ρV (x, t)dt = max {w(u, x, t)dt + [dV ]} u E Inserting Ito's Lemma
∂V ∂V 1 ∂2V ρV (x, t)dt = max w(u, x, t)dt + + a + b(x, t)2 dt u ∂t ∂x 2 ∂x2 Next 1.5 lectures about solving for V using this equation with Ito's Lemma. First, Merton. Then, stopping problem.
• Intuition
ρV (x, t)dt = max w(u, x, t)dt + E [dV ] | {z } u | {z } | {z } required rate of return instantaneous dividend instantaneous capital gain
required rate of return = instantaneous return; instantaneous capital gain = instantaneous change in the program.
• Derivation
Towards Condensed Bellman ∗ Let w(x, u, t) = instantaneous payo function · Arguments: x is state variable, u is control variable, t is time ∗ Let x0 = x+∆x and t0 = t+∆t . We'll end up thinking about what happens as ∆t → 0 · Just write down Bellman equation as we did in discrete time
34 n o V (x, t) = max w(x, u, t)∆t + (1 + ρ∆t)−1 EV (x0, t0) u V (x, t) = max w(x, u, t)∆t + (1 + ρ∆t)−1 EV (x0, t0) u | {z } | {z } ow payo ×time step discounted value of continuation ∗ 3 simple steps algebra with (1 + ρ∆t) term:
(1 + ρ∆t) V (x, t) = max {(1 + ρ∆t) w(x, u, t)∆t + EV (x0, t0)} u
ρ∆tV (x, t) = max {(1 + ρ∆t) w(x, u, t)∆t + EV (x0, t0) − V (x, t)} u n o ρ∆tV (x, t) = max w(x, u, t)∆t + ρw(x, u, t) (∆t)2 + EV (x0, t0) − V (x, t) u ∗ Let ∆t → 0 . Terms of order (dt)2 = 0
ρV (x, t)dt = max {w(x, u, t)dt + [dV ]} (*) u E Substitute using Ito's Lemma
∗ Substitute for E [dV ] · where
∂V ∂V 1 ∂2V ∂V dV = + a(x, u, t) + b(x, u, t)2 dt + b(x, u, t)dz ∂t ∂x 2 ∂x2 ∂x ∂V since bdz = 0 E ∂x ∂V ∂V 1 ∂2V [dV ] = + a + b2 dt E ∂t ∂x 2 ∂x2 ∗ Substituting this expression into (*) we get
∂V ∂V 1 ∂2V ρV (x, t)dt = max w(u, x, t)dt + + a + b(x, t)2 dt u ∂t ∂x 2 ∂x2 · which is a partial dierential equation (in x and t)
• Interpretation Sequence Problem:
∞ e−ρtw (x(t), u(t), t) dt ˆt Interpretation of Bellman (*) before substuting for Ito's Lemma:
ρV (x, t)dt = max w(x, u, t) + E [dV ] | {z } u | {z } | {z } required rate of return instantaneous dividend instantaneous capital gain
35 ∗ note that instantaneous capital gain ≡ instantaneous change in the program ∗ And when we plug in Ito's Lemma ∗ ∂V ∂V 1 ∂2V Instantaneous capital gain = + a + b(x, t)2 ∂t ∂x 2 ∂x2
Application 1: Merton's Consumption
• Merton's Consumption
Consumer has 2 assets ∗ Risk free: return r ∗ Equity: return r + π with proportional variance σ2 Invests share θ in assets, and consumes at rate c
• Ito Process that characterizes dynamics of x, her cash on hand:
dx = [rx + πxθ − c] dt + θxσdz
Intuition ∗ Drift term: earn rxdt every period on assets. A fraction θx is getting an additional equity premium π (πxθdt). Consuming at rate c, so depletes assets at rate cdt. ∗ Standard Deviation term: cost to holding equities is volatility. Volatility scales with the amount of assets in risky category: θxσdz. So, a = [rx + πxθ − c] and b = θxσ aka equation of motion for the state variable
• Distinguishing Variables
x is a stock (not in the sense of equity, but a quantity) c is a ow ( ow means rate). ∗ Need to make sure units match (usually use annual in economics) ∗ i.e. spending $1000/48hrs is ~$180,000/year · He can easily consume at a rate that is above his total stock of wealth ∗ c is bounded between 0 and ∞ · Not between 0 and x as it was in discrete time
• Bellman Equation
∂V 1 ∂2V ρV (x, t)dt = max w(u, x, t)dt + [rx + πxθ − c] + (θxσ)2 dt c,θ ∂x 2 ∂x2
Note the maximization over both c and θ, so two FOCs ∗ In world of continuous time, these can be changed instantaneously.
36 · (In principle, in every single instant, could be picking a new consumption level c and asset allocation level θ) · It will turn out that in CRRA world, he will prefer a constant asset allocation θ, even though he is constantly changing his consumption V doesn't depend directly on t, so rst term was removed ∗ If someone was nitely lived, then function would depend on t dt can be removed since notationally redundant
Solving with CRRA utility c1−γ [Lecture] u(c) = 1−γ • It turns out that in this problem, the value function inherits similar properties to the utilty function: x1−γ V (x) = ψ 1−γ
If given utility function is CRRA c1−γ , then it can be proved that x1−γ • u(c) = 1−γ V (x) = ψ 1−γ
This is just a scaled version of the utility function (by ψ) This gives us single solution to Bellman equation
Solving for Policy Functions [given value function x1−γ ] • V (x) = ψ 1−γ
Assume x1−γ : nd values for policy functions that solve Bellman Equation • V (x) = ψ 1−γ We need to know ψ, θ and optimal consumption policy rule for c The restriction x1−γ allows us to do this V (x) = ψ 1−γ
Taking x1−γ , re-write Bellman plugging in for c1−γ , ∂V , ∂2V • V (x) = ψ 1−γ u(c) = 1−γ ∂x ∂x2
x1−γ n h γ i o ρψ dt = max u(c)dt + ψx−γ [(r + θπ)x − c] − ψx−γ−1(θσx)2 dt 1 − γ c,θ 2
x1−γ c1−γ h γ i ρψ = max + ψx−γ [(r + θπ)x − c] − ψx−γ−1(θσx)2 1 − γ c,θ 1 − γ 2
• Two FOCs, θ and c :
−γ γ −γ−1 2 FOCθ : ψx πx − 2 2 ψx θ(σx) = 0 0 −γ −γ FOCc : u (c) = c = ψx
• Simplifying − 1 c = ψ γ x π θ = γσ2 Interpretation: Nice result: Put more into equities the higher the equity premium π; put less into equities with higher RRA γ and higher variability of equities σ2 Why is variance of equities replacing familiar covariance term from Asset Pricing lecture? In Merton's world, all the stocasticity is coming from equity returns.
37 • Plugging back in, we get: 1 2 − γ ρ 1 π ψ = γ + 1 − γ r + 2γσ2 • Special case: γ = 1 implies c = ρx
So MPC ' .05 (Marginal Propensity to Consume) But we also get π 0.06 θ = γσ2 = 1(0.16)2 = 2.34 Saying to borrow money so have assets of 2.34 in equity [and therefore 1.34 in liabilities] We don't know what's wrong here. This model doesn't match up with what wise souls think you should do.
• Can we x by increasing γ? Remember no one in classroom had γ ≥ 5
With , 0.06 γ = 5 θ = 5(0.16)2 = 0.47 ∗ Recommends 47% allocation into equities Typical retiree: No liabilities; $600,000 of social security (bonds), $200,000 in house, less than $200,000 in 401(k) 401(k) usually 60% bonds and 40% stocks Equity allocation is 8% of total assets
Solving with Log utility u(c) = ln(c) [PS5] • Given that guess for value function is V (x) = ψ ln(x) + φ
n h 2 i o Set up is the same: ∂V 1 ∂ V 2 • ρV (x, t)dt = maxc,θ w(u, x, t)dt + ∂x [rx + πxθ − c] + 2 ∂x2 (θxσ) dt • When substitute for V (x) and u(c):
1 ρ [φ + ψ ln(x)] = max ln(c) + [(r + θπ)x − c] ψx−1 − ψ(θσx)2ψx−2 c,θ 2 • FOCs:
1 FOCc : c = ψ x π FOCθ : θ = σ2 • Plugging back into Bellman ψπ2 ρφ + ρψ ln(x) = ψr − ln(ψ) − 1 + 2σ2 + ln(x) • Observation to solve for parameters
We can re-write the above with just constants on RHS: ρψ ln(x) − ln(x) = ψr − ln(ψ) − 1 + ψπ2 2σ2 − ρφ ∗ Given that the RHS is constant, this will only hold if ρψ = 1. [Otherwise LHS would be increasing in x, while RHS won't increase, a contradiction] Hence, and ψπ2 . This means: · ρψ = 1 0 = ψr − ln(ψ) − 1 + 2σ2 − ρφ The Bellman equation is satis ed ∀x i and ψπ2 ∗ ρψ = 1 ρφ = ψr − ln(ψ) − 1 + 2σ2
38 Giving us
1 ∗ ψ = ρ 1 r π2 ∗ φ = ρ ρ + ln(ρ) − 1 + 2ρσ2
• Optimal Policy Plugging back into FOCs and π ∗ c = ρx θ = σ2
General Continuous Time Problem Most General Form
• Equation of motion of the state variable
dx = a(x, u, t)dt + b(x, u, t)dz, Note dx depends on choice of control u
• Using Ito's Lemma, Continuous Time Bellman Equation
∗ ∂V ∂V ∗ 1 ∂2V ∗ 2 ρV (x, t) = w(x, u , t) + ∂t + ∂x a(x, u , t) + 2 ∂x2 b(x, u , t) u∗ = u(x, t) =optimal value of control variable
Properties: PDE and Boundary Conditions
• Value function is a 2nd order PDE • V is the 'dependent' variable; x and t are the 'independent' variables • PDE will have continuum of solutions Need structure from economics to nd the right solution to this equationthe solution the makes economic sensefrom the in nite number of solutions that make mathematical sense. =⇒ These restrictions are called boundary conditions, which we need to solve PDE =⇒ Need to use economic logic to derive boundary conditions Name of the game in these problems is to use economic logic to nd right solution, and eliminate all other solutions (Any policy generates a solution, we're not looking for any solution that corresponds to any policy, but for the solution that corresponds to the optimal policy)
• [In Merton's problem, we exploited 'known' functional form of value function, which imposes boundary conditions]
Two key examples of boundary conditions in economics 1. Terminal Condition
• Suppose problem ends at date T
39 i.e. have already agreed to sell oil well on date T • Then we know that V (x, T ) = Ω(x, T ) ∀x [Note this is at date T ] Where Ω(x, t) is (some known, exogenous) termination function. Note that it can be state dependenti.e. change with the price of oil. • Can solve problem using techniques analogous to backward induction • This restriction gives enough structure to pin down solution to the PDE
2. Stationary ∞−horizon problem
• Value function doesn't depend on t =⇒ becomes an ODE
1 ρV (x) = w(x, u∗) + a(x, u∗)V 0 + b(x, u∗)2V 00 2 • Note the use of V 0 since now only one argument for V , can drop the ∂ notation • We will use Value Matching and Smooth Pasting
Motivating Example for Value Matching and Smooth Pasting:
• Consider the optimal stopping problem: V (x, t) = max w(x, t)∆t + (1 + ρ∆t)−1EV (x0, t0), Ω(x, t)
• where Ω(x, t) is the termination payo in the stopping problem
the solution is characterized by the stopping rule: ∗ if x > x∗(t) continue; if x ≤ x∗(t) stop ∗ Motivation: i.e. irreversibly closing a production facility Assume that dx = a(x, t)dt + b(x, t)dz In continuation region, value function characterized using Ito's Lemma:
h ∂V ∂V 1 ∂2V 2i ∗ ρV (x, t)dt = w(x, t)dt + ∂t + ∂x a(x, t) + 2 ∂x2 b(x, t) dt
Value Matching
• Value Matching: All continuous problems must have continuous value functions
V (x, t) = Ω(x, t)
for all x, t such that x(t) ≤ x∗(t)
(at boundary and to left of boundary)
40 A discontinuity is not permitted, otherwise one could get a better payo from not following the policy Intuition: an optimal value function has to be continuous at the stopping boundary. ∗ If there was a discontinuity, can't be optimal to quit (if V is above Ω) or to have continued so far (if V below Ω) See graph [remember V on y axis, x on x-axis]
Smooth Pasting
Derivatives with respect to x must be equal at the boundary. Using problem above:
• Smooth Pasting:
∗ ∗ Vx (x (t), t) = Ωx (x (t), t) Intuition ∗ V and Ω must have same slope at boundary. · Proof intuition: if slopes are not equal at convex kink, then can show that the policy of stopping at the boundary is dominated by the policy of continuing for just another instant and then stopping. (It doesn't mean the latter is the optimal policy, but shows that initial policy of stopping at the boundary is sub-optimal) ∗ If value functions don't smooth paste at x∗(t), then stopping at x∗(t) can't be optimal: ∗ Better to stop an instant earlier or later √ · If there is a (convex) kink at the boundary, then the gain from waiting is in ∆t and the cost from waiting is in ∆t. So there can't be a kink at the boundary.
41 ∗ Think of graph [remember V on y axis, x on x-axis]
Will need a 3rd boundary condition from the following lecture
Lecture 10
Third Boundary Condition, Stopping Problem and ODE Strategies
Statement of Stopping Problem
• Assume that a continuous time stochastic process x(t) is an Ito Process,
dx = adt + bdz
i.e. x is the price of a commodity produced by this rm
• While in operation, rm has ow pro t ` w(x) = x
(Or w(x) = x − c if there is a cost of production)
• Assume rm can costlessly (permanently) exit the industry and realize termination payo
Ω = 0
• Intuitively, rm will have a stationary stopping rule
if x > x∗ continue if x ≤ x∗ stop (exit)
• Draw the Value Function (Ω = 0 in below graph)
42 Characterizing Solution as an ODE
• Overall Value Function
V (x) = max x∆t + (1 + ρ∆t)−1EV (x0), 0 Where x0 = x(t + ∆t), and ρ is interest rate • In stopping region V (x) = 0 • In continuation region General
ρV (x)dt = xdt + E(dV ) Since: V (x) = x∆t + (1 + ρ∆t)−1EV (x0) =⇒ (1 + ρ∆t)V (x) = (1 + ρ∆t)x∆t + EV (x0) =⇒ ρ∆tV (x) = (1 + ρ∆t)x∆t + EV (x0) − V (x). Let ∆t → 0 and multiply out. Terms of order (dt)2 = 0. Substitute for E(dV ) using Ito's Lemma ∗ 2nd Order ODE:
b2 ρV = x + aV 0 + V 00 2 Interpretation ∗ Value function must satisfy this dierential equation in the continuation region · We've derived a 2nd order ODE that restricts the form of the value function V in the continuation region · 2nd order because highest order derivative is 2nd order, ordinary because only depends on one variable (x),
drift from jensen's inequality drift in argument z }| { z}|{ b2 ρV = x + aV 0 + V 00 |{z} |{z} 2 required return dividend | {z } capital gain =how V is instantly drifting Term b2 00 is drift resulting from brownian motion'jiggling and jaggling': it is negative ∗ 2 V if concave, and positive if convex (as in this case).
• Intuition: x∗ almost always negative. [don't confuse with V (x) = Ω = 0. This is the value you get when stopping; but can decide on some policy of x for when to stop: x∗] Why? Option value love the ability take the chance at future pro ts. Even if drift is negative, will still stay in, since through Brownian motion may end up pro table again.
• And now, we have continuum of solutions to ODE, so need restrictions to get to single solution
43 Third Boundary Condition Boundary Conditions *Three variables we need to pin down: two constants in 2nd order ODE, and free boundary (x∗) • Value Matching: V (x∗) = 0 • Smooth Pasting: V 0(x∗) = 0 • Third Boundary Condition for large x
As x → ∞ the option value of exiting goes to zero ∗ It is so unlikely that pro ts would become negative (and if it happened, would likely be very negatively discounted) that value of being able to shut down is close to zero ∗ As x gets very large, value function approaches that of the rm that does not have the option to shut down (which would mean it would theoretically need to keep running at major losses) Hence V converges to the value associated with the policy of never exiting the industry [equation derived later]
V (x) lim = 1 x→∞ 1 a ρ x + ρ ∗ Denominator is value function of rm that can't shut down [Or can think of it as 1 a ] · limx→∞ V (x) = ρ x + ρ We can also write:
1 lim V 0(x) = x→∞ ρ ∗ Intuition: at large x, how does value function change with one more unit of price x? · Use sequence problem: · Having one more unit of price produces same exact sequence shifted up by one unit all the way to in nity. What is the additional value of a rm with price one unit higher? · ∞ 1 e−ρtx(t)dt − e−ρt(x0)(t)dt = e−ρt · 1 · dt = ˆ ˆ ˆ0 ρ | {z } | {z } old oil well new oil well So if I raise price by 1, I raise value of rm by 1 · ρ
Solving 2nd Order ODEs
De nitions
• Goal: Solve for F with independent variable x • Complete Equation Generic 2nd order ODE:
44 F 00(x) + A(x)F 0(x) + B(x)F (x) = C(x)
• Reduced Equation
C(x) is replaced by 0
F 00(x) + A(x)F 0(x) + B(x)F (x) = 0
Theorem 4.1
• Any solution Fˆ(x), of the reduced equation can be expressed as a linear combination of any two linearly independent solutions of the reduced equation, F1 and F2
Fˆ(x) = C1F1(x) + C2F2(x) provided F1,F2linearly independent
Note that two solutions are linearly independent if there do not exist constants A1 and A2 such that
A1F1(x) + A2F2(x) = 0 ∀x
Theorem 4.2
• The general solution of the complete equation is the sum of
any particular solution of the complete equation and the general solution of the reduced equation
Solving Stopping Problem
1. State Complete and Reduced Equation
• Complete Equation
Dierential equation that characterizes continuation region
b2 ρV = x + aV 0 + V 00 2 • Reduced equation
b2 0 = −ρV + aV 0 + V 00 2
2. Guess Form of Solution to nd General Solution to Reduced Equation
45 • First challenge is to nd solutions of the reduced equation • Consider class
erx
• To con rm this is in fact a solution, dierentiate and plug in to nd:
b2 0 = −ρerx + arerx + r2erx 2 This implies (dividing through by erx)
b2 0 = −ρ + ar + r2 2 • Apply quadratic formula
−a ± pa2 + 2b2ρ r = b2 • Let r+ represent the positive root and let r− represent the negative root • General solution to reduced equation Any solution to the reduced equation can be expressed
+ − C+er + C−er 3. Find Particular Solution to Complete Equation
Want a solution to the complete equation 0 b2 00 • ρV = x + aV + 2 V • Consider speci c example: payo function of policy never leave the industry • The value of this policy is
∞ E e−ρtx(t)dt ˆ0 • In solving for value of the policy without the expectation, we'll also need to know E [x(t)]:
t E [x(t)] = E x(0) + dx(t) ˆ0
t t = E x(0) + [adt + bdz(t)] = E x(0) + at + bdz(t) ˆ0 ˆ0 [ t as it is the Brownian motion term]. Hence: 0 bdz(t) = 0 ´ E [x(t)] = x(0) + at
• The value of the policy in V (x) = form, without the expectation
46 Derived by integration by parts
udv = uv − vdu ˆ ˆ
Taking ∞ −ρt : E 0 e x(t)dt ´ ∞ ∞ E e−ρtx(t)dt = E e−ρt [x(0) + at] dt ˆ0 ˆ0 ∞ ∞ x(0) ∞ = E e−ρt [x(0)] dt + e−ρt [at] dt = + E e−ρt [at] dt ˆ0 ˆ0 ρ ˆ0 Think of dv = e−ρt and u = at Then 1 −ρt and v = − ρ e du = adt So if ∞ −ρt then 1 −ρt ∞ ∞ 1 −ρt udv = 0 e (at)dt uv − vdu = [−at ρ e ]0 − 0 − ρ e adt Note ´ 1 −ρt ∞´ 1 ´ ´ − ρ e ]0 = ρ 1 −ρt ∞ 1 −ρt ∞ 1 = [−at ρ e ]0 − [ ρ2 e a]0 = 0 + a ρ2 1 a = x(0) + ρ ρ
• Hence our candidate particular solution takes the form
1 a V (x) = x + ρ ρ
Con rm that this is a solution to the complete equation 0 b2 00 • ρV = x + aV + 2 V
1 a a 1 b2 ρ x + = x + = x + a + · 0 ρ ρ ρ ρ 2
4. Put Pieces Together
• General solution to reduced equation
p 2 2 + − −a ± a + 2b ρ C+er x + C−er x with roots r+, r− = b2 • Particular solution to general equation
1 a x + ρ ρ
• Hence the general solution to the complete equation is
[Sum of particular solution to complete equation and general solution to reduced equation]
1 a + − V (x) = x + + C+er x + C−er x ρ ρ
47 • Now just need to apply boundary conditions
5. Apply Boundary Conditions
• 3 Conditions
Value Matching: V (x∗) = 0 Smooth Pasting: V 0(x∗) = 0 0 1 limx→∞ V (x) = ρ • This gives us three conditions on the general solution
+ [from 0 1 ] C = 0 limx→∞ V (x) = ρ ∗ Which then implies:
− ∗ ∗ 1 ∗ a − r x [Value Matching] (1) V (x ) = ρ x + ρ + C e = 0 0 ∗ 1 − − r−x∗ [Smooth Pasting] (2) V (x ) = ρ + r C e = 0
• Simplifying from (1) and (2)
Plugging (1) into (2)
− ∗ (1) implies − r x 1 ∗ a ∗ C e = − ρ x + ρ · Plugging this into (2) we get 1 − 1 ∗ a ∗ ρ − r ρ x + ρ = 0 Simplifying: − ∗ a 1 ∗ a 1 = r x + ρ =⇒ r− = x + ρ Gives us ∗ 1 a ∗ x = r− − ρ √ 2 2 Plugging in for − −a− a +2b ρ r = b2 We get our solution for x∗
b2 a x∗ = − − < 0 (∗) a + pa2 + 2b2ρ ρ
The general solution to the complete equation can be expressed as:
• the sum of a particular solution to the complete dierential equation and the general solution to the reduced equation
Note: this is usually done in search problems by looking for the particular solution of the policy never leave. Then you can get a general solution, and then use value matching and smooth pasting to come up with the threshold value.
48 Lecture 12
Discrete Adjustment - Lumpy Investment
Lumpy Investment Motivation
• Plant level data suggests individual establishments do not smooth adjustment of their capital stocks.
Instead, investment is lumpy. Ex rms build new plant all at once, not a little bit each year
largest investment episode, rm i • Doms & Donne 1993: 1972-1989: . If investment were spread total investment, rm i this ratio would be 1 . 18 Instead average value was 1 . Hence, on average rms did 25% of investment in 1 year over 4 18 year period
• We go from convex to a ne cost functions:
smooth convex cost functions assume small adjustments are costlessly reversible use a ne functions, where small adjustments are costly to reverse
Lumpy Investment
Capital Adjustment Costs Notation
• Capital adjustment has xed and variable costs
Upward Adjustment Cost: CU + cuI
∗ CU is xed cost, cU is variable cost
Downward Adjustment Cost: CD + cD (−I).
∗ CD + cD |I| is the general case. Here we usually assume cD > 0 and I < 0, so we get CD + cD (−I) ∗ CD is xed cost, cD is variable cost
Firm's Problem
• Firm loses pro ts if the actual capital stock deviates from target capital stock x∗
Deviations (x − x∗) generate instantaneous negative payo
• Functional form:
b b − (x − x∗)2 = − X2 2 2 where X = (x − x∗).
49 • X is an Ito Process between adjustments:
dX = αdt + σdz
where dz are Brownian increments
Value Function Representation
• Overall Value Function
( ∞ ) ∞ b X V (X) = max e−ρ(τ−t) − X2 dτ − e−ρ(τ(n)−t)A (n) E ˆ 2 τ τ=t n=1
C + c I if I > 0 A (n) = U U n n CD + cD |In| if In < 0 Notation:
τ (n) = date of nth adjustment; A (n) = cost of nth adjustment; In = investment in nth adjustment ∗ Below x∗: Adjust capital at X = U . Adjust to X = u. ∗ Above x∗: Adjust capital at X = D. Adjust to X = d.
• Value Function in all 3 Regions
1. Between Adjustments: Bellman Equation
b 1 ρV (X) = − X2 + αV 0 (X) + σ2V 00 (X) 2 2 h 2 i Using Ito's Lemma: ∂V ∂V 1 ∂ V 2 ∗ E [dV ] = ∂t + ∂x a + 2 ∂x2 b dt Functional form solved for below [PS6 2.d] 2. Action Region Below [If X ≤ U]
V (X) = V (u) − [CU + cU (u − X)] This implies
0 V (X) = cU ∀X ≤ U 1. Action Region Above [If X ≥ D]
V (X) = V (d) − [CD + cD (X − d)] This implies
0 V (X) = −cD ∀X ≥ D
50 Boundary Conditions 1. Value Matching
lim V (X) = V (u) − [CU + cU (u − U)] = V (U) = lim V (X) X↑U X↓U
lim V (X) = V (d) − [CD + cD (D − d)] = V (D) = lim V (X) X↓D X↑D 2. Smooth Pasting
0 0 0 lim V (X) = cU = V (U) = lim V (X) X↑U X↓U
0 0 0 lim V (X) = −cD = V (D) = lim V (X) X↓D X↑D
First Order Conditions
• Intuition: When making capital adjustment, willing to move until the marginal value of moving = marginal cost
This gives us two important optimality condtitions:
0 V (u) = cU
0 V (d) = −cD
51 Solving for Functional Form of V (X) in Continuation Region PS6 2.d
2 2 Question: Show that b X σ+2αX 2α is a solution to the continuation region equa- V (X) = − 2 ρ + ρ2 + ρ3 tion: b 2 0 1 2 00 . ρV (X) = − 2 X + αV (X) + 2 σ V (X) Show that this is the expected present value of the rm's payo stream assuming that adjustment costs are in nite.
This is guess and check, where we've been provided the guess:
b X2 σ + 2αX 2α2 V (X) = − + + 2 ρ ρ2 ρ3 Calculate V 0(X) and V 00(X):
b 2X 2α2 b V 0(X) = − + ,V 00(X) = − 2 ρ ρ2 ρ We plug back into the continuous time Bellmann Equation from 1 −b 2 0 1 2 00 : V (X) = ρ 2 X + αV (X) + 2 σ V (X) 1 −b b 2X 2α2 1 b b 2αX σ2 2α 2 V (X) = X2 − α + − σ2 = − X2 + + + ρ 2 2 ρ ρ2 2 ρ 2ρ ρ ρ ρ2
52 b X2 σ2 + 2αX 2α = − + + 2 ρ ρ2 ρ3 So our guessed solution works as a solution to the continuous Bellmann.
We also know that this works as the expected present value of the rm's payo stream, given that this is the equivalent to the solution to the continuous time Bellman for the continuation region, because the in nite adjustment costs mean that u = −∞ and d = ∞, hence the solution will hold for X ∈ R.
Appendix: Math Reminders
In nite Sum of a Geometric Series with |x| < 1:
∞ X 1 xk = 1 − x k=0
L'Hopital's Rule
If lim f(x) = lim g(x) = 0 or ± ∞ x→c x→c
f 0(x) and lim exists x→c g0(x)
and g0(x) 6= 0
f(x) f 0(x) then lim = lim x→c g(x) x→c g0(x)
Probability
Expectation of RV: 1 . E(exp(˜a)) = exp[E[˜a] + 2 V ar[˜a]] When a˜ is a random variable, or anything x ∼ N(µ, σ2) −→Ax + B ∼ N(Aµ + B,A2σ2) 1 2 2 −→ E [exp (Ax + B)] = exp Aµ + B + 2 A σ [Since 2 exp exp 1 ; so if standard normal: 1 2 ] x ∼ N(µ, σ )−→ E( (x)) = (E(x) + 2 V ar(x)) = exp(µ + 2 σ ) In case of Asset pricing t has stochastic returns Ri We have: t t i i 1 i 2 Ri = exp(ri + σ εt − 2 [σ ] ) where i has unit variance. Hence i is a normally distributed random variable: i εt εt εt ∼ N(0, 1) From above we see that this implies: i i t 1 i 2 i t 1 i 2 i2 2 t σ εt + ri − 2 [σ ] ∼ N(σ ∗ 0 + ri − 2 [σ ] , σ ∗ 1 ) = N(ri − 1 i 2 i2 2 [σ ] , σ )
53 Then: t t i i 1 i 2 which as we've seen is now exp(a random variable) so we can use E[Ri] = E[exp(ri + σ εt − 2 [σ ] )] 1 t 1 i 2 1 i2 = exp(mean + 2 V ar) = exp(ri − 2 [σ ] + 2 σ ) t = exp(ri )
Covariance: If A and B are random variables, then
V (A + B) = V (A) + V (B) + 2Cov(A, B)
Log Approximation: For small x: ln(1 + x) ≈ x 1 + x ≈ ex
54