Unit #23 : Lagrange Multipliers Goals: • to Study Constrained Optimization

Unit #23 : Lagrange Multipliers

Goals: • To study constrained optimization; that is, the maximizing or minimizing of a function subject to a constraint (or side condition). Constrained Optimization - Examples - 1

In the previous section, we saw some of the diﬃculties of working with optimization when there are multiple variables. Many of those problems can be cast into an important class of problems called constrained optimization problems, which can be solved in an alternative way.

Examples of Problems with Constraints

1. Show that the rectangle of maximum area that has a given perimeter p is a square. The function to be maximized: A(x, y) = xy The constraint: 2x + 2y = p Constrained Optimization - Examples - 2

2. Find the point on the sphere x2 + y2 + z2 = 4 that is closest to the point (3, 3, 5). Here we want to minimize the distance D(x, y, z) = p(x − 3)2 + (y − 3)2 + (z − 5)2 subject to the constraint x2 + y2 + z2 = 4 Lagrange Multipliers - 1

Lagrange Multipliers

To solve optimization problems when we have constraints on our choice of x and y, we can use the method of Lagrange multipliers. Suppose we want to maximize the function f(x, y) subject to the constraint g(x, y) = k. We consider the relative positions of some sample level curves on the next page. Lagrange Multipliers - 2

Black lines: contours of f(x, y). Blue lines: the constraint g(x, y) = k. f = 10 f = 20

f = 30

g(x,y)= k

f = 40

f = 50

f = 60

On the diagram, locate the maximum and minimum of f(x, y) (label with M and m respectively) on the level curve g(x, y) = k.

Draw in the gradient vectors ∇f and ∇g at M and m. Describe, in words, the relationship between ∇f(M) and ∇g(M), and also between ∇f(m) and ∇g(m). The Lagrange Multiplier Method - 1

Description of the Lagrange Multiplier Method

Step 1. Find all values of (x, y) and λ such that

∇f(x, y) = λ∇g(x, y)

g(x, y) = k

Step 2. Evaluate f at all the points (x, y) obtained in Step 1. (The largest value is the maximum and the smallest is the minimum.) The Lagrange Multiplier Method - 2

Note that Step 1 really amounts to solving three equations in three unknowns (x, y, and λ). The equations can be found by rewriting the gradient equations as follows:

fx(x, y) = λgx(x, y)

fy(x, y) = λgy(x, y)

g(x, y) = k

The same method applies to functions of three (or more) variables. In this case of three variables, we would solve four equations in four unknowns. Lagrange Multiplier Method - Linear Constraint - 1

Example: Consider the problem of ﬁnding the maximum and minimum values of x2 + y2 f(x, y) = x − 10y + , 20 subject to the constraint x + y = 10 . Sketch the graphical meaning of the constraint. y

x Lagrange Multiplier Method - Linear Constraint - 2

Write down the three equations obtained by the method of Lagrange multipliers.

Lagrange Multiplier Method - Linear Constraint - 3

Solve these equations, and compare the values at the resulting points to ﬁnd the maximum and minimum values. Lagrange Multiplier Method - Linear Constraint - 4

80 70 −350 −345 68 −330 70 −340 −360 66 −335 −355 60 64 −350 50 −345 62 −330 −340 −360 −335 40 60 −355

−350 58 30 −345 −330 −340 −360 56 −335 −355 20 54 −350 −345 10 52 −330 −340 −335

0 50 −90 −80 −70 −60 −50 −40 −30 −20 −10 0 10 −60 −55 −50 −45 −40 Lagrange Multiplier Method - Non-Linear Constraint - 1

Example: Find the points on the curve x4 + y4 = 1 that are closest to and furthest from the origin. Lagrange Multiplier Method - Non-Linear Constraint - 2

x4 + y4 = 1 Lagrange Multiplier Method - Non-Linear Constraint - 3

x4 + y4 = 1 Lagrange Multiplier Method - Non-Linear Constraint - 4

1.5 1.5

1 1

0.5 0.5

0 0

−0.5 −0.5

−1 −1

−1.5 −1.5 −1.5 −1 −0.5 0 0.5 1 1.5 −1.5 −1 −0.5 0 0.5 1 1.5 1.5

0.5

−0.5

−1

−1.5 −1.5 −1 −0.5 0 0.5 1 1.5 The Meaning of the Lagrange Multiplier - 1

The Meaning of the Lagrange Multiplier

λ is called the Lagrange multiplier. It has its own signiﬁcance, as can be seen from the following discussion.

• Recall that ||∇~f(x, y)|| represents the rate of increase of the value of f if you move from (x, y) in the direction indicated by the gradient of f. • Similarly, ||∇~g(x, y)|| represents the rate of increase of the value of g if you move in the direction of the gradient of g. If ∇~f(x, y) = λ∇~g(x, y) and λ > 0, then ∇~f(x, y) and ∇~g(x, y) have the same direction. (If λ < 0 they have opposite directions.) If the vectors are parallel, then the value of λ represents the ratio ||∇~f(x, y)|| |λ| = ||∇~g(x, y)|| The Meaning of the Lagrange Multiplier - 2

Thus, λ gives the rate of increase of f (our objective) divided by the rate of increase of g (our constraint value).

In other words, λ gives the approximate increase in the optimum value of f when the value of constraint g is increased by 1.

Example: f(x, y) is the output of a factory (number of units shipped), and g(x, y) = k is the manager’s budget (in thousands of dollars). If they ﬁnd the optimal production values x and y using Lagrange multipliers, and then ﬁnd λ = 29, what does this mean? Labour and Capital - Constrained Optimization - 1

Labour and Capital - Constrained Optimization

Suppose that the quantity q of a product depends on the number of workers, W , and the number of units of capital investment, K, and is represented by the Cobb- Douglas function q = 6W 3/4K1/4 In addition, labour costs are $10 per worker, capital costs are $20 per unit, and the budget is $3,200. We will ask several questions about this model. Use the method of Lagrange multipliers to ﬁnd the optimum number of workers and optimum number of units of capital. Labour and Capital - Constrained Optimization - 2

q = 6W 3/4K1/4 Labour and Capital - Interpreting the Lagrange Multiplier - 1

Labour and Capital - Interpreting the Lagrange Multiplier

The next part of the problem involves the marginal productivity of labour and the marginal productivity of capital. These concepts have to be trans- lated into mathematical terms before the problem can be attempted. The marginal productivity of labour refers to the extra amount that would be produced if W were increased by one. That is, it is the value of ∆q when ∆W = 1. This means it is also the same as ∆q ∆W Labour and Capital - Interpreting the Lagrange Multiplier - 2

Since a change of 1 is presumably very small compared to the value of W , we can ∆q ∂q use the fact that ≈ , and interpret the marginal productivity of labour ∆W ∂W ∂q to be the partial derivative . Similarly, the marginal productivity of capital ∂W ∂q should be interpreted as . ∂K Thus,

∂q marginal productivity of labour = ∂W

∂q marginal productivity of capital = ∂K Labour and Capital - Interpreting the Lagrange Multiplier - 3

Check that at the optimum values of W and K, the ratio of the marginal productivity of labour to the marginal productivity of capital is the same as the ratio of the cost of a unit of labour to the cost of a unit of capital. The Lagrange Multiplier - Labour and Capital - Part 2 - 1

Recompute the optimum values of W and K when the budget is increased by $1. Check that increasing the budget by $1 allows the production of λ extra units of the good, where λ is the Lagrange multiplier. The Lagrange Multiplier - Labour and Capital - Part 2 - 2

350 1400

2000

800

200

300 1200

1600

600

250 1000 1800

400

200 1400

800

200 1200 1600 150

600

1000 100 400

50 800 1200 1000 200 600 800 0 400200 400600200 0 50 100 150 200 250 300 350 60 980 40.8 922

55 960 1000 921 40.6 921.5922 940 920 50 980 40.4 920 920.5921 960 1000 919 45 40.2 921.5922 900 919.5920 940 918 980 880 920.5921 40 920 960 40 918.5919 921.5922 860 900 940 919.5920 35 39.8 918 840 880 920.5921 820 920 918.5919 39.6 30 860 900 800 919.5920 840 880 918 780 820 39.4 25 860 918.5919 760 800 840 740 780 820 39.2 20 215 220 225 230 235 240 245 250 255 260 265 239 239.2 239.4 239.6 239.8 240 240.2 240.4 240.6 240.8 241