0.1. TANGENT SPACES AND LAGRANGE MULTIPLIERS 1 0.1 Tangent Spaces and Lagrange Multipliers ~ n+k k If a di®erentiable function G = (G1;:::;Gk): E ! E then the surface S de¯ned by S = f~x j G~ (~x) = ~vg is called the level surface for G~ (~x) = ~v. Note that each of the functions n+k Gi : E ! R. If we denote by Si the level surface for the equation Tk Gi(~x) = vi, then S = i=1 Si. 0 0 0 ~ 0 0 n+k k Suppose that ~x = (x1; : : : ; xn+k) 2 S and that G (~x ) 2 L(E ; E ) has rank k. Let ±i;j = 1 if i = j and 0 if i 6= j. With respect to the standard basis f~ej = (±1;j; : : : ; ±n+k;j) j j = 1; 2; : : : ; n + kg for En+k and the analogous smaller basis for Ek, we note that the matrix ~ 0 0 [G (~x )]k£(n+k) has its whole set of k row vectors linearly independent, and 0 0 0 these row vectors are the gradient vectors rG1(~x ); rG2(~x );:::; rGk(~x ). Let Á~ : R ! S be a di®erentiable function for which Á~(0) = ~x0. Then we call the vector ~v = Á~0(0) a tangent vector to S at ~x0. 0 De¯nition 0.1.1. The tangent space T~x0 (S) at ~x 2 S is the set of all tangent vectors to S at ~x0. The translate 0 0 ~x + T~x0 = f~x + ~v j ~v 2 T~x0 (S)g is called the tangent plane to the surface S, with point of tangency at ~x0. A translate ~a + V = f~a + ~x j ~x 2 V g of a vector subspace V of En is called an a±ne subspace of En. An a±ne subspace is a vector subspace if and only if ~a 2 V . (See Exercise 1.) Theorem 0.1.1. Let G~ : En+k ! Ek be a di®erentiable function. Let ~x0 2 S = f~x j G~ (~x) = ~vg: ~ 0 0 Suppose G (~x ) has rank k. Then the tangent space T~x0 (S) is the vector subspace ¡ 0 0 ¢? T~x0 (S) = spanRfrG1(~x );:::; rGk(~x )g of dimension n. In words, T~x0 (S) is the orthogonal complement of the span 0 0 of the k gradient vectors rG1(~x );:::; rGk(~x ). 2 ~0 ~ Proof. Suppose ¯rst that ~v = Á (0) 2 T~x0 (S). This implies that Á maps 0 into each level surface Gi(~x) = vi. We will show that ~v ? rGi(~x ) for ~ each i = 1; : : : ; k. In fact, Gi(Á(t)) ´ vi, a real constant. We di®erentiate 0 ~ ~0 using the Chain Rule to ¯nd that Gi(Á(0))Á (0) = 0. In terms of the matrix 0 ~0 representation of the left side of the latter equation, we have rGi(~x )¢Á (0) = 0 0 ? 0, so that ~v ? rGi(~x ). This shows that T~x0 (S) ⊆ rGi(~x ) for each i. This implies that ¡ 0 0 ¢? T~x0 (S) ⊆ spanRfrG1(~x );:::; rGk(~x )g : ~ 0 0 The hypothesis that rank(G (~x )) = k implies that dimT~x0 (S) · n. If we can show that the tangent space is at least n-dimensional, then it will have to be the entire orthogonal complement of the span of the gradient vectors as claimed. Thus it will su±ce to produce a linearly independent set of n vectors in the tangent space. Because the rank of a matrix is also the number of linearly indepen- dent column vectors, it follows that the matrix [G~ 0(~x0)] has k independent columns. We can rearrange the order of the n elements of the standard basis of En+k to arrange that the ¯rst k columns are linearly independent. By the Implicit Function Theorem, there exists an open set U ½ Ek containing 0 0 n 0 0 (x1; : : : ; xk) and an open set V ½ E containing (xk+1; : : : ; xk+n) such that there are unique di®erentiable functions x1 = Ã1(xk+1; : : : ; xk+n) . xk = Ãk(xk+1; : : : ; xk+n) solving the equation ~ G(Ã1(xk+1; : : : ; xk+n);:::;Ãk(xk+1; : : : ; xk+n); xk+1; : : : ; xk+n) = ~v: Next we de¯ne n di®erentiable curves on S by the equations ~ 0 0 0 0 0 0 0 0 0 Á1(t) = (Ã1(xk+1 + t; xk+2; : : : ; xk+n);:::;Ãk(xk+1 + t; xk+2; : : : ; xk+n); xk+1 + t; xk+2; : : : ; xk+n) . ~ 0 0 0 0 0 0 0 0 0 Án(t) = (Ã1(xk+1; : : : ; xn+k¡1; xk+n + t);:::;Ãk(xk+1; : : : ; xk+n¡1; xk+n + t); xk+1; : : : xk+n¡1; xk+n + t) ~0 In comparing the vectors Ái(0) for i = 1; : : : ; n, observe that for each of these vectors the ¯nal n entries are all 0 except for a single entry which is 1. The location of the 1 is di®erent for each of these vectors. Thus the n vectors are independent and the theorem is proved. 0.1. TANGENT SPACES AND LAGRANGE MULTIPLIERS 3 Corollary 0.1.1. Let G~ : Ek+n ! Ek be a di®erentiable function and let ~x0 2 S = f~x j G~ (~x) = ~vg: Suppose ~x0 is a local extreme point of a di®erentiable function f : S ! R ~ 0 0 and that G (~x ) has rank k. Then there exist numbers ¸1; : : : ; ¸k such that 0 0 0 rf(~x ) = ¸1rG1(~x ) + ¢ ¢ ¢ + ¸krGk(~x ) (1) The numbers ¸1; : : : ; ¸k are called Lagrange multipliers. Proof. If Á~ : R ! S is a di®erentiable curve on S with Á~(0) = ~x0, let Ã(t) = f(Á~(t)). Since this function has an extreme point at 0, we have Ã0(0) = rf(Á~(0)) ¢ Á~0(0) = 0: It follows from Theorem 0.1.1 that rf(~x0) is orthogonal to the tangent space 0 T~x0 (S). Since the co-dimension of T~x0 (S) is k, it follows that rf(~x ) lies in 0 0 the span of the k vectors rG1(~x );:::; rGk(~x ). This proves the corollary. The method of Lagrange multipliers permits an optimization problem to be replaced by a problem of solving a system of equations. From the k + n components of the vectors in Equation 1, we obtain a system of k + n equations in the n+2k unknowns x1; : : : ; xk+n; ¸1; : : : ; ¸k. We get k additional equations from the k components of the equation G~ (~x) = ~v. Thus we obtain a system of n+2k equations in n+2k unknowns. Although we have replaced a calculus problem with an algebraic problem, the algebraic problem can be challenging. Nevertheless, the method of Lagrange multipliers is a powerful tool for optimization problems. Example 0.1.1. We will begin with a three-dimensional example. Consider the surface S de¯ned by the equation x4 +y4 +z4 = 1 in E3, shown in Figure 1. We will ¯nd both the maximum and the minimum values of the function f(~x) = x2+y2+z2 on S. (In e®ect, we are determining the closest and furthest distances from the origin on S.) In this example, we denote ~x = (x; y; z). Observe that if we de¯ne G(~x) = x4 + y4 + z4 then S = G¡1(f1g). Hence S is closed because G is continuous. S is also bounded. (Why?) Hence the function f must achieve both a maximum and a minimum value somewhere on S. Since S is smooth at all points and since rG is non-vanishing on S, 4 1 y 0.5 0 -0.5 -1 1 0.5 z 0 -0.5 -1 -1 -0.5 0 x 0.5 1 Figure 1: x4 + y4 + z4 = 1 the extreme points must occur at those points for which rf(~x) = ¸rG(~x). This yields the following system of equations. x(1 ¡ 2¸x2) = 0 y(1 ¡ 2¸y2) = 0 z(1 ¡ 2¸z2) = 0 x4 + y4 + z4 = 1 The reader should check the following by making the necessary calculations. ² If none of the three variables is zero, then x2 = y2 = z2 = 1 showing p p 2¸ 3 that ¸ = § 2 . This implies that f(x; y; z) = 3. ² If exactly one of the three variables is zero, then atp a point satisfying the system of equations we must have f(x; y; z) = 2. ² If exactly two of the variables are zero, then at a point satisfying the system we must have f(x; y; z) = 1. 0.1. TANGENT SPACES AND LAGRANGE MULTIPLIERS 5 p It follows that the maximum value of f on S is 3. But the reader should be able to explain why at least one of the variables must be non-zero. Thus the minimum value is 1. There is also an easy way to explain even from the outset why f(x; y; z) ¸ 1 everywhere on S. Exercises 0.1. 0 n 1. Prove that the tangent plane ~x + T~x0 (S) is a vector subspace of E if 0 and only if ~x 2 T~x0 (S). 2. Describe both the tangent space and the tangent³ plane to the´ sphere n¡1 n 0 p1 p1 p1 S = f~x 2 E j k~xk = 1g at the point ~x = n ; n ;:::; n . 3. The sphere S3 ½ E4 is de¯ned by ( ) X4 3 2 S = ~x j xi = 1: i=1 3 P4 De¯ne f : S ! R by f(~x) = i=1 aixi where ai is a constant for each i 2 f1; 2; 3; 4g.