Multivariate Calculus∗
Total Page:16
File Type:pdf, Size:1020Kb
Chapter 3 - Multivariate Calculus∗ Justin Leducy These lecture notes are meant to be used by students entering the University of Mannheim Master program in Economics. They constitute the base for a pre-course in mathematics; that is, they summarize elementary concepts with which all of our econ grad students must be familiar. More advanced concepts will be introduced later on in the regular coursework. A thorough knowledge of these basic notions will be assumed in later coursework. Although the wording is my own, the definitions of concepts and the ways to approach them is strongly inspired by various sources, which are mentioned explicitly in the text or at the end of the chapter. Justin Leduc I have slightly restructured and amended these lecture notes provided by my predecessor Justin Leduc in order for them to suit the 2016 course schedule. Any mistakes you might find are most likely my own. Simona Helmsmueller ∗This version: 2018 yCenter for Doctoral Studies in Economic and Social Sciences. 1 Contents 1 Multivariate functions 4 1.1 Invertibility . .4 1.2 Convexity, Concavity, and Multivariate Real-Valued Functions . .6 2 Multivariate differential calculus 10 2.1 Introduction . 10 2.2 Revision: single-variable differential calculus . 12 2.3 Partial Derivatives and the Gradient . 15 2.4 Differentiability of real-valued functions . 18 2.5 Differentiability of vector-valued functions . 22 2.6 Higher Order Partial Derivatives and the Taylor Approximation Theorems . 23 2.7 Characterization of convex functions . 26 3 Integral theory 28 3.1 Univariate integral calculus . 28 3.2 The definite integral and the fundamental theorem of calculus . 29 3.3 Multivariate extension . 29 A Appendix - Directional Derivatives 31 2 Preview What you should take away from this chapter: 1. Multivariate functions • Know when an inverse to a function exists and when it does not • Be able to graphically and analytically determine whether a function is convex, concave or quasiconcave. 2. Multivariate differential calculus • Have an intuition about the meaning of derivatives as measures of small change • Know all the calculation rules for derivatives (sum, quotient, product, chain rule) • Be able to calculate the Gradient, Jacobian and Hessian of a function (if defined) • Know the difference between partial and total derivatives • Be able to write down the Taylor approximation of a function • Know what a function's differentials tell us about its continuity and convexity 3. Integral theory • Know and be able to apply the calculation rules, including integration by parts • Know and be able to apply Fubini's theorem 3 1 Multivariate functions Before digging into the details of calculus, let us start with a chapter revising or introducing some concepts and key properties of functions in general. While not directly relevant for the following sections on differential and integral calculus, the definitions and theorems will be important for the chapter on optimization and for your following master classes. 1.1 Invertibility Before we start the process, let us make sure that our vocables are in line. In this chapter, the term vector is to be understood in its classical sense, i.e., as a column vector composed of real numbers, unless otherwise specified. The use of a row vector will thus be explicitly mentioned via a transposition symbol such as a 0. I required as a prerequisite that you be clear with the concept of a function, f, its domain, X, its codomain, Y , and its image, f(X). If Y ⊆ R, i.e., if f maps into the real line, we say that f is a real valued function. If X ⊆ R, i.e., if f takes real numbers as inputs, we call f n a function of a single variable, or univariate function. Finally, if X ⊆ R , n > 1, i.e., if f takes as input vectors of length n, n > 1, we call f a multivariate function. In this chapter, we will look at the insights from calculus and derive generalizations thereof to real-valued n multivariate functions, i.e., functions with domain R , n > 1 and codomain in R. Then, n m we will discuss functions going from R to R , with n > 1 and m > 1. Although we could further generalize the concepts for infinite-dimensional spaces (e.g. function spaces), and n this happens to be very close to the generalization for finite-dimensional spaces (e.g. R , n 2 N), we restrict ourselves to finite-dimensional vector spaces. The reason is twofold: (i) you will not be expected to be able to work with infinite dimensional spaces in the master's curriculum, (ii) if you grasp the generalization for finite dimensional spaces, then you will easily grasp that for infinite dimensional spaces as it is presented in books. Throughout the discussion, we will try to exploit geometric intuition as much as we can. The most common way to geometrically represent a function is to plot its graph. Let f be a function with domain X, codomain Y , and image f(X). Analytically, the graph of a function is a collection of ordered pairs (x; f(x)), with x 2 X and f(x) 2 Y . Hence, the graph is an element of the cartesian product X × Y , a space with dimension dim(X × Y ) = dim(X)+dim(Y ). In the next chapter, we will discuss a different geometrical representation of functions which is slightly less demanding in terms of dimension. Finally, let us come back to the promised discussion on inverse functions. Let f be a function with domain X, codomain Y , and image f(X). In the preliminary chapter, I said, that, under some conditions, one can define an inverse function f −1 mapping from Y to X, and such that f −1(f(x)) = x. What precisely are these conditions? There are two of them, both quite intuitive. The first one is that we want our function to be well defined over the 4 codomain Y , i.e., we want that it be defined for any element y in Y . This will not be the case, if the image of f, f(X) is a proper subset of Y , for then we would not know what value to associate to f −1(y), y 2 Y n f(X). A first natural condition, then, is to require that the image and the codomain of f coincide, i.e., f(X) = Y . If a function satisfies this condition, it is said to be onto1. In other words, if f is onto, then, for any element, y, that we pick in the codomain, Y , of f, there exists an x, in the domain, X, of f, such that f(x) = y. It is a condition on the codomain of f 2. Definition 1.1. (Surjective functions) A function f : X ! Y is said to be surjective or onto, if for every y 2 Y there exists at least one x 2 X with f(x) = y. The second requirement is intuitive too. Namely, we know that for any function f, if we take an element x in its domain X, then, by definition, there exists a unique y in the image, f(X), such that y = f(x). The converse, however, need not be true, i.e., take an element y in f(X) and consider its set of antecedents, i.e., the set of x such that f(x) = y, there is no a priori reason that this set be a singleton3. But if the converse is not true, then, once we consider the inverse mapping f −1 we will end up having several images associated to a single y in f(X). That, in turn, would disqualify our inverse map for the label of \function". The second condition, then is that the converse holds, i.e., that every distinct element x of the domain X maps into a distinct element y of the image f(X). If it is the case, we say that a function f is one-to-one4. Definition 1.2. (Injective functions) A function f : X ! Y is said to be injective or one-to-one, if the following property holds for all x1; x2 2 X: f(x1) = f(x2) ) x1 = x2: These definitions allow us to derive the following important theorem about the inverse function: Theorem 1.1. (Existence and definition of the inverse function) Let f : X ! Y be onto and one-to-one (i.e. bijective). Then there exists a unique bijective function f −1 : Y ! X with f −1(f(x)) = x. Conversely, if such a function exists, then f is bijective. 1a.k.a. surjective. 2Note that this is not a very demanding condition, as it suffices to properly choose the codomain of our function. 3A singleton is a set that contains a single element. 4a.k.a. injective 5 1.2 Convexity, Concavity, and Multivariate Real-Valued Func- tions Our second effort in this section is devoted to the notions of convexity and concavity of functions. Their importance stems from optimization and will thus be emphasized in the next chapter. For now, allow me to simply proceed with the formal discussion. Definition 1.3. (Convex Real Valued Function) n Let X ⊆ R . A function f : X ! R is convex if and only if X is a convex set and for any two x; y 2 X and λ 2 [0; 1] we have f(λx + (1 − λ)y) ≤ λf(x) + (1 − λ)f(y) Moreover, if this statement holds strictly whenever y 6= x and λ 2 (0; 1), we say that f is strictly convex. Definition 1.4. (Concave Real-Valued Function) n Let X ⊆ R .