DOCSLIB.ORG

Appendix D Matrix Calculus

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Appendix D Matrix Calculus Appendix D Matrix calculus From too much study, and from extreme passion, cometh madnesse. Isaac Newton [86, §5] − D.1 Directional derivative, Taylor series D.1.1 Gradients Gradient of a differentiable real function f(x) : RK R with respect to its vector domain is defined → ∂f(x) ∂x1 ∂f(x) ∆ K f(x) = ∂x2 R (1354) ∇ . ∈ . ∂f(x) ∂xK while the second-order gradient of the twice differentiable real function with respect to its vector domain is traditionally called the Hessian ; ∂2f(x) ∂2f(x) ∂2f(x) 2 ∂ x1 ∂x1∂x2 ··· ∂x1∂xK ∂2f(x) ∂2f(x) ∂2f(x) ∆ 2 K 2 ∂x2∂x1 ∂ x2 ∂x2∂x S f(x) = . ··· . K (1355) ∇ . ... ∈ . ∂2f(x) ∂2f(x) ∂2f(x) 2 1 2 ∂xK ∂x ∂xK ∂x ··· ∂ xK © 2001 Jon Dattorro. CO&EDG version 04.18.2006. All rights reserved. 501 Citation: Jon Dattorro, Convex Optimization & Euclidean Distance Geometry, Meboo Publishing USA, 2005. 502 APPENDIX D. MATRIX CALCULUS The gradient of vector-valued function v(x) : R RN on real domain is a row-vector → ∆ v(x) = ∂v1(x) ∂v2(x) ∂vN (x) RN (1356) ∇ ∂x ∂x ··· ∂x ∈ h i while the second-order gradient is 2 2 2 2 ∆ ∂ v1(x) ∂ v2(x) ∂ vN (x) N v(x) = 2 2 2 R (1357) ∇ ∂x ∂x ··· ∂x ∈ h i Gradient of vector-valued function h(x) : RK RN on vector domain is → ∂h1(x) ∂h2(x) ∂hN (x) ∂x1 ∂x1 ··· ∂x1 ∂h1(x) ∂h2(x) ∂hN (x) ∆ ∂x2 ∂x2 ∂x2 h(x) = . ··· . ∇ . . (1358) ∂h1(x) ∂h2(x) ∂hN (x) ∂x ∂x ∂x K K ··· K K×N = [ h (x) h (x) hN (x) ] R ∇ 1 ∇ 2 · · · ∇ ∈ while the second-order gradient has a three-dimensional representation dubbed cubix ;D.1 ∂h1(x) ∂h2(x) ∂hN (x) ∇ ∂x1 ∇ ∂x1 · · · ∇ ∂x1 ∂h1(x) ∂h2(x) ∂hN (x) ∆ 2 ∂x2 ∂x2 ∂x2 h(x) = ∇ . ∇ . · · · ∇ . ∇ . . (1359) ∂h1(x) ∂h2(x) ∂hN (x) ∂x ∂x ∂x ∇ K ∇ K · · · ∇ K 2 2 2 K×N×K = [ h (x) h (x) hN (x) ] R ∇ 1 ∇ 2 · · · ∇ ∈ where the gradient of each real entry is with respect to vector x as in (1354). D.1The word matrix comes from the Latin for womb ; related to the prefix matri- derived from mater meaning mother. D.1. DIRECTIONAL DERIVATIVE, TAYLOR SERIES 503 The gradient of real function g(X) : RK×L R on matrix domain is → ∂g(X) ∂g(X) ∂g(X) ∂X11 ∂X12 ··· ∂X1L ∂g(X) ∂g(X) ∂g(X) ∆ K×L ∂X21 ∂X22 ∂X2 R g(X) = . ··· . L ∇ . ∈ . ∂g(X) ∂g(X) ∂g(X) ∂XK1 ∂XK2 ∂XKL ··· (1360) g(X) ∇X(:,1) g(X) × × = ∇X(:,2) RK 1 L ... ∈ g(X) ∇X(:,L) th where the gradient X(:,i) is with respect to the i column of X . The strange appearance of∇ (1360) in RK×1×L is meant to suggest a third dimension perpendicular to the page (not a diagonal matrix). The second-order gradient has representation ∂g(X) ∂g(X) ∂g(X) ∇ ∂X11 ∇ ∂X12 · · · ∇ ∂X1L ∂g(X) ∂g(X) ∂g(X) ∆ K×L×K×L 2 ∂X21 ∂X22 ∂X2 R g(X) = ∇ . ∇ . · · · ∇ . L ∇ . ∈ . ∂g(X) ∂g(X) ∂g(X) ∂XK1 ∂XK2 ∂XKL ∇ ∇ · · · ∇ (1361) g(X) ∇∇X(:,1) g(X) × × × × = ∇∇X(:,2) RK 1 L K L ... ∈ g(X) ∇∇X(:,L) where the gradient is with respect to matrix X . ∇ 504 APPENDIX D. MATRIX CALCULUS Gradient of vector-valued function g(X) : RK×L RN on matrix domain is a cubix → g (X) g (X) gN (X) ∇X(:,1) 1 ∇X(:,1) 2 · · · ∇X(:,1) ∆ g (X) g (X) gN (X) g(X) = ∇X(:,2) 1 ∇X(:,2) 2 · · · ∇X(:,2) ∇ ... ... ... g (X) g (X) gN (X) ∇X(:,L) 1 ∇X(:,L) 2 · · · ∇X(:,L) K×N×L = [ g (X) g (X) gN (X) ] R (1362) ∇ 1 ∇ 2 · · · ∇ ∈ while the second-order gradient has a five-dimensional representation; g (X) g (X) gN (X) ∇∇X(:,1) 1 ∇∇X(:,1) 2 · · · ∇∇X(:,1) ∆ g (X) g (X) gN (X) 2g(X) = ∇∇X(:,2) 1 ∇∇X(:,2) 2 · · · ∇∇X(:,2) ∇ ... ... ... g (X) g (X) gN (X) ∇∇X(:,L) 1 ∇∇X(:,L) 2 · · · ∇∇X(:,L) 2 2 2 K×N×L×K×L = [ g (X) g (X) gN (X) ] R (1363) ∇ 1 ∇ 2 · · · ∇ ∈ The gradient of matrix-valued function g(X) : RK×L RM×N on matrix domain has a four-dimensional representation called quartix→ g (X) g (X) g N (X) ∇ 11 ∇ 12 · · · ∇ 1 ∆ g21(X) g22(X) g2N (X) RM×N×K×L g(X) = ∇ . ∇ . · · · ∇ . (1364) ∇ . ∈ gM (X) gM (X) gMN (X) ∇ 1 ∇ 2 · · · ∇ while the second-order gradient has six-dimensional representation 2 2 2 g (X) g (X) g N (X) ∇ 11 ∇ 12 · · · ∇ 1 2 2 2 2 ∆ g21(X) g22(X) g2N (X) RM×N×K×L×K×L g(X) = ∇ . ∇ . · · · ∇ . ∇ . ∈ 2 2 2 gM (X) gM (X) gMN (X) ∇ 1 ∇ 2 · · · ∇ (1365) and so on. D.1. DIRECTIONAL DERIVATIVE, TAYLOR SERIES 505 D.1.2 Product rules for matrix-functions Given dimensionally compatible matrix-valued functions of matrix variable f(X) and g(X) T X f(X) g(X) = X(f) g + X(g) f (1366) ∇ ∇ ∇ while [35, §8.3] [205] T T T X tr f(X) g(X) = X tr f(X) g(Z) + tr g(X) f(Z) (1367) ∇ ∇ Z←X These expressions implicitly apply as well to scalar-, vector-, or matrix-valued functions of scalar, vector, or matrix arguments. D.1.2.0.1 Example. Cubix. Suppose f(X) : R2×2 R2 = XTa and g(X) : R2×2 R2 = Xb . We wish to find → → T T 2 X f(X) g(X) = X a X b (1368) ∇ ∇ using the product rule. Formula (1366) calls for T 2 T T X a X b = X(X a) Xb + X(Xb) X a (1369) ∇ ∇ ∇ Consider the first of the two terms: T X(f) g = X(X a) Xb ∇ ∇ (1370) = (XTa) (XTa) Xb ∇ 1 ∇ 2 2 2 2 The gradient of XTa forms a cubix in R × × . T T ∂(X a)1 ∂(X a)2 (1371) ∂X11 ∂X11 II II II II II II ∂(XTa)1 ∂(XTa)2 (Xb)1 ∂X12 ∂X12 T R2×1×2 X(X a) Xb = ∇ T T ∈ ∂(X a)1 ∂(X a)2 (Xb) ∂X21 ∂X21 2 II II II II II II ∂(XTa)1 ∂(XTa)2 ∂X22 ∂X22 506 APPENDIX D. MATRIX CALCULUS Because gradient of the product (1368) requires total change with respect to change in each entry of matrix X , the Xb vector must make an inner product with each vector in the second dimension of the cubix (indicated by dotted line segments); a1 0 0 a T 1 b1X11 + b2X12 2×1×2 X(X a) Xb = R ∇ a 0 b1X21 + b2X22 ∈ 2 0 a2 (1372) a (b X + b X ) a (b X + b X ) 2×2 = 1 1 11 2 12 1 1 21 2 22 R a2(b1X11 + b2X12) a2(b1X21 + b2X22) ∈ = abTXT where the cubix appears as a complete 2 2 2 matrix. In like manner for × × the second term X(g) f ∇ b1 0 b 0 T 2 X11a1 + X21a2 2×1×2 X(Xb) X a = R ∇ 0 b X12a1 + X22a2 ∈ (1373) 1 0 b2 2 2 = XTabT R × ∈ The solution T 2 T T T T X a X b = ab X + X ab (1374) ∇ can be found from Table D.2.1 or verified using (1367). 2 D.1.2.1 Kronecker product A partial remedy for venturing into hyperdimensional representations, such as the cubix or quartix, is to first vectorize matrices as in (29). This device gives rise to the Kronecker product of matrices ; a.k.a, direct product ⊗ or tensor product. Although it sees reversal in the literature, [211, §2.1] we adopt the definition: for A Rm×n and B Rp×q ∈ ∈ B A B A B qA 11 12 ··· 1 ∆ B21A B22A B2qA pm×qn B A = . ··· . R (1375) ⊗ . ∈ Bp1A Bp2A BpqA ··· D.1. DIRECTIONAL DERIVATIVE, TAYLOR SERIES 507 One advantage to vectorization is existence of a traditional two-dimensional matrix representation for the second-order gradient of a real function with respect to a vectorized matrix. For example, from n×n § § § §A.1.1 no.22 ( D.2.1) for square A , B R [96, 5.2] [10, 3] ∈ 2 2 2 tr(AXBXT ) = 2 vec(X)T (BT A) vec X = B AT + BT A Rn ×n ∇vec X ∇vec X ⊗ ⊗ ⊗ ∈ (1376) To disadvantage is a large new but known set of algebraic rules and the fact that its mere use does not generally guarantee two-dimensional matrix representation of gradients. D.1.3 Chain rules for composite matrix-functions Given dimensionally compatible matrix-valued functions of matrix variable f(X) and g(X) [137, §15.7] T T X g f(X) = X f f g (1377) ∇ ∇ ∇ 2 T T 2 T 2 g f(X) = X X f f g = f f g + X f g X f (1378) ∇X ∇ ∇ ∇ ∇X ∇ ∇ ∇f ∇ D.1.3.1 Two arguments T T T T X g f(X) , h(X) = X f f g + X h h g (1379) ∇ ∇ ∇ ∇ ∇ D.1.3.1.1 Example. Chain rule for two arguments. [28, §1.1] g f(x)T , h(x)T =(f(x)+ h(x))TA (f(x)+ h(x)) (1380) x εx f(x)= 1 , h(x)= 1 (1381) εx x 2 2 T T 1 0 T ε 0 T x g f(x) , h(x) = (A + A )(f + h)+ (A + A )(f + h) ∇ 0 ε 0 1 (1382) T T 1+ ε 0 T x1 εx1 x g f(x) , h(x) = (A + A ) + ∇ 0 1+ ε εx2 x2 (1383) 508 APPENDIX D.
Load more

Recommended publications
  • The Directional Derivative the Derivative of a Real Valued Function (Scalar field) with Respect to a Vector
    Math 253 The Directional Derivative The derivative of a real valued function (scalar field) with respect to a vector. f(x + h) f(x) What is the vector space analog to the usual derivative in one variable? f 0(x) = lim − ? h 0 h ! x -2 0 2 5 Suppose f is a real valued function (a mapping f : Rn R). ! (e.g. f(x; y) = x2 y2). − 0 Unlike the case with plane figures, functions grow in a variety of z ways at each point on a surface (or n{dimensional structure). We'll be interested in examining the way (f) changes as we move -5 from a point X~ to a nearby point in a particular direction. -2 0 y 2 Figure 1: f(x; y) = x2 y2 − The Directional Derivative x -2 0 2 5 We'll be interested in examining the way (f) changes as we move ~ in a particular direction 0 from a point X to a nearby point . z -5 -2 0 y 2 Figure 2: f(x; y) = x2 y2 − y If we give the direction by using a second vector, say ~u, then for →u X→ + h any real number h, the vector X~ + h~u represents a change in X→ position from X~ along a line through X~ parallel to ~u. →u →u h x Figure 3: Change in position along line parallel to ~u The Directional Derivative x -2 0 2 5 We'll be interested in examining the way (f) changes as we move ~ in a particular direction 0 from a point X to a nearby point .
    [Show full text]
  • Mathematics for Dynamicists - Lecture Notes Thomas Michelitsch
    Mathematics for Dynamicists - Lecture Notes Thomas Michelitsch To cite this version: Thomas Michelitsch. Mathematics for Dynamicists - Lecture Notes. Doctoral. Mathematics for Dynamicists, France. 2006. cel-01128826 HAL Id: cel-01128826 https://hal.archives-ouvertes.fr/cel-01128826 Submitted on 10 Mar 2015 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Mathematics for Dynamicists Lecture Notes c 2004-2006 Thomas Michael Michelitsch1 Department of Civil and Structural Engineering The University of Sheffield Sheffield, UK 1Present address : http://www.dalembert.upmc.fr/home/michelitsch/ . Dedicated to the memory of my parents Michael and Hildegard Michelitsch Handout Manuscript with Fractal Cover Picture by Michael Michelitsch https://www.flickr.com/photos/michelitsch/sets/72157601230117490 1 Contents Preface 1 Part I 1.1 Binomial Theorem 1.2 Derivatives, Integrals and Taylor Series 1.3 Elementary Functions 1.4 Complex Numbers: Cartesian and Polar Representation 1.5 Sequences, Series, Integrals and Power-Series 1.6 Polynomials and Partial Fractions
    [Show full text]
  • Topic 7 Notes 7 Taylor and Laurent Series
    Topic 7 Notes Jeremy Orloff 7 Taylor and Laurent series 7.1 Introduction We originally defined an analytic function as one where the derivative, defined as a limit of ratios, existed. We went on to prove Cauchy's theorem and Cauchy's integral formula. These revealed some deep properties of analytic functions, e.g. the existence of derivatives of all orders. Our goal in this topic is to express analytic functions as infinite power series. This will lead us to Taylor series. When a complex function has an isolated singularity at a point we will replace Taylor series by Laurent series. Not surprisingly we will derive these series from Cauchy's integral formula. Although we come to power series representations after exploring other properties of analytic functions, they will be one of our main tools in understanding and computing with analytic functions. 7.2 Geometric series Having a detailed understanding of geometric series will enable us to use Cauchy's integral formula to understand power series representations of analytic functions. We start with the definition: Definition. A finite geometric series has one of the following (all equivalent) forms. 2 3 n Sn = a(1 + r + r + r + ::: + r ) = a + ar + ar2 + ar3 + ::: + arn n X = arj j=0 n X = a rj j=0 The number r is called the ratio of the geometric series because it is the ratio of consecutive terms of the series. Theorem. The sum of a finite geometric series is given by a(1 − rn+1) S = a(1 + r + r2 + r3 + ::: + rn) = : (1) n 1 − r Proof.
    [Show full text]
  • Estimations of the Trace of Powers of Positive Self-Adjoint Operators by Extrapolation of the Moments∗
    Electronic Transactions on Numerical Analysis. ETNA Volume 39, pp. 144-155, 2012. Kent State University Copyright 2012, Kent State University. http://etna.math.kent.edu ISSN 1068-9613. ESTIMATIONS OF THE TRACE OF POWERS OF POSITIVE SELF-ADJOINT OPERATORS BY EXTRAPOLATION OF THE MOMENTS∗ CLAUDE BREZINSKI†, PARASKEVI FIKA‡, AND MARILENA MITROULI‡ Abstract. Let A be a positive self-adjoint linear operator on a real separable Hilbert space H. Our aim is to build estimates of the trace of Aq, for q ∈ R. These estimates are obtained by extrapolation of the moments of A. Applications of the matrix case are discussed, and numerical results are given. Key words. Trace, positive self-adjoint linear operator, symmetric matrix, matrix powers, matrix moments, extrapolation. AMS subject classifications. 65F15, 65F30, 65B05, 65C05, 65J10, 15A18, 15A45. 1. Introduction. Let A be a positive self-adjoint linear operator from H to H, where H is a real separable Hilbert space with inner product denoted by (·, ·). Our aim is to build estimates of the trace of Aq, for q ∈ R. These estimates are obtained by extrapolation of the integer moments (z, Anz) of A, for n ∈ N. A similar procedure was first introduced in [3] for estimating the Euclidean norm of the error when solving a system of linear equations, which corresponds to q = −2. The case q = −1, which leads to estimates of the trace of the inverse of a matrix, was studied in [4]; on this problem, see [10]. Let us mention that, when only positive powers of A are used, the Hilbert space H could be infinite dimensional, while, for negative powers of A, it is always assumed to be a finite dimensional one, and, obviously, A is also assumed to be invertible.
    [Show full text]
  • The Mean Value Theorem Math 120 Calculus I Fall 2015
    The Mean Value Theorem Math 120 Calculus I Fall 2015 The central theorem to much of differential calculus is the Mean Value Theorem, which we'll abbreviate MVT. It is the theoretical tool used to study the first and second derivatives. There is a nice logical sequence of connections here. It starts with the Extreme Value Theorem (EVT) that we looked at earlier when we studied the concept of continuity. It says that any function that is continuous on a closed interval takes on a maximum and a minimum value. A technical lemma. We begin our study with a technical lemma that allows us to relate 0 the derivative of a function at a point to values of the function nearby. Specifically, if f (x0) is positive, then for x nearby but smaller than x0 the values f(x) will be less than f(x0), but for x nearby but larger than x0, the values of f(x) will be larger than f(x0). This says something like f is an increasing function near x0, but not quite. An analogous statement 0 holds when f (x0) is negative. Proof. The proof of this lemma involves the definition of derivative and the definition of limits, but none of the proofs for the rest of the theorems here require that depth. 0 Suppose that f (x0) = p, some positive number. That means that f(x) − f(x ) lim 0 = p: x!x0 x − x0 f(x) − f(x0) So you can make arbitrarily close to p by taking x sufficiently close to x0.
    [Show full text]
  • AP Calculus AB Topic List 1. Limits Algebraically 2. Limits Graphically 3
    AP Calculus AB Topic List 1. Limits algebraically 2. Limits graphically 3. Limits at infinity 4. Asymptotes 5. Continuity 6. Intermediate value theorem 7. Differentiability 8. Limit definition of a derivative 9. Average rate of change (approximate slope) 10. Tangent lines 11. Derivatives rules and special functions 12. Chain Rule 13. Application of chain rule 14. Derivatives of generic functions using chain rule 15. Implicit differentiation 16. Related rates 17. Derivatives of inverses 18. Logarithmic differentiation 19. Determine function behavior (increasing, decreasing, concavity) given a function 20. Determine function behavior (increasing, decreasing, concavity) given a derivative graph 21. Interpret first and second derivative values in a table 22. Determining if tangent line approximations are over or under estimates 23. Finding critical points and determining if they are relative maximum, relative minimum, or neither 24. Second derivative test for relative maximum or minimum 25. Finding inflection points 26. Finding and justifying critical points from a derivative graph 27. Absolute maximum and minimum 28. Application of maximum and minimum 29. Motion derivatives 30. Vertical motion 31. Mean value theorem 32. Approximating area with rectangles and trapezoids given a function 33. Approximating area with rectangles and trapezoids given a table of values 34. Determining if area approximations are over or under estimates 35. Finding definite integrals graphically 36. Finding definite integrals using given integral values 37. Indefinite integrals with power rule or special derivatives 38. Integration with u-substitution 39. Evaluating definite integrals 40. Definite integrals with u-substitution 41. Solving initial value problems (separable differential equations) 42. Creating a slope field 43.
    [Show full text]
  • 1 Approximating Integrals Using Taylor Polynomials 1 1.1 Definitions
    Seunghee Ye Ma 8: Week 7 Nov 10 Week 7 Summary This week, we will learn how we can approximate integrals using Taylor series and numerical methods. Topics Page 1 Approximating Integrals using Taylor Polynomials 1 1.1 Definitions . .1 1.2 Examples . .2 1.3 Approximating Integrals . .3 2 Numerical Integration 5 1 Approximating Integrals using Taylor Polynomials 1.1 Definitions When we first defined the derivative, recall that it was supposed to be the \instantaneous rate of change" of a function f(x) at a given point c. In other words, f 0 gives us a linear approximation of f(x) near c: for small values of " 2 R, we have f(c + ") ≈ f(c) + "f 0(c) But if f(x) has higher order derivatives, why stop with a linear approximation? Taylor series take this idea of linear approximation and extends it to higher order derivatives, giving us a better approximation of f(x) near c. Definition (Taylor Polynomial and Taylor Series) Let f(x) be a Cn function i.e. f is n-times continuously differentiable. Then, the n-th order Taylor polynomial of f(x) about c is: n X f (k)(c) T (f)(x) = (x − c)k n k! k=0 The n-th order remainder of f(x) is: Rn(f)(x) = f(x) − Tn(f)(x) If f(x) is C1, then the Taylor series of f(x) about c is: 1 X f (k)(c) T (f)(x) = (x − c)k 1 k! k=0 Note that the first order Taylor polynomial of f(x) is precisely the linear approximation we wrote down in the beginning.
    [Show full text]
  • Appendix a Short Course in Taylor Series
    Appendix A Short Course in Taylor Series The Taylor series is mainly used for approximating functions when one can identify a small parameter. Expansion techniques are useful for many applications in physics, sometimes in unexpected ways. A.1 Taylor Series Expansions and Approximations In mathematics, the Taylor series is a representation of a function as an infinite sum of terms calculated from the values of its derivatives at a single point. It is named after the English mathematician Brook Taylor. If the series is centered at zero, the series is also called a Maclaurin series, named after the Scottish mathematician Colin Maclaurin. It is common practice to use a finite number of terms of the series to approximate a function. The Taylor series may be regarded as the limit of the Taylor polynomials. A.2 Definition A Taylor series is a series expansion of a function about a point. A one-dimensional Taylor series is an expansion of a real function f(x) about a point x ¼ a is given by; f 00ðÞa f 3ðÞa fxðÞ¼faðÞþf 0ðÞa ðÞþx À a ðÞx À a 2 þ ðÞx À a 3 þÁÁÁ 2! 3! f ðÞn ðÞa þ ðÞx À a n þÁÁÁ ðA:1Þ n! © Springer International Publishing Switzerland 2016 415 B. Zohuri, Directed Energy Weapons, DOI 10.1007/978-3-319-31289-7 416 Appendix A: Short Course in Taylor Series If a ¼ 0, the expansion is known as a Maclaurin Series. Equation A.1 can be written in the more compact sigma notation as follows: X1 f ðÞn ðÞa ðÞx À a n ðA:2Þ n! n¼0 where n ! is mathematical notation for factorial n and f(n)(a) denotes the n th derivation of function f evaluated at the point a.
    [Show full text]
  • Notes for Math 136: Review of Calculus
    Notes for Math 136: Review of Calculus Gyu Eun Lee These notes were written for my personal use for teaching purposes for the course Math 136 running in the Spring of 2016 at UCLA. They are also intended for use as a general calculus reference for this course. In these notes we will briefly review the results of the calculus of several variables most frequently used in partial differential equations. The selection of topics is inspired by the list of topics given by Strauss at the end of section 1.1, but we will not cover everything. Certain topics are covered in the Appendix to the textbook, and we will not deal with them here. The results of single-variable calculus are assumed to be familiar. Practicality, not rigor, is the aim. This document will be updated throughout the quarter as we encounter material involving additional topics. We will focus on functions of two real variables, u = u(x;t). Most definitions and results listed will have generalizations to higher numbers of variables, and we hope the generalizations will be reasonably straightforward to the reader if they become necessary. Occasionally (notably for the chain rule in its many forms) it will be convenient to work with functions of an arbitrary number of variables. We will be rather loose about the issues of differentiability and integrability: unless otherwise stated, we will assume all derivatives and integrals exist, and if we require it we will assume derivatives are continuous up to the required order. (This is also generally the assumption in Strauss.) 1 Differential calculus of several variables 1.1 Partial derivatives Given a scalar function u = u(x;t) of two variables, we may hold one of the variables constant and regard it as a function of one variable only: vx(t) = u(x;t) = wt(x): Then the partial derivatives of u with respect of x and with respect to t are defined as the familiar derivatives from single variable calculus: ¶u dv ¶u dw = x ; = t : ¶t dt ¶x dx c 2016, Gyu Eun Lee.
    [Show full text]
  • Operations on Power Series Related to Taylor Series
    Operations on Power Series Related to Taylor Series In this problem, we perform elementary operations on Taylor series – term by term differen­ tiation and integration – to obtain new examples of power series for which we know their sum. Suppose that a function f has a power series representation of the form: 1 2 X n f(x) = a0 + a1(x − c) + a2(x − c) + · · · = an(x − c) n=0 convergent on the interval (c − R; c + R) for some R. The results we use in this example are: • (Differentiation) Given f as above, f 0(x) has a power series expansion obtained by by differ­ entiating each term in the expansion of f(x): 1 0 X n−1 f (x) = a1 + a2(x − c) + 2a3(x − c) + · · · = nan(x − c) n=1 • (Integration) Given f as above, R f(x) dx has a power series expansion obtained by by inte­ grating each term in the expansion of f(x): 1 Z a1 a2 X an f(x) dx = C + a (x − c) + (x − c)2 + (x − c)3 + · · · = C + (x − c)n+1 0 2 3 n + 1 n=0 for some constant C depending on the choice of antiderivative of f. Questions: 1. Find a power series representation for the function f(x) = arctan(5x): (Note: arctan x is the inverse function to tan x.) 2. Use power series to approximate Z 1 2 sin(x ) dx 0 (Note: sin(x2) is a function whose antiderivative is not an elementary function.) Solution: 1 For question (1), we know that arctan x has a simple derivative: , which then has a power 1 + x2 1 2 series representation similar to that of , where we subsitute −x for x.
    [Show full text]
  • 5 the Dirac Equation and Spinors
    5 The Dirac Equation and Spinors In this section we develop the appropriate wavefunctions for fundamental fermions and bosons. 5.1 Notation Review The three dimension differential operator is : ∂ ∂ ∂ = , , (5.1) ∂x ∂y ∂z We can generalise this to four dimensions ∂µ: 1 ∂ ∂ ∂ ∂ ∂ = , , , (5.2) µ c ∂t ∂x ∂y ∂z 5.2 The Schr¨odinger Equation First consider a classical non-relativistic particle of mass m in a potential U. The energy-momentum relationship is: p2 E = + U (5.3) 2m we can substitute the differential operators: ∂ Eˆ i pˆ i (5.4) → ∂t →− to obtain the non-relativistic Schr¨odinger Equation (with = 1): ∂ψ 1 i = 2 + U ψ (5.5) ∂t −2m For U = 0, the free particle solutions are: iEt ψ(x, t) e− ψ(x) (5.6) ∝ and the probability density ρ and current j are given by: 2 i ρ = ψ(x) j = ψ∗ ψ ψ ψ∗ (5.7) | | −2m − with conservation of probability giving the continuity equation: ∂ρ + j =0, (5.8) ∂t · Or in Covariant notation: µ µ ∂µj = 0 with j =(ρ,j) (5.9) The Schr¨odinger equation is 1st order in ∂/∂t but second order in ∂/∂x. However, as we are going to be dealing with relativistic particles, space and time should be treated equally. 25 5.3 The Klein-Gordon Equation For a relativistic particle the energy-momentum relationship is: p p = p pµ = E2 p 2 = m2 (5.10) · µ − | | Substituting the equation (5.4), leads to the relativistic Klein-Gordon equation: ∂2 + 2 ψ = m2ψ (5.11) −∂t2 The free particle solutions are plane waves: ip x i(Et p x) ψ e− · = e− − · (5.12) ∝ The Klein-Gordon equation successfully describes spin 0 particles in relativistic quan- tum field theory.
    [Show full text]
  • Multivariable and Vector Calculus
    Multivariable and Vector Calculus Lecture Notes for MATH 0200 (Spring 2015) Frederick Tsz-Ho Fong Department of Mathematics Brown University Contents 1 Three-Dimensional Space ....................................5 1.1 Rectangular Coordinates in R3 5 1.2 Dot Product7 1.3 Cross Product9 1.4 Lines and Planes 11 1.5 Parametric Curves 13 2 Partial Differentiations ....................................... 19 2.1 Functions of Several Variables 19 2.2 Partial Derivatives 22 2.3 Chain Rule 26 2.4 Directional Derivatives 30 2.5 Tangent Planes 34 2.6 Local Extrema 36 2.7 Lagrange’s Multiplier 41 2.8 Optimizations 46 3 Multiple Integrations ........................................ 49 3.1 Double Integrals in Rectangular Coordinates 49 3.2 Fubini’s Theorem for General Regions 53 3.3 Double Integrals in Polar Coordinates 57 3.4 Triple Integrals in Rectangular Coordinates 62 3.5 Triple Integrals in Cylindrical Coordinates 67 3.6 Triple Integrals in Spherical Coordinates 70 4 Vector Calculus ............................................ 75 4.1 Vector Fields on R2 and R3 75 4.2 Line Integrals of Vector Fields 83 4.3 Conservative Vector Fields 88 4.4 Green’s Theorem 98 4.5 Parametric Surfaces 105 4.6 Stokes’ Theorem 120 4.7 Divergence Theorem 127 5 Topics in Physics and Engineering .......................... 133 5.1 Coulomb’s Law 133 5.2 Introduction to Maxwell’s Equations 137 5.3 Heat Diffusion 141 5.4 Dirac Delta Functions 144 1 — Three-Dimensional Space 1.1 Rectangular Coordinates in R3 Throughout the course, we will use an ordered triple (x, y, z) to represent a point in the three dimensional space.
    [Show full text]