DOCSLIB.ORG

1.15 Tensor Calculus 2: Tensor Functions

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

Section 1.15 1.15 Tensor Calculus 2: Tensor Functions 1.15.1 Vector-valued functions of a vector Consider a vector-valued function of a vector a = a(b), ai = ai (b j ) This is a function of three independent variables b1 , b2 , b3 , and there are nine partial derivatives ∂ai / ∂b j . The partial derivative of the vector a with respect to b is defined to be a second-order tensor with these partial derivatives as its components: ∂a(b) ∂ai ≡ ei ⊗ e j (1.15.1) ∂b ∂b j It follows from this that −1 ∂a ∂b ∂a ∂b ∂ai ∂bm = or = I, = δ ij (1.15.2) ∂b ∂a ∂b ∂a ∂bm ∂a j To show this, with ai = ai (b j ), bi = bi (a j ) , note that the differential can be written as ∂a ∂a ∂b j ∂a ∂b j ∂a ∂b j ∂a ∂b j da = 1 db = 1 da = da 1 + da 1 + da 1 1 ∂ j ∂ ∂ i 1 ∂ ∂ 2 ∂ ∂ 3 ∂ ∂ b j b j ai b j a1 b j a2 b j a3 Since da1 ,da2 ,da3 are independent, one may set da2 = da3 = 0 , so that ∂a ∂b j 1 = 1 ∂b j ∂a1 Similarly, the terms inside the other brackets are zero and, in this way, one finds Eqn. 1.15.2. 1.15.2 Scalar-valued functions of a tensor Consider a scalar valued function of a (second-order) tensor φ = φ(T), T = T e ⊗ e ij i j . This is a function of nine independent variables, φ = φ(Tij ) , so there are nine different partial derivatives: Solid Mechanics Part III 124 Kelly Section 1.15 ∂φ ∂φ ∂φ ∂φ ∂φ ∂φ ∂φ ∂φ ∂φ , , , , , , , , ∂T11 ∂T12 ∂T13 ∂T21 ∂T22 ∂T23 ∂T31 ∂T32 ∂T33 The partial derivative of φ with respect to T is defined to be a second-order tensor with these partial derivatives as its components: ∂φ ∂φ ≡ ei ⊗ e j Partial Derivative with respect to a Tensor (1.15.3) ∂T ∂Tij The quantity ∂φ(T) / ∂T is also called the gradient of φ with respect to T. Thus differentiation with respect to a second-order tensor raises the order by 2. This agrees with the idea of the gradient of a scalar field where differentiation with respect to a vector raises the order by 1. Derivatives of the Trace and Invariants Consider now the trace: the derivative of trA , with respect to A can be evaluated as follows: ∂ ∂A ∂A ∂A trA = 11 + 22 + 33 ∂A ∂A ∂A ∂A ∂A11 ∂A22 ∂A33 = ei ⊗ e j + ei ⊗ e j + ei ⊗ e j (1.15.4) ∂Aij ∂Aij ∂Aij = e1 ⊗ e1 + e 2 ⊗ e 2 + e3 ⊗ e3 = I Similarly, one finds that {▲Problem 1} 2 3 ∂(trA) ∂(trA ) ∂(trA ) T = I = 2A T = 3(A 2 ) ∂A ∂A ∂A (1.15.5) ∂((trA) 2 ) ∂((trA)3 ) = 2(trA)I = 3(trA) 2 I ∂A ∂A Derivatives of Trace Functions From these and 1.11.17, one can evaluate the derivatives of the invariants {▲Problem 2}: ∂I A = I ∂A ∂II A = I I − A T Derivatives of the Invariants (1.15.6) ∂A A ∂III 2 A = (A T ) − I A T + II I = III A −T ∂A A A A Solid Mechanics Part III 125 Kelly Section 1.15 Derivative of the Determinant An important relation is ∂ (det A) = (det A)A −T (1.15.7) ∂A which follows directly from 1.15.6c. Other Relations The total differential can be written as ∂φ ∂φ ∂φ dφ = dT + dT + dT + ∂T 11 ∂T 12 ∂T 13 11 12 13 (1.15.8) ∂φ ≡ : dT ∂T This total differential gives an approximation to the total increment in φ when the increments of the independent variables T11 , are small. The second partial derivative is defined similarly: ∂∂22φφ ≡eei ⊗⊗⊗ j e pq e, (1.15.9) ∂∂TT ∂TTij ∂ pq the result being in this case a fourth-order tensor. Consider a scalar-valued function of a tensor, φ(A) , but now suppose that the components of A depend upon some scalar parameter t: φ = φ(A(t)) . By means of the chain rule of differentiation, ∂φ dAij φ = (1.15.10) ∂Aij dt which in symbolic notation reads (see Eqn. 1.10.10e) T dφ ∂φ dA ∂φ dA = : = tr (1.15.11) dt ∂A dt ∂A dt Identities for Scalar-valued functions of Symmetric Tensor Functions Let C be a symmetric tensor, C = CT . Then the partial derivative of φ = φ(C(T)) with respect to T can be written as {▲Problem 3} Solid Mechanics Part III 126 Kelly Section 1.15 ∂φ ∂φ (1) = 2T for C = TT T ∂T ∂C ∂φ ∂φ (2) = 2 C for C = TTT (1.15.12) ∂T ∂T ∂φ ∂φ ∂φ ∂φ ∂φ (3) = 2T = 2 T = T + T for C = TT and symmetric T ∂T ∂C ∂C ∂C ∂C Scalar-valued functions of a Symmetric Tensor Consider the expression ∂φ(A) ∂φ(Aij ) B = Bij = (1.15.13) ∂A ∂Aij If A is a symmetric tensor, there are a number of ways to consider this expression: two possibilities are that φ can be considered to be (i) a symmetric function of the 9 variables Aij (ii) a function of 6 independent variables: φ = φ(A11 , A12 , A13 , A22 , A23 , A33 ) where 1 A = (A + A ) = A = A 12 2 12 21 12 21 1 A = (A + A ) = A = A 13 2 13 31 13 31 1 A = (A + A ) = A = A 23 2 23 32 23 32 Looking at (i) and writing φ = φ(A11 , A12 (A12 ),, A21 (A12 ),), one has, for example, ∂φ ∂φ ∂A ∂φ ∂A ∂φ ∂φ ∂φ = 12 + 21 = + = 2 , ∂A12 ∂A12 ∂A12 ∂A21 ∂A12 ∂A12 ∂A21 ∂A12 the last equality following from the fact that φ is a symmetrical function of the Aij . Thus, depending on how the scalar function is presented, one could write ∂φ ∂φ ∂φ (i) B11 = , B12 = , B13 = , etc. ∂A11 ∂A12 ∂A13 ∂φ 1 ∂φ 1 ∂φ (ii) B11 = , B12 = , B13 = , etc. ∂A11 2 ∂A12 2 ∂A13 Solid Mechanics Part III 127 Kelly Section 1.15 1.15.3 Tensor-valued functions of a tensor The derivative of a (second-order) tensor A with respect to another tensor B is defined as ∂A ∂Aij ≡ ei ⊗ e j ⊗ e p ⊗ eq (1.15.14) ∂B ∂B pq and forms therefore a fourth-order tensor. The total differential dA can in this case be written as ∂A dA = : dB (1.15.15) ∂B Consider now ∂A ∂Aij = ei ⊗ e j ⊗ e k ⊗ el ∂A ∂Akl The components of the tensor are independent, so ∂A11 ∂A11 ∂Amn = 1, = 0, etc. = δ mpδ nq (1.15.16) ∂A11 ∂A12 ∂Apq and so ∂A = e ⊗ e ⊗ e ⊗ e = I , (1.15.17) ∂A i j i j the fourth-order identity tensor of Eqn. 1.12.4. Example Consider the scalar-valued function φ of the tensor A and vector v (the “dot” can be omitted from the following and similar expressions), φ(A, v) = v ⋅ Av The gradient of φ with respect to v is ∂φ ∂v ∂v = ⋅ Av + v ⋅ A = Av + vA = (A + A T )v ∂v ∂v ∂v On the other hand, the gradient of φ with respect to A is ∂φ ∂A = v ⋅ v = v ⋅ Iv = v ⊗ v ∂A ∂A Solid Mechanics Part III 128 Kelly Section 1.15 ■ Consider now the derivative of the inverse, ∂A −1 / ∂A . One can differentiate A −1A = 0 using the product rule to arrive at −1 ∂A − ∂A A = −A 1 ∂A ∂A One needs to be careful with derivatives because of the position of the indices in 1.15.14); it looks like a post-operation of both sides with the inverse leads to −1 −1 −1 −1 −1 ∂A / ∂A = −A (∂A / ∂A)A = −Aik Ajl ei ⊗ e j ⊗ e k ⊗ el . However, this is not correct (unless A is symmetric). Using the index notation (there is no clear symbolic notation), one has −1 ∂ ∂Aim −1 Amj Amj = −Aim (ei ⊗ e j ⊗ e k ⊗ el ) ∂Akl ∂Akl −1 ∂ ∂Aim −1 −1 Amj −1 → Amj Ajn = −Aim Ajn ∂Akl ∂Akl −1 (1.15.18) ∂Aim −1 −1 → δ mn = −Aim δ mkδ jl Ajn ∂Akl ∂ −1 Aij −1 −1 → = −Aik Al j (ei ⊗ e j ⊗ e k ⊗ el ) ∂Akl ■ 1.15.4 The Directional Derivative The directional derivative was introduced in §1.6.11. The ideas introduced there can be extended to tensors. For example, the directional derivative of the trace of a tensor A, in the direction of a tensor T, is d d ∂ A (trA)[T] = tr(A + εT) = (trA + εtrT) = trT (1.15.19) dε ε =0 dε ε =0 As a further example, consider the scalar function φ(A) = u ⋅ Av , where u and v are constant vectors. Then d ∂ Aφ(A,u, v)[T] = [u ⋅ (A + εT)v] = u ⋅ Tv (1.15.20) dε ε =0 Also, the gradient of φ with respect to A is ∂φ ∂ = (u ⋅ Av) = u ⊗ v (1.15.21) ∂A ∂A Solid Mechanics Part III 129 Kelly Section 1.15 and it can be seen that this is an example of the more general relation ∂φ ∂ φ[T] = : T (1.15.22) A ∂A which is analogous to 1.6.41. Indeed, ∂φ ∂ φ[w] = ⋅ w x ∂x ∂φ ∂ φ[T] = : T (1.15.23) A ∂A ∂v ∂ v[w] = w u ∂u Example (the Directional Derivative of the Determinant) It was shown in §1.6.11 that the directional derivative of the determinant of the 2× 2 matrix A, in the direction of a second matrix T, is ∂ A (det A)[T] = A11T22 + A22T11 − A12T21 − A21T12 This can be seen to be equal to det A (A −T : T), which will now be proved more generally for tensors A and T: d ∂ A (det A)[T] = det(A + εT) dε ε =0 d = det[A(I + εA −1T)] dε ε =0 d = det A det(I + εA −1T) dε ε =0 The last line here follows from (1.10.16a).
Recommended publications
  • The Directional Derivative the Derivative of a Real Valued Function (Scalar field) with Respect to a Vector

    The Directional Derivative the Derivative of a Real Valued Function (Scalar field) with Respect to a Vector

    Math 253 The Directional Derivative The derivative of a real valued function (scalar field) with respect to a vector. f(x + h) f(x) What is the vector space analog to the usual derivative in one variable? f 0(x) = lim − ? h 0 h ! x -2 0 2 5 Suppose f is a real valued function (a mapping f : Rn R). ! (e.g. f(x; y) = x2 y2). − 0 Unlike the case with plane figures, functions grow in a variety of z ways at each point on a surface (or n{dimensional structure). We'll be interested in examining the way (f) changes as we move -5 from a point X~ to a nearby point in a particular direction. -2 0 y 2 Figure 1: f(x; y) = x2 y2 − The Directional Derivative x -2 0 2 5 We'll be interested in examining the way (f) changes as we move ~ in a particular direction 0 from a point X to a nearby point . z -5 -2 0 y 2 Figure 2: f(x; y) = x2 y2 − y If we give the direction by using a second vector, say ~u, then for →u X→ + h any real number h, the vector X~ + h~u represents a change in X→ position from X~ along a line through X~ parallel to ~u. →u →u h x Figure 3: Change in position along line parallel to ~u The Directional Derivative x -2 0 2 5 We'll be interested in examining the way (f) changes as we move ~ in a particular direction 0 from a point X to a nearby point .
  • Estimations of the Trace of Powers of Positive Self-Adjoint Operators by Extrapolation of the Moments∗

    Estimations of the Trace of Powers of Positive Self-Adjoint Operators by Extrapolation of the Moments∗

    Electronic Transactions on Numerical Analysis. ETNA Volume 39, pp. 144-155, 2012. Kent State University Copyright 2012, Kent State University. http://etna.math.kent.edu ISSN 1068-9613. ESTIMATIONS OF THE TRACE OF POWERS OF POSITIVE SELF-ADJOINT OPERATORS BY EXTRAPOLATION OF THE MOMENTS∗ CLAUDE BREZINSKI†, PARASKEVI FIKA‡, AND MARILENA MITROULI‡ Abstract. Let A be a positive self-adjoint linear operator on a real separable Hilbert space H. Our aim is to build estimates of the trace of Aq, for q ∈ R. These estimates are obtained by extrapolation of the moments of A. Applications of the matrix case are discussed, and numerical results are given. Key words. Trace, positive self-adjoint linear operator, symmetric matrix, matrix powers, matrix moments, extrapolation. AMS subject classifications. 65F15, 65F30, 65B05, 65C05, 65J10, 15A18, 15A45. 1. Introduction. Let A be a positive self-adjoint linear operator from H to H, where H is a real separable Hilbert space with inner product denoted by (·, ·). Our aim is to build estimates of the trace of Aq, for q ∈ R. These estimates are obtained by extrapolation of the integer moments (z, Anz) of A, for n ∈ N. A similar procedure was first introduced in [3] for estimating the Euclidean norm of the error when solving a system of linear equations, which corresponds to q = −2. The case q = −1, which leads to estimates of the trace of the inverse of a matrix, was studied in [4]; on this problem, see [10]. Let us mention that, when only positive powers of A are used, the Hilbert space H could be infinite dimensional, while, for negative powers of A, it is always assumed to be a finite dimensional one, and, obviously, A is also assumed to be invertible.
  • 1.2 Topological Tensor Calculus

    1.2 Topological Tensor Calculus

    PH211 Physical Mathematics Fall 2019 1.2 Topological tensor calculus 1.2.1 Tensor fields Finite displacements in Euclidean space can be represented by arrows and have a natural vector space structure, but finite displacements in more general curved spaces, such as on the surface of a sphere, do not. However, an infinitesimal neighborhood of a point in a smooth curved space1 looks like an infinitesimal neighborhood of Euclidean space, and infinitesimal displacements dx~ retain the vector space structure of displacements in Euclidean space. An infinitesimal neighborhood of a point can be infinitely rescaled to generate a finite vector space, called the tangent space, at the point. A vector lives in the tangent space of a point. Note that vectors do not stretch from one point to vector tangent space at p p space Figure 1.2.1: A vector in the tangent space of a point. another, and vectors at different points live in different tangent spaces and so cannot be added. For example, rescaling the infinitesimal displacement dx~ by dividing it by the in- finitesimal scalar dt gives the velocity dx~ ~v = (1.2.1) dt which is a vector. Similarly, we can picture the covector rφ as the infinitesimal contours of φ in a neighborhood of a point, infinitely rescaled to generate a finite covector in the point's cotangent space. More generally, infinitely rescaling the neighborhood of a point generates the tensor space and its algebra at the point. The tensor space contains the tangent and cotangent spaces as a vector subspaces. A tensor field is something that takes tensor values at every point in a space.
  • Notes for Math 136: Review of Calculus

    Notes for Math 136: Review of Calculus

    Notes for Math 136: Review of Calculus Gyu Eun Lee These notes were written for my personal use for teaching purposes for the course Math 136 running in the Spring of 2016 at UCLA. They are also intended for use as a general calculus reference for this course. In these notes we will briefly review the results of the calculus of several variables most frequently used in partial differential equations. The selection of topics is inspired by the list of topics given by Strauss at the end of section 1.1, but we will not cover everything. Certain topics are covered in the Appendix to the textbook, and we will not deal with them here. The results of single-variable calculus are assumed to be familiar. Practicality, not rigor, is the aim. This document will be updated throughout the quarter as we encounter material involving additional topics. We will focus on functions of two real variables, u = u(x;t). Most definitions and results listed will have generalizations to higher numbers of variables, and we hope the generalizations will be reasonably straightforward to the reader if they become necessary. Occasionally (notably for the chain rule in its many forms) it will be convenient to work with functions of an arbitrary number of variables. We will be rather loose about the issues of differentiability and integrability: unless otherwise stated, we will assume all derivatives and integrals exist, and if we require it we will assume derivatives are continuous up to the required order. (This is also generally the assumption in Strauss.) 1 Differential calculus of several variables 1.1 Partial derivatives Given a scalar function u = u(x;t) of two variables, we may hold one of the variables constant and regard it as a function of one variable only: vx(t) = u(x;t) = wt(x): Then the partial derivatives of u with respect of x and with respect to t are defined as the familiar derivatives from single variable calculus: ¶u dv ¶u dw = x ; = t : ¶t dt ¶x dx c 2016, Gyu Eun Lee.
  • 5 the Dirac Equation and Spinors

    5 the Dirac Equation and Spinors

    5 The Dirac Equation and Spinors In this section we develop the appropriate wavefunctions for fundamental fermions and bosons. 5.1 Notation Review The three dimension differential operator is : ∂ ∂ ∂ = , , (5.1) ∂x ∂y ∂z We can generalise this to four dimensions ∂µ: 1 ∂ ∂ ∂ ∂ ∂ = , , , (5.2) µ c ∂t ∂x ∂y ∂z 5.2 The Schr¨odinger Equation First consider a classical non-relativistic particle of mass m in a potential U. The energy-momentum relationship is: p2 E = + U (5.3) 2m we can substitute the differential operators: ∂ Eˆ i pˆ i (5.4) → ∂t →− to obtain the non-relativistic Schr¨odinger Equation (with = 1): ∂ψ 1 i = 2 + U ψ (5.5) ∂t −2m For U = 0, the free particle solutions are: iEt ψ(x, t) e− ψ(x) (5.6) ∝ and the probability density ρ and current j are given by: 2 i ρ = ψ(x) j = ψ∗ ψ ψ ψ∗ (5.7) | | −2m − with conservation of probability giving the continuity equation: ∂ρ + j =0, (5.8) ∂t · Or in Covariant notation: µ µ ∂µj = 0 with j =(ρ,j) (5.9) The Schr¨odinger equation is 1st order in ∂/∂t but second order in ∂/∂x. However, as we are going to be dealing with relativistic particles, space and time should be treated equally. 25 5.3 The Klein-Gordon Equation For a relativistic particle the energy-momentum relationship is: p p = p pµ = E2 p 2 = m2 (5.10) · µ − | | Substituting the equation (5.4), leads to the relativistic Klein-Gordon equation: ∂2 + 2 ψ = m2ψ (5.11) −∂t2 The free particle solutions are plane waves: ip x i(Et p x) ψ e− · = e− − · (5.12) ∝ The Klein-Gordon equation successfully describes spin 0 particles in relativistic quan- tum field theory.
  • A Some Basic Rules of Tensor Calculus

    A Some Basic Rules of Tensor Calculus

    A Some Basic Rules of Tensor Calculus The tensor calculus is a powerful tool for the description of the fundamentals in con- tinuum mechanics and the derivation of the governing equations for applied prob- lems. In general, there are two possibilities for the representation of the tensors and the tensorial equations: – the direct (symbolic) notation and – the index (component) notation The direct notation operates with scalars, vectors and tensors as physical objects defined in the three dimensional space. A vector (first rank tensor) a is considered as a directed line segment rather than a triple of numbers (coordinates). A second rank tensor A is any finite sum of ordered vector pairs A = a b + ... +c d. The scalars, vectors and tensors are handled as invariant (independent⊗ from the choice⊗ of the coordinate system) objects. This is the reason for the use of the direct notation in the modern literature of mechanics and rheology, e.g. [29, 32, 49, 123, 131, 199, 246, 313, 334] among others. The index notation deals with components or coordinates of vectors and tensors. For a selected basis, e.g. gi, i = 1, 2, 3 one can write a = aig , A = aibj + ... + cidj g g i i ⊗ j Here the Einstein’s summation convention is used: in one expression the twice re- peated indices are summed up from 1 to 3, e.g. 3 3 k k ik ik a gk ∑ a gk, A bk ∑ A bk ≡ k=1 ≡ k=1 In the above examples k is a so-called dummy index. Within the index notation the basic operations with tensors are defined with respect to their coordinates, e.
  • 18.700 JORDAN NORMAL FORM NOTES These Are Some Supplementary Notes on How to Find the Jordan Normal Form of a Small Matrix. Firs

    18.700 JORDAN NORMAL FORM NOTES These Are Some Supplementary Notes on How to Find the Jordan Normal Form of a Small Matrix. Firs

    18.700 JORDAN NORMAL FORM NOTES These are some supplementary notes on how to find the Jordan normal form of a small matrix. First we recall some of the facts from lecture, next we give the general algorithm for finding the Jordan normal form of a linear operator, and then we will see how this works for small matrices. 1. Facts Throughout we will work over the field C of complex numbers, but if you like you may replace this with any other algebraically closed field. Suppose that V is a C-vector space of dimension n and suppose that T : V → V is a C-linear operator. Then the characteristic polynomial of T factors into a product of linear terms, and the irreducible factorization has the form m1 m2 mr cT (X) = (X − λ1) (X − λ2) ... (X − λr) , (1) for some distinct numbers λ1, . , λr ∈ C and with each mi an integer m1 ≥ 1 such that m1 + ··· + mr = n. Recall that for each eigenvalue λi, the eigenspace Eλi is the kernel of T − λiIV . We generalized this by defining for each integer k = 1, 2,... the vector subspace k k E(X−λi) = ker(T − λiIV ) . (2) It is clear that we have inclusions 2 e Eλi = EX−λi ⊂ E(X−λi) ⊂ · · · ⊂ E(X−λi) ⊂ .... (3) k k+1 Since dim(V ) = n, it cannot happen that each dim(E(X−λi) ) < dim(E(X−λi) ), for each e e +1 k = 1, . , n. Therefore there is some least integer ei ≤ n such that E(X−λi) i = E(X−λi) i .
  • Tensor Calculus and Differential Geometry

    Tensor Calculus and Differential Geometry

    Course Notes Tensor Calculus and Differential Geometry 2WAH0 Luc Florack March 10, 2021 Cover illustration: papyrus fragment from Euclid’s Elements of Geometry, Book II [8]. Contents Preface iii Notation 1 1 Prerequisites from Linear Algebra 3 2 Tensor Calculus 7 2.1 Vector Spaces and Bases . .7 2.2 Dual Vector Spaces and Dual Bases . .8 2.3 The Kronecker Tensor . 10 2.4 Inner Products . 11 2.5 Reciprocal Bases . 14 2.6 Bases, Dual Bases, Reciprocal Bases: Mutual Relations . 16 2.7 Examples of Vectors and Covectors . 17 2.8 Tensors . 18 2.8.1 Tensors in all Generality . 18 2.8.2 Tensors Subject to Symmetries . 22 2.8.3 Symmetry and Antisymmetry Preserving Product Operators . 24 2.8.4 Vector Spaces with an Oriented Volume . 31 2.8.5 Tensors on an Inner Product Space . 34 2.8.6 Tensor Transformations . 36 2.8.6.1 “Absolute Tensors” . 37 CONTENTS i 2.8.6.2 “Relative Tensors” . 38 2.8.6.3 “Pseudo Tensors” . 41 2.8.7 Contractions . 43 2.9 The Hodge Star Operator . 43 3 Differential Geometry 47 3.1 Euclidean Space: Cartesian and Curvilinear Coordinates . 47 3.2 Differentiable Manifolds . 48 3.3 Tangent Vectors . 49 3.4 Tangent and Cotangent Bundle . 50 3.5 Exterior Derivative . 51 3.6 Affine Connection . 52 3.7 Lie Derivative . 55 3.8 Torsion . 55 3.9 Levi-Civita Connection . 56 3.10 Geodesics . 57 3.11 Curvature . 58 3.12 Push-Forward and Pull-Back . 59 3.13 Examples . 60 3.13.1 Polar Coordinates in the Euclidean Plane .
  • Matrix Calculus

    Matrix Calculus

    Appendix D Matrix Calculus From too much study, and from extreme passion, cometh madnesse. Isaac Newton [205, §5] − D.1 Gradient, Directional derivative, Taylor series D.1.1 Gradients Gradient of a differentiable real function f(x) : RK R with respect to its vector argument is defined uniquely in terms of partial derivatives→ ∂f(x) ∂x1 ∂f(x) , ∂x2 RK f(x) . (2053) ∇ . ∈ . ∂f(x) ∂xK while the second-order gradient of the twice differentiable real function with respect to its vector argument is traditionally called the Hessian; 2 2 2 ∂ f(x) ∂ f(x) ∂ f(x) 2 ∂x1 ∂x1∂x2 ··· ∂x1∂xK 2 2 2 ∂ f(x) ∂ f(x) ∂ f(x) 2 2 K f(x) , ∂x2∂x1 ∂x2 ··· ∂x2∂xK S (2054) ∇ . ∈ . .. 2 2 2 ∂ f(x) ∂ f(x) ∂ f(x) 2 ∂xK ∂x1 ∂xK ∂x2 ∂x ··· K interpreted ∂f(x) ∂f(x) 2 ∂ ∂ 2 ∂ f(x) ∂x1 ∂x2 ∂ f(x) = = = (2055) ∂x1∂x2 ³∂x2 ´ ³∂x1 ´ ∂x2∂x1 Dattorro, Convex Optimization Euclidean Distance Geometry, Mεβoo, 2005, v2020.02.29. 599 600 APPENDIX D. MATRIX CALCULUS The gradient of vector-valued function v(x) : R RN on real domain is a row vector → v(x) , ∂v1(x) ∂v2(x) ∂vN (x) RN (2056) ∇ ∂x ∂x ··· ∂x ∈ h i while the second-order gradient is 2 2 2 2 , ∂ v1(x) ∂ v2(x) ∂ vN (x) RN v(x) 2 2 2 (2057) ∇ ∂x ∂x ··· ∂x ∈ h i Gradient of vector-valued function h(x) : RK RN on vector domain is → ∂h1(x) ∂h2(x) ∂hN (x) ∂x1 ∂x1 ··· ∂x1 ∂h1(x) ∂h2(x) ∂hN (x) h(x) , ∂x2 ∂x2 ··· ∂x2 ∇ .
  • Math 1320-9 Notes of 3/30/20 11.6 Directional Derivatives and Gradients

    Math 1320-9 Notes of 3/30/20 11.6 Directional Derivatives and Gradients

    Math 1320-9 Notes of 3/30/20 11.6 Directional Derivatives and Gradients • Recall our definitions: f(x0 + h; y0) − f(x0; y0) fx(x0; y0) = lim h−!0 h f(x0; y0 + h) − f(x0; y0) and fy(x0; y0) = lim h−!0 h • You can think of these definitions in several ways, e.g., − You keep all but one variable fixed, and differentiate with respect to the one special variable. − You restrict the function to a line whose direction vector is one of the standard basis vectors, and differentiate that restriction with respect to the corresponding variable. • In the case of fx we move in the direction of < 1; 0 >, and in the case of fy, we move in the direction for < 0; 1 >. Math 1320-9 Notes of 3/30/20 page 1 How about doing the same thing in the direction of some other unit vector u =< a; b >; say. • We define: The directional derivative of f at (x0; y0) in the direc- tion of the (unit) vector u is f(x0 + ha; y0 + hb) − f(x0; y0) Duf(x0; y0) = lim h−!0 h (if this limit exists). • Thus we restrict the function f to the line (x0; y0) + tu, think of it as a function g(t) = f(x0 + ta; y0 + tb); and compute g0(t). • But, by the chain rule, d f(x + ta; y + tb) = f (x ; y )a + f (x ; y )b dt 0 0 x 0 0 y 0 0 =< fx(x0; y0); fy(x0; y0) > · < a; b > • Thus we can compute the directional derivatives by the formula Duf(x0; y0) =< fx(x0; y0); fy(x0; y0) > · < a; b > : • Of course, the partial derivatives @=@x and @=@y are just directional derivatives in the directions i =< 0; 1 > and j =< 0; 1 >, respectively.
  • Trace Inequalities for Matrices

    Trace Inequalities for Matrices

    Bull. Aust. Math. Soc. 87 (2013), 139–148 doi:10.1017/S0004972712000627 TRACE INEQUALITIES FOR MATRICES KHALID SHEBRAWI ˛ and HUSSIEN ALBADAWI (Received 23 February 2012; accepted 20 June 2012) Abstract Trace inequalities for sums and products of matrices are presented. Relations between the given inequalities and earlier results are discussed. Among other inequalities, it is shown that if A and B are positive semidefinite matrices then tr(AB)k ≤ minfkAkk tr Bk; kBkk tr Akg for any positive integer k. 2010 Mathematics subject classification: primary 15A18; secondary 15A42, 15A45. Keywords and phrases: trace inequalities, eigenvalues, singular values. 1. Introduction Let Mn(C) be the algebra of all n × n matrices over the complex number field. The singular values of A 2 Mn(C), denoted by s1(A); s2(A);:::; sn(A), are the eigenvalues ∗ 1=2 of the matrix jAj = (A A) arranged in such a way that s1(A) ≥ s2(A) ≥ · · · ≥ sn(A). 2 ∗ ∗ Note that si (A) = λi(A A) = λi(AA ), so for a positive semidefinite matrix A, we have si(A) = λi(A)(i = 1; 2;:::; n). The trace functional of A 2 Mn(C), denoted by tr A or tr(A), is defined to be the sum of the entries on the main diagonal of A and it is well known that the trace of a Pn matrix A is equal to the sum of its eigenvalues, that is, tr A = j=1 λ j(A). Two principal properties of the trace are that it is a linear functional and, for A; B 2 Mn(C), we have tr(AB) = tr(BA).
  • Fast Higher-Order Functions for Tensor Calculus with Tensors and Subtensors

    Fast Higher-Order Functions for Tensor Calculus with Tensors and Subtensors

    Fast Higher-Order Functions for Tensor Calculus with Tensors and Subtensors Cem Bassoy and Volker Schatz Fraunhofer IOSB, Ettlingen 76275, Germany, [email protected] Abstract. Tensors analysis has become a popular tool for solving prob- lems in computational neuroscience, pattern recognition and signal pro- cessing. Similar to the two-dimensional case, algorithms for multidimen- sional data consist of basic operations accessing only a subset of tensor data. With multiple offsets and step sizes, basic operations for subtensors require sophisticated implementations even for entrywise operations. In this work, we discuss the design and implementation of optimized higher-order functions that operate entrywise on tensors and subtensors with any non-hierarchical storage format and arbitrary number of dimen- sions. We propose recursive multi-index algorithms with reduced index computations and additional optimization techniques such as function inlining with partial template specialization. We show that single-index implementations of higher-order functions with subtensors introduce a runtime penalty of an order of magnitude than the recursive and iterative multi-index versions. Including data- and thread-level parallelization, our optimized implementations reach 68% of the maximum throughput of an Intel Core i9-7900X. In comparison with other libraries, the aver- age speedup of our optimized implementations is up to 5x for map-like and more than 9x for reduce-like operations. For symmetric tensors we measured an average speedup of up to 4x. 1 Introduction Many problems in computational neuroscience, pattern recognition, signal pro- cessing and data mining generate massive amounts of multidimensional data with high dimensionality [13,12,9]. Tensors provide a natural representation for massive multidimensional data [10,7].