Appendix: Calculus of Variations Jianzhong Wu This appendix provides a very brief, first-line introduction to calculus of variations, an extension of multivariable calculus that was first introduced by Leonhard Euler in 1733. The background material is expected to be sufficient for those who are mainly interested in application rather than mathematical development of variational methods for molecular modeling. To get a more comprehensive understanding of this fascinating subject, the reader is referred to standard texts of mathematical physics such as: 1. Mathematical Methods of Physics, J. Mathews and R. L. Walker, Addison-Wesley, 1970. 2. Calculus of Variations, I. M. Gelfand and S. V. Fomin, Dover Books on Mathe- matics, 2000. 3. Variational Methods in Mathematical Physics, P. Blanchard and E. Brüning, Springer- Verlag, 1992. A.1 Functional A functional is an extension of what we mean by a multivariable function. When we write a multivariable function f (z), where z is an n-dimensional variable, we mean = ( , ,..., ) ( ) that for each set of numbers z z1 z2 zn , there is a number f z associated ( ) = 2 = n 2 with it. Simple examples of multivariable functions are f z z i=1 zi or f (z) = a · z, where a is an n-dimensional vector. When we write a functional, F[y], we mean that for each smooth (differentiable) function y(x), there is a number F[y] related to it. In other words, a functional maps a function into a number, or a functional is a function of functions. The integral [ ]= 1 ( ) F y 0 y x dx provides a simple example of functionals. For each smooth function y(x), its integration from 0 to 1 yields a number. While the “input” of a © Springer Science+Business Media Singapore 2017 315 J. Wu (ed.), Variational Methods in Molecular Modeling, Molecular Modeling and Simulation, DOI 10.1007/978-981-10-2502-0 316 Appendix: Calculus of Variations (a) (b) Fig. A.1 While the input for a multidimensional function is a vector, the input for a functional is a smooth function y(x). a An n-dimensional vector z contains a set of numbers affiliated with its dimensionality; b A one-dimensional function y(x) may be understood as a vector of infinite dimensionality multivariable function is an n-dimensional vector, the “input” for a functional is a function. By comparing the similarity between a function and a vector, we see that a functional is a function of infinite dimensionality. Schematically, Fig.A.1 illustrates the difference between the inputs for a multi-dimensional function and a functional. A.2 Variational Problem To illustrate how a functional can be used to solve a realistic problem, consider the time required for a ball to fall along some frictionless path with two ends fixed at positions A and B, as indicated in Fig. A.2. For simplicity, assume that the path is two-dimensional and that it can be described by a smooth function y = y(x).Let t denote the time required for the ball to go from A to B along a frictionless path. What path y(x) should be chosen to make t a minimum? Fig. A.2 Calculus of variations can be used to identify a frictionless path that yields the shortest traveling time for a ball falling from point A to B Appendix: Calculus of Variations 317 For convenience, we put point A at the origin of a coordinate system and measure y downward. At any instant, the ball speed is ds v = , (A.1) dt where v denotes the magnitude of speed, s represents the length along the path, and t is time. Rearrangement of Eq. (A.1)gives ds dt = , v (A.2) and thus the total traveling time is B ds t = . (A.3) A v The differential length of the path ds is ds = 1 + y2dx, (A.4) where y = dy/dx represents the slope of the path. Because the ball starts at point A, conservation of energy requires that at any vertical distance y, the loss of potential energy per unit mass at y is equal to the gain in the kinetic energy per unit mass, i.e., gy = v2/2(A.5) where g stands for the gravity constant. Substituting Eqs. (A.4) and (A.5)into Eq. (A.3)gives x f 1 + y2 t = dx. (A.6) 0 2gy Equation(A.6) indicates that the total time t can be found if we know y as a function of x. For any path with ends fixed at A(0, 0) and B(x f , y f ), there is a corresponding time for the ball to travel from A to B. Therefore, the total traveling time is a functional of path y(x), that is, t = F[y(x)]. The essential problem in calculus of variations is functional minimization,1 i.e., to find a function that minimizes a given functional. In the above example, we want to know the path y(x) with two ends fixed at A and B that gives the minimum descent time. To answer this question, we need to know how a functional responds to a change in its “input”, where the “input” is not an ordinary variable, but a function. 1Functional maximization can be concerted to minimization by trivially adding a negative sign. 318 Appendix: Calculus of Variations A.3 Functional Derivative To obtain the unknown function that minimizes a functional, we use functional dif- ferentiation as discussed below. It is not much different from the partial derivative used in finding the minimum of a multidimensional function. The variation of a functional with respect to its “input” is described by a functional derivative: δ [ ( )] [ ( ) + εδ( − )]− [ ( )] F y x ≡ F y x x x F y x lim δy(x ) ε→0 ε F[y + εδ]−F[y] = lim δ (A.7) ε→0 εδ dF[y] = δ(x − x) dy where ε is a real number, and δ(x − x) stands for the Dirac delta function. As shown in Fig. A.3, the Dirac function δ(x − x0) represents a generalized probability density that is normalized and has a value of infinite at x = x0. According to Eq. (A.7), the functional derivative δF[y(x)]/δy(x) can be understood as the change in functional F[y(x)] with respect to a change in the input function y(x) at the point x = x. Because the functional derivative is in general dependent on x, δF[y(x)]/δy(x) is a function of x. The functional derivative defined above can be similarly applied to a function. Suppose f (y) is a function of y, its functional derivative with respect to y is δ f (y) = f (y)δ(x − x). (A.8) δy(x) In a special case f (y) = y,wehave δy(x) = δ(x − x). (A.9) δy(x) Equation(A.9) says that the functional derivative of a function with respect to itself is a Dirac delta function. Fig. A.3 One-dimensional Dirac function δ(x − x0) represents a probability density that is everywhere zero except at x = x0 where it is infinite (∞) Appendix: Calculus of Variations 319 Functional derivative may be considered as a natural extension of a partial deriva- tive of a multivariable function to infinite dimensionality. To see this, consider again a multivariable function f (z), where z stands for an n-dimensional vector. Partial derivative ∂ f/∂zi describes the change in f (z) with respect to an infinitesimal change in the ith dimension of z while keeping all other dimensions unchanged, i.e., n ∂ f ∂ f df = δ dz = dz . (A.10) ∂z ij i ∂z i j=1 i i where δij stands for the Kronecker delta function, i.e., δij = 1fori = j and zero otherwise. Similarly, the change of a functional with respect to its “input” (function) at a point x can be written as dF dF δF = dx δ(x − x )δy = δy . (A.11) dy dy x Comparing Eqs. (A.10) and (A.11), we see that the variable x can be understood as a continuous index of function y(x), similar to ias an index of vector z. As all partial derivatives of a multi-dimensional function vanish at the minimum point, a functional F[y] reaches an minimum when δF[y(x)] = 0(A.12) δy(x) for all values of x. A.4 Chain Rules of Functional Derivative A functional derivative obeys chain rules similar to those for a partial derivative. For example, the chain rule of a partial derivative of a multivariable function f (z) can be written as ∂ f {g(z)} n ∂ f ∂g = j , (A.13) ∂z ∂g ∂z i j=1 j i where g(z) is an n-dimensional function of vector z. The analogous chain rule for a functional derivative is δF{G[y(x)]} δF δG(x) = dx , (A.14) δy(x) δG(x) δy(x) where the summation of discrete indices in Eq. (A.13) is replaced by an integral over the continuous indices. In particular, if F[y(x)]=y(x),wehave 320 Appendix: Calculus of Variations δy(x) δG(x) δ(x − x) = dx . (A.15) δG(x) δy(x) Equation(A.15) represents a general relation between the reciprocals of functional derivatives. It can be shown that the functional derivative of a function is commutable with a normal derivative, i.e., δ(df/dx) d δ f = (A.16) δy dx δy where both f and g are functions of x. In a special case, the functional derivative of y(x) is δ dy(x) d δy(x) dδ(x − x) = = .