Chapter 8 Canonical Duality Theory: Connections between Nonconvex Mechanics and Global Optimization David Y. Gao and Hanif D. Sherali Dedicated to Professor Gilbert Strang on the occasion of his 70th birthday Summary. This chapter presents a comprehensive review and some new developments on canonical duality theory for nonconvex systems. Based on a tricanonical form for quadratic minimization problems, an insightful re- lation between canonical dual transformations and nonlinear (or extended) Lagrange multiplier methods is presented. Connections between complemen- tary variational principles in nonconvex mechanics and Lagrange duality in global optimization are also revealed within the framework of the canonical duality theory. Based on this framework, traditional saddle Lagrange duality and the so-called biduality theory, discovered in convex Hamiltonian systems and d.c. programming, are presented in a unified way; together, they serve as a foundation for the triality theory in nonconvex systems. Applications are illustrated by a class of nonconvex problems in continuum mechanics and global optimization. It is shown that by the use of the canonical dual trans- formation, these nonconvex constrained primal problems can be converted into certain simple canonical dual problems, which can be solved to obtain all extremal points. Optimality conditions (both local and global) for these extrema can be identified by the triality theory. Some new results on gen- eral nonconvex programming with nonlinear constraints are also presented as applications of this canonical duality theory. This review brings some fun- damentally new insights into nonconvex mechanics, global optimization, and computational science. Key words: Duality, triality, Lagrangian duality, nonconvex mechanics, global optimization, nonconvex variations, canonical dual transformations, critical point theory, semilinear equations, NP-hard problems, quadratic pro- gramming David Y. Gao, Department of Mathematics, Virginia Tech, Blacksburg, VA 24061, U.S.A. e-mail: [email protected] Hanif D. Sherali, Grado Department of Industrial and Systems Engineering, Virginia Tech, Blacksburg, VA 24061, U.S.A., e-mail: [email protected] D.Y. Gao, H.D. Sherali, (eds.), Advances in Applied Mathematics and Global Optimization 257 Advances in Mechanics and Mathematics 17, DOI 10.1007/978-0-387-75714-8_8, © Springer Science+Business Media, LLC 2009 258 D.Y.Gao,H.D.Sherali 8.1 Introduction Complementarity and duality are two inspiring, closely related concepts. To- gether they play fundamental roles in multidisciplinary fields of mathematical science, especially in engineering mechanics and optimization. The study of complementarity and duality in mathematics and mechanics has had a long history since the well-known Legendre transformation was formally introduced in 1787. This elegant transformation plays a key role in complementary duality theory. In classical mechanical systems, each energy function definedinaconfiguration space is linked via the Legendre trans- formation with a complementary energy in the dual (source) space, through which the Lagrangian and Hamiltonian can be formulated. In static systems, the convex total potential energy leads to a saddle Lagrangian through which a beautiful saddle min-max duality theory can be constructed. This saddle Lagrangian plays a central role in classical duality theory in convex analy- sis and constrained optimization. In convex dynamic systems, however, the total action is usually a nonconvex d.c. function, that is, the difference of convex kinetic energy and total potential functions. In this case, the classical Lagrangian is no longer a saddle function, but the Hamiltonian is convex in each of its variables. It turns out that instead of the Lagrangian, the Hamilto- nian has been extensively used in convex dynamics. From a geometrical point of view, Lagrangian and Hamiltonian structures in convex systems and d.c. programming display an appealing symmetry, which was widely studied by their founders. Unfortunately, such a symmetry in nonconvex systems breaks down. It turns out that in recent times, tremendous effort and attention have been focused on the role of symmetry and symmetry-breaking in Hamilto- nian mechanics in order to gain a deeper understanding into nonlinear and nonconvex phenomena (see Marsden and Ratiu, 1995). The earliest examples of the Lagrangian duality in engineering mechanics are probably the complementary energy principles proposed by Haar and von K´arm´an in 1909 for elastoperfectly plasticity and Hellinger in 1914 for contin- uum mechanics. Since the boundary conditions in Hellinger’s principle were clarified by E. Reissner in 1953 (see Reissner, 1996), the complementary— dual variational principles and methods have been studied extensively for more than 50 years by applied mathematicians and engineers (see Arthurs, 1980, Noble and Sewell, 1972).1 The development of mathematical duality theory in convex variational analysis and optimization has had a similar his- tory since W. Fenchel proposed the well-known Fenchel transformation in 1949. After the revolutionary concepts of superpotential and subdifferentials introduced by J. J. Moreau in 1966 in the study of frictional mechanics, 1 Eric Reissner (PhD 1938) was a professor in the Department of Mathematics at MIT from 1949 to 1969. According to Gil Strang, since Reissner moved to the Department of Mechanical and Aerospace Engineering at University of California, San Diego in 1969, many applied mathematicians in the field of continuum mechanics, especially solid mechanics, switched from mathematical departments to engineering schools in the United States. 8 Canonical Duality Theory 259 the modern mathematical theory of duality has been well developed by cele- brated mathematicians such as R. T. Rockafellar (1967, 1970, 1974), Moreau (1968), Ekeland (1977, 2003), I. Ekeland and R. Temam (1976), F. H. Clarke (1983, 1985), Auchmuty (1986, 2001), G. Strang (1979—1986), and Moreau, Panagiotopoulos, and Strang (1988). Mathematically speaking, in linear elas- ticity where the total potential energy is convex, the Hellinger—Reissner com- plementary variational principle in engineering mechanics is equivalent to a Fenchel—Moreau—Rockafellar type dual variational problem. The so-called generalized complementary variational principle is actually the saddle La- grangian duality theory, which serves as the foundation for hybrid/mixed finite element methods, and has been subjected to extensive study during the past 40 years (see Strang and Fix (1973), Oden and Lee (1977), Pian and Tong (1980), Pian and Wu (2006), Han (2005), and the references cited therein). Early in the beginning of the last century, Haar and von K´arm´an (1909) had already realized that in nonlinear variational problems of continuum me- chanics, the direct approaches for solving minimum potential energy (primal problem) can only provide upper bounding solutions. However, the minimum complementary energy principle (i.e., the maximum Lagrangian dual prob- lem) provides a lower bound (the mathematical proof of Haar—von K´arm´an’s principle was given by Greenberg in 1949). In safety analysis of engineering structures, the upper and lower bounding approximations to the so-called col- lapse states of the elastoplastic structures are equally important to engineers. Therefore, the primal—dual variational methods have been studied extensively by engineers for solving nonsmooth nonlinear problems (see Gao, 1991, 1992, Maier, 1969, 1970, Temam and Strang, 1980, Casciaro and Cascini, 1982, Gao, 1986, Gao and Hwang, 1988, Gao and Cheung, 1989, Gao and Strang, 1989b, Gao and Wierzbicki, 1989, Gao and Onate, 1990, Tabarrok and Rim- rott, 1994). The article by Maier et al. (2000) serves as an excellent survey on the developments for applications of the Lagrangian duality in engineering structural mechanics. In mathematical programming and computational sci- ence, the so-called primal—dual interior point methods are also based on the Lagrangian duality theory, which has emerged as a revolutionary technique during the last 15 years. Complementary to the interior-point methods, the so-called pan-penalty finite element programming developed by Gao in 1988 (1988a,b) is indeed a primal—dual exterior-point method. He proved that in rigid-perfectly plastic limit analysis, the exterior penalty functional and the associated perturbation method possess an elegant physical meaning, which ledtoanefficient dimension rescaling technique in large-scale nonlinear mixed finite element programming problems (Gao, 1988b). In mathematical programming and analysis, the subject of complementar- ity is closely related to constrained optimization, variational inequality, and fixed point theory. Through the classical Lagrangian duality, the KKT condi- tions of constrained optimization problems lead to corresponding complemen- tarity problems. The primal—dual schema has continued to evolve for linear 260 D.Y.Gao,H.D.Sherali and convex mathematical programming during the past 20 years (see Walk, 1989, Wright, 1998). However, for nonconvex systems, it is well known that the KKT conditions are only necessary under certain regularity conditions for global optimality. Moreover, the underlying nonlinear complementarity problems are fundamentally difficult due to the nonmonotonicity of the non- linear operators, and also, many problems in global optimization are NP-hard. The well-developed Fenchel—Moreau—Rockafellar duality theory will produce a so-called duality gap between the primal