Lagrange Multipliers Martin J. Gander Archimedes, Bernoulli, Lagrange, Mechanics Archimedes Varignon Pontryagin, Lions: Bernoulli From Lagrange Multipliers to Optimal Bernoulli’s Rule Lever Double Lever Control and PDE Constraints Varignon example Lagrange Multiplier Constraints Martin J. Gander Multiplier Method Optimization [email protected] Optimal Control Hamiltonian Maximum Principle Pontryagin University of Geneva Adjoint PDE Constraint Optimization Joint work with Felix Kwok and Gerhard Wanner Lions Adjoint Conclusion University of Nice, October 2014 Lagrange Archimedes’ Law of the Lever Multipliers Martin J. Gander Archimedes (287-212 BC): “Two bodies are in equilibrium Mechanics if their weights are inversely proportional to their arm length” Archimedes Varignon Bernoulli Bernoulli’s Rule Lever Double Lever Varignon example Lagrange Multiplier Constraints Multiplier Method Optimization Optimal Control Hamiltonian Maximum Principle Pontryagin Adjoint PDE Constraint Optimization Lions Adjoint Conclusion Lagrange Archimedes’ Law of the Lever Multipliers Martin J. Gander Archimedes (287-212 BC): “Two bodies are in equilibrium Mechanics if their weights are inversely proportional to their arm length” Archimedes Varignon Bernoulli Bernoulli’s Rule Lever Double Lever Varignon example Lagrange Multiplier Constraints Multiplier Method Optimization Optimal Control Hamiltonian Beautiful proof of Archimedes: Maximum Principle Pontryagin 1) Axiom: equal weights at equal distances are in equilibrium Adjoint PDE Constraint Optimization Lions Adjoint Conclusion 2) Redistribute weights to reach symmetric configuration Lagrange Multipliers Martin J. Gander Mechanics Archimedes Varignon Bernoulli Bernoulli’s Rule Lever Double Lever Varignon example Lagrange Multiplier Constraints Multiplier Method Optimization Optimal Control Hamiltonian Maximum Principle Pontryagin Adjoint PDE Constraint Optimization Lions Adjoint Conclusion Lagrange Pierre Varignon Multipliers Varignon (1654-1722): Nouvelle M´ecanique (frontispiece Martin J. Gander “dont le projet fut donn´een M.DC.LXXXVII”) published Mechanics Archimedes posthumously in 1725 Varignon Bernoulli Bernoulli’s Rule Hundreds of results illustrated in 64 plates of figures: Lever Double Lever Varignon example Lagrange Multiplier Constraints Multiplier Method Optimization Optimal Control Hamiltonian Maximum Principle Pontryagin Adjoint PDE Constraint Optimization Lions Adjoint Conclusion Lagrange Pierre Varignon Multipliers Martin J. Gander Mechanics Archimedes Varignon Bernoulli Bernoulli’s Rule Lever Double Lever Varignon example Lagrange Multiplier Constraints Multiplier Method Optimization Optimal Control Hamiltonian Maximum Principle Pontryagin Adjoint PDE Constraint Optimization Lions Adjoint Conclusion Lagrange Johann Bernoulli Multipliers Johann Bernoulli (1667-1748): Invents a general rule Martin J. Gander (“r`egle”) and announces it to Mr. le Chev. Renau (Bernard Mechanics Archimedes Renau d’Eli¸cagaray, “le petit Renau”) in a letter, with copy Varignon to Varignon: Bernoulli Bernoulli’s Rule “Votre projet d’une nouvelle mechanique fourmille d’un Lever Double Lever grand nombre d’exemples, dont quelques uns `aen juger par Varignon example les figures paroissent assez compliqu´es; mais je vous deffie de Lagrange Multiplier m’en proposer un `avotre choix, que je ne resolve sur le Constraints Multiplier Method champ et comme en jouant par ma dite r`egle.” Optimization Optimal Control Hamiltonian “Your project of a new theory of mechanics is Maximum Principle Pontryagin swarming of examples, some of which, judging from Adjoint PDE Constraint the figures, appear to be quite complicated; but I Optimization Lions challenge you to propose any one of them to me, Adjoint and I solve it on the spot with my so-called rule” Conclusion Varignon had difficulties admitting that all his work over decades was declared to be so easy and contested the general truth of this rule. Lagrange Bernoulli’s Rule in Varignon’s Book Multipliers Martin J. Gander Mechanics Archimedes Varignon Bernoulli Bernoulli’s Rule Lever Double Lever Varignon example Lagrange Multiplier Constraints Multiplier Method Optimization Optimal Control Hamiltonian Maximum Principle “In every equilibrium of arbitrary forces, no matter how they Pontryagin Adjoint are applied, and in which directions they act the ones on the PDE Constraint Optimization others, either indirectly or directly, the sum of the positive Lions Adjoint energies will be equal to the sum of the negative energies Conclusion taken positively.” Varignon gives however the wrong date 1717 for the letter of Bernoulli, which was later copied by Lagrange. Lagrange Lagrange Explains Bernoulli’s Rule Multipliers Joseph-Louis Lagrange (1736-1813): “M´ecanique Martin J. Gander analytique” from 1788 Mechanics Archimedes Varignon Bernoulli Bernoulli’s Rule Lever Double Lever Varignon example Lagrange Multiplier Constraints Multiplier Method Optimization Optimal Control Assume the lever is in equilibrium: Archimedes implies Hamiltonian Maximum Principle Pontryagin P b Pa = Qb =⇒ = Adjoint PDE Constraint Q a Optimization Lions Bernoulli’s idea: apply a “virtual velocity” during an Adjoint infinitely small time interval to displace the lever. Then Conclusion P dq a ∝ dp and b ∝ dq =⇒ = − =⇒ Pdp + Qdq = 0 Q dp Lagrange So Forget Archimedes ! Multipliers Applying the virtual velocity principle, Martin J. Gander Mechanics Archimedes Varignon Bernoulli Bernoulli’s Rule Lever Double Lever Varignon example Lagrange Multiplier Constraints Multiplier Method we get Optimization Optimal Control Pdp + Qdq = 0. Hamiltonian Maximum Principle b Pontryagin Now if dp = dx for an arbitrary dx, then dq = − a dx, and Adjoint PDE Constraint hence Optimization Lions b Adjoint Pdp + Qdq =0 =⇒ P − Q dx = 0 ∀dx Conclusion a P b =⇒ = (Archimedes) Q a Lagrange A More Complicated System Multipliers Martin J. Gander Mechanics Archimedes Varignon Bernoulli Bernoulli’s Rule Lever Double Lever Varignon example Lagrange Multiplier Constraints Multiplier Method Optimization Applying the rule of Bernoulli, we get Optimal Control Hamiltonian Maximum Principle Pdp + Qdq + Rdr = 0 (∗) Pontryagin Adjoint PDE Constraint b d bd Optimization If dp = dx, then dq = − a dx, and dr = − c dq = ac dx, and Lions Adjoint b bd Conclusion (∗) =⇒ P − Q + R dx = 0 ∀dx a ac Lagrange A System of Varignon Style Multipliers Martin J. Gander To use Mechanics Archimedes Varignon Pdp +Qdq +Rdr = 0 (∗) Bernoulli Bernoulli’s Rule Lever Double Lever we compute Varignon example Lagrange 2 2 2 Multiplier p = (x −a) + (y −b) + (z −c) Constraints Multiplier Method q Optimization Optimal Control 1 Hamiltonian dp = ·((x−a)dx+(y−b)dy+(z−c)dz) Maximum Principle p Pontryagin Adjoint and similarly q, dq, and r, dr. PDE Constraint Optimization Lions Inserted into (∗) we get for equilibrium Adjoint Xdx + Ydy + Zdz = 0. Conclusion x−a x−f x−l y−b y−g y−m with X = P p + Q q + R r , Y = P p + Q q + R r z−c z−h z−n and Z = P p + Q q + R r . Lagrange Same System with Constraint Multipliers Martin J. Gander Mechanics Archimedes Motion (dx, dy, dz) Varignon restricted to a surface L = 0: Bernoulli Bernoulli’s Rule ∂L ∂L ∂L Lever dL = dx+ dy+ dz = 0 (1) Double Lever ∂x ∂y ∂z Varignon example Lagrange Multiplier together with the equation Constraints Multiplier Method Optimization Optimal Control Xdx + Ydy + Zdz = 0 (2) Hamiltonian Maximum Principle Pontryagin ∂L Multiply (1) by λ = −Z/ ∂z and add Adjoint PDE Constraint to (2) to obtain Optimization Lions Adjoint ∂L ∂L ∂L X + λ ∂x · dx + Y + λ ∂y · dy = 0 , Z + λ ∂z = 0 . Conclusion This means applying the virtual velocity argument, without constraints, to Xdx + Ydy + Zdz + λdL = 0 “Il n’est pas difficile de prouver par la th´eorie de l’´elimination des ´equations lin´eaires...” Lagrange The Multiplier Method Multipliers Physical interpretation of λ by Lagrange: Martin J. Gander ∂L ∂L ∂L Mechanics λ( ∂x , ∂y , ∂z ) represents the force that holds the particle on Archimedes Varignon the surface L = 0 Bernoulli Bernoulli’s Rule Lever ´ Double Lever “Equation g´en´erale” for ALL problems of equilibria: Varignon example Lagrange Multiplier Constraints Multiplier Method Optimization Optimal Control ◮ “M´ethode tr`es-simple” in Section IV of the first edition Hamiltonian Maximum Principle from 1788. Pontryagin ◮ Adjoint In the second edition from 1811, Lagrange baptizes the PDE Constraint Optimization method Lions Adjoint Conclusion Lagrange Lagrange Multiplier in Optimization Multipliers Lagrange, Th´eorie de Fonctions Analytiques (1797): “Th´eorie des Martin J. Gander fonctions analytiques, contenant les principes du calcul diff´erentiel, Mechanics Archimedes d´egag´es de toute consid´eration d’infiniment petits, d’´evanouissans, de Varignon limites ou de fluxions, et r´eduits `al’analyse alg´ebrique des quantit´es Bernoulli Bernoulli’s Rule finies” Lever Double Lever Constrained optimization problem: Varignon example Lagrange Multiplier f (x) −→ max, g(x) = 0, Constraints Multiplier Method n n m Optimization with f : R → R and g : R → R , m < n. Optimal Control Hamiltonian Maximum Principle Elimination of the constraints: Pontryagin Adjoint − Rm Rn m PDE Constraint x = (y, u), y ∈ , u ∈ Optimization Lions g(x) = 0 and implicit function theorem =⇒ y = y(u) Adjoint Conclusion Unconstrained optimization problem: f (y(u), u) −→ max . Lagrange Lagrange Multiplier in Optimization Multipliers Martin J. Gander Necessary condition for a local maximum of f (y(u), u): Mechanics Archimedes df ∂f ∂y ∂f Varignon = + = 0 Bernoulli