Control of Error and Convergence in ODE Solvers Kjell Gustafsson Integration method + Differential equation Control of Error and Convergence in ODE Solvers Kjell Gustafsson Tekn. Lic, Civ. Ing. Kristianstads Nation Akademisk avhandling som för avläggande av teknisk doktors- examen vid Tekniska fakulteten vid Universitetet i Lund kommer att offentligen försvaras i sal M:A, Maskinhuset, Lunds Tekniska Högskola, fredagen den 22 maj 1992, kl 10.15. A PhD thesis to be defended in public in room M:A, the M-building, Lund Institute of Technology, on Friday May 221992, at 10.15. Jr Control of Error and Convergence in ODE Solvers Kjell Gustafsson Integration method + Differential equation Lund 1992 Cover figure: A block diagram demonstrating the feedback control viewpoint of an implicit in- tegration method, cf. Chapter 5. Department of Automatic Control Lund Institute of Technology Box 118 S-221 00 LUND Sweden © Kjell Gustafsson, 1992 Printed in Sweden by Studentlitteratur's printing-office Lund 1992 . Document nmme Department of Automatic Control DOCTORAL DISSERTATION Lund Institute of Technology : Date of iuuc P.O. Box 118 i March 1992 S-221 00 Lund Sweden , Document Number ISRN LUTFD2/TFRT--1036--SE Author(s) Supervisor Kjell Gustafsson Karl Johan Åström, Gustaf Söderlind, Per Hagander Sponsoring organisation Swedish Research Council for Engineering Science» (TFR), contract 222/91-405. Nordiska Forskarut- bildningsakademien, contract 31065.34.024/91). Title and subtitle Control of Error and Convergence in ODE Solvers Abstract Feedback is a general principle that can be used in ma:iy different contexts. In this thesis it is applied to numerical integration of ordinary differential equations. Ar. ..ivanced integration method includes parameters and variables that should be adjusted during the execu'or In addition, the integration method should be able to automatically handle situations such as: initia!:z -. ;on, restart after failures, etc. In this thesis we regard the algorithms for parameter adjustment and s.. division as a controller. The controller measures different variables that tell the current status of the inte;- ..tion, and based on this information it decides how to continue. The design of the controller is vital in or •: lo accurately and efficiently solve i- 'arge clas» of ordinary differential equations. The application of feedback control may appear farf tched, but numerical integration methods are in fact dynamical systems. This is often overlooked in traditional numerical analysis. We derive ciynamic models that describe the behavior of the integration metho as well as the standard control algori'.ims in use today. Using these models it is possible to analyze proper';.es of current algorithms, and also expl iin some generally observed misbehaviors. Further, we use the acquireri insight to derive new and improved control algorithms, both for explicit and implicit Runge-Kutta methods. In the explicit case, the new controller gives good overall performance. In particular it • vercomes the problem with an oscillating ctepsize sequence that is often experienced when the stepsize is r-stricted by numerical stability. The controller for implicit methods is designed so that it tracks changes in Ke differential equation better than current algorithms. In addition, it includes a new strategy for the equaion solver, which allows the stepsize to vary more freely. This leads to smoother error control without excessive operations on the iteration matrix. Key words numerical integration, ordinary differential equations, it :al value problems, simulation, Runge-Kutta meth- ods, stepsize selection, error control, convergence cont.-i•', numerical analysis Classification system and/or index terms (if any) Supplementary bibliographical information ISSN and key title •l-N I 0280-5316 j Language Number of pages Heapient '* notes English 184 Security classification The report may be ordered from the Department of Automatic Control or borrowed through f S.- JJniver§ity Library 2, Box 1010, S-221 03 Lund, Sweden. Telex 33248 lubbia fond t Contents Preface 7 Acknowledgements 7 1. Introduction 9 1.1 Is This Thesis Really Needed? 10 1.2 Scope of the Thesis 12 1.3 Thesis Outline 14 2. Runge-Kutta Methods 16 2.1 Integration Methods in General 16 2.2 Runge-Kutta Methods 23 2.3 The Standard Stepsize Selection Rule 37 2.4 Equation Solvers 38 2.5 Standard Error and Convergence Control 44 2.6 Control Issues in Runge-Kutta Methods 47 3. The Stepsize-Error Relation 50 3.1 The Limitations of the Asymptotic Model 50 3.2 Different Operating Regions 58 3.3 An Improved Asymptotic Model 61 3.4 Outside the Asymptotic Region 64 3.5 Stepsize Limited by Stability 67 3.6 Experimental Verification of the Models 74 3.7 Conclusions 82 4. Control of Explicit Runge-Kutta Methods 84 4.1 Problems with the Standard Controller 84 4.2 Control Objectives 89 4.3 Changing the Controller Structure 90 4.4 Tuning the Controller Parameters 93 4.5 Restart Strategies 97 4.6 Numerical Examples 99 4.7 The Complete Controller 110 5. Control of Implicit Runge-Kutta Methods 113 5.1 A Predictive Error Controller 115 5.2 The Iteration Error in the Equation Solver 129 5.3 Convergence Control 134 5.4 The Complete Controller 153 6. Conclusions 158 7. References 163 A. Some Common Runge-Kutta Methods 171 A.1 Explicit Methods 172 A.2 Implicit Methods 177 A.3 Method Properties 179 B. List of Notations 183 Preface "It's one small step for man, one giant leap for mankind." Neil Armstrong The size of a step should be related to its environment. This is partic- ularly true in numerical integration of ordinary differential equations, where the stepsize is used to control the quality of the produced solu- tion. A large stepsize leads to a large error, while a too small stepsize is inefficient due to the many steps needed to carry out the integration. What is regarded as large or small, depends on the differential equation and the required accuracy. "The choice of stepsize in an integration method is a problem you control guys should have a look at," said Professor Gustaf Söderlind, when handing out project suggestions at the end of a course on numeri- cal integration methods in 1986/87. Michael Lundh and I got interested in the problem and started working. We soon realized that the standard rule for selecting stepsize could be interpreted as a pure integrating con- troller. A generalization to PI or PID control is then rather reasonable. We experimented with PI control and achieved good results [Gustafsson et a]., 1988]. After having finished the project both Michael and I continued work- ing on other research topics. I was, however, rather annoyed that we did not have a good explanation of why a PI controller made such a differ- ence in performance in the case where numerical stability restricts the stepsize. What was it in the integration method that changed so dra- matically? After a couple of months I returned to the problem, strongly encouraged by Professors Karl Johan Åström, Gustaf Söderlind, and Do- cent Per Hagander. Soon I was hooked. Pursuing the feedback control viewpoint turned out to be very rewarding. Not only was it possible to ex- plain the superiority of the PI controller, but the approach also provided means to ..., well, continue on reading, and you will learn all about it. Acknowledgements This will probably be the most read section of this thesis. I have looked forward to writing it, since it gives me an opportunity to express my gratitude to all dear friends. They have been of invaluable assistance during the years I have been working with this material. I have benefited immensely from their feedback on my ideas, from their help to get the right perspectives, and from their encouragement and support in times of intellectual standstill. Thank you! There are a few friends I would like to mention in particular: Working in an interdisciplinary area I have enjoyed the pleasure of several supervisors. I want to express my gratitude to Karl Johan Åström for teaching me about control, and enthusiastically backing me up when I have tried to apply feedback ideas in a somewhat different area. He has taught me to pose the right questions, and I have benefited immensely from his vast research experience. With admirable teaching skill Gustaf Söderlind introduced me to the subject of numerical integra- tion. He has been a tremendous source ofinformation and has teasingly presented one challenging research problem after another. I am also in- debted to Per Hagander. He has provided many constructive pieces of advice. I appreciate him for taking the time to penetrate even minute details, and for allowing ample time for all our discussions. Among my fellow graduate students I would like to mention Michael Lundh, Bo Bernhardsson, and Ola Dahl. The early part of the work was done together with Michael and I am grateful for his contributions. Bo has been a dear and close friend for more than a decade. We have a common fascination for life in general and engineering in particular. Thanks for sharing your insight and ideas with me. Thanks Ola for surviving in the same office with me, patiently ove^ooking me blocking the computer and spreading my things everywhere. My colleagues at the Department of Automatic Control in Lund form a friendly and inspiring community. The creative and open-minded at- mosphere has influenced me a lot. In addition, the extra-ordinary access to computing power and high quality software, excellently maintained by Leif Andersson and Anders Blomdell, has made work very stimulating. I have plagued Gustaf Söderlind, Karl Johan Åström, Per Hagan- der, and Björn Wittenmark with several versions of the manuscript. Their constructive criticism have greatly improved the quality of the final product.