A Graduate Introduction to Numerical Methods Robert M. Corless • Nicolas Fillion A Graduate Introduction to Numerical Methods From the Viewpoint of Backward Error Analysis 123 Robert M. Corless Nicolas Fillion Applied Mathematics Applied Mathematics University of Western Ontario University of Western Ontario London, ON, Canada London, ON, Canada ISBN 978-1-4614-8452-3 ISBN 978-1-4614-8453-0 (eBook) DOI 10.1007/978-1-4614-8453-0 Springer New York Heidelberg Dordrecht London Library of Congress Control Number: 2013955042 © Springer Science+Business Media New York 2013 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of pub- lication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com) “At its highest level, numerical analysis is a mixture of science, art, and bar-room brawl.” T. W. Korner¨ The Pleasures of Counting, CUP, 1996, p. 505. Dedicated to our mentors Foreword It is a great privilege to be able to write these few words of introduction to this fine book. Computational mathematics is now recognised as a central tool in all aspects of applied mathematics. Scientific modelling falls well short of the mark if it attempts to describe problems and predict outcomes, without numerical com- putations. Thus, an understanding and appreciation of numerical methods are vital components in the training of scientists and engineers. What are numerical methods? Clearly, they are methods for obtaining numerical results. But what numerical results are we looking for? This depends on whom you ask, but a general point of view is to look for common ideas and systematic struc- tures. Thus, linear algebra is central to much of numerical analysis because many scientific problems we need to solve are nothing more than linear equation solu- tions and eigenvalue calculations. But more than this, many other problem types are capable of being expressed in linear algebra terms, and other calculations require efficient linear algebra computations within their core. Many years ago I was told that it had been estimated that if a random computer was stopped at a random time, there would be more than even chances that it would be caught in the middle of an LU factorization. Even if this were true once, it might no longer be true, but it is no more than an exaggeration of the undoubtedly true statement that computational linear algebra is very important and fundamental to science. Numerical linear algebra occupies Part II of this four-part book and covers famil- iar topics as well as many topics that deserve to be familiar. If all the reader wants are the algorithms, then these are there, but the authors are scholars and the reader is not let off so easily. You are dragged gently but firmly to a higher world in which the algorithms are presented in the context of a deductive science. You learn judg- ment and understanding, and you benefit from the authors’ combined experience and knowledge. ButifthisisPartII, what of Part I? Even more fundamental issues are needed before linear algebra can be properly presented, such as the fundamental ideas of computer arithmetic, and the limitations of practical computation in a finite-word computer. Questions about the roots of equations, about the evaluation of series vii viii Foreword and about partial fractions are presented in the entertaining, but at the same time informative, style that characterizes the work as a whole. If the key ideas in Parts I and II are algebraic, the last two parts are calculus- based. In terms of complexity, the first half of the book deals mainly with problems whose solutions in principle are exact, but the second half is about problems for which there is an intrinsic approximation in what is being evaluated. Central to Part III is interpolation, where f (x) is estimated from values of f (xi) based on a set (n) {x1,x2,...,xn}, with an error usually expressed in terms of the behavior of f .The four chapters that comprise this part represent areas in which the authors have made many of their own original contributions. These chapters represent a high point of this very high book. Part IV deals with differential equations and related problems. There are detailed studies of both initial value and boundary value ordinary differential equation prob- lems. Finally, there is a chapter each on delay differential equations and on various types of partial differential equations. The book is rounded out with three useful appendix chapters, presented at the end of this book. I love this book. Auckland, New Zealand John Butcher Preface About This Book This book is designed to be used by mathematicians, engineers, and computer scien- tists as a graduate-level introduction to numerical analysis and its methods. Readers are expected to have had courses or experience in calculus, linear algebra, complex variables, differential equations, and programming. Of course, many students will be missing some of that material, and we encourage generalized review, especially of linear algebra. The book is intended to be suitable both for one-semester and for two-semester courses. It gathers important and recent material from floating-point arithmetic, nu- merical linear algebra, polynomials, interpolation, numerical differentiation and in- tegration, and numerical solutions of differential equations. Our guiding principle for the selection of material and the choice of perspective is that numerical methods should be discussed as a part of a more general practice of mathematical modeling as is found in applied mathematics and engineering. Once mostly absent from texts on numerical methods, this desideratum has become an integral part of much of the active research in various fields of numerical analysis (see, e.g., Enright 2006a). However, because the intended audience is so broad that we cannot really presume a common background in application material, while we focus on applicable compu- tational mathematics, we will not present many actual applications. We believe that the best-compromise approach is to use a perspective on the quality of numerical solution known as backward error analysis, together with the theory of condition- ing or sensitivity of a problem, already known to Turing and widely practiced and written on by J. H. Wilkinson, W. Kahan, and others.1 These ideas, very important although not a panacea, will be introduced progressively. The basic underpinning of the backward error idea, that a numerical method’s errors should be analyzable in 1 The first explicit use of backward error analysis is credited by Wilkinson (1971) to Wallace Givens, and indeed, it is already present in Von Neumann and Goldstine (1947) (see also Grcar 2011), but it is broadly agreed that it was Wilkinson himself who began the systematic exploitation of the idea in a broad collection of contexts. ix x Preface the same terms as whatever physical (or chemical or biological or social or what- have-you) modeling errors, is readily understandable across all fields of application. As Wilkinson (1971 p. 554) pointed out, backward error analysis has the advantage that rounding errors are put on the same footing as errors in the original data and the effect of these has usually to be considered in any case. The notion of the sensitivity of the problem to changes in its data is also one that is easy to get across to application-oriented students. As Chap. 1 explains, this means that we favor a residual-based a posteriori type of backward error analysis that pro- vides numerical solutions that are readily interpretable in the broader context of mathematical modeling. The pedagogical problem that (we hope!) justifies the existence of this book is that even though many excellent numerical analysis books exist, no single one of them that we know of is suitable for such a broad introductory graduate course—at least, not one that provides a unifying perspective based on the concept of backward error analysis, which we think is the most valuable aspect of this present book. Some older books do hold this perspective, most notably Henrici (1982), but that book is dated in other respects nowadays. Other differences between this book and the general numerical analysis liter- ature is that it uses the Lagrange and Hermite interpolational bases heavily, with a complex-variable focus, both because of the recent recognition of the superior- ity of this approach, and in order to introduce topics in an example-based format.