Claude-Pierre Jeannerod { Nathalie Revol Floating Point Arithmetic and Rounding Error Analysis Rounding error analysis Old and nontrivial question [von Neumann, Turing, Wilkinson, ...] In this lecture, two approaches: A priori analysis: ! Claude-Pierre's lecture I Goal: bound on jjx^ − xjj=jjxjj for any input and format I Tool: the many nice properties of floating-point I Ideal: readable, provably tight bound + short proof A posteriori, automatic analysis: ! this lecture I Goal:^x and enclosure of^x − x for given input and format I Tool: interval arithmetic based on floating-point I Ideal: a narrow interval computed fast Introduction to interval arithmetic Cons and pros Assessing the numerical quality using IA Conclusions Claude-Pierre Jeannerod { Nathalie Revol Floating Point Arithmetic and Rounding Error Analysis Interval arithmetic: implementation using floating-point arithmetic Implementation using floating-point arithmetic: use directed rounding modes (cf. IEEE 754 standard) p p p[2; 3] = [RD( 2); RU( 3)] Advantage: every result is guaranteed, in the sense that the exact, unknown result, belongs to the computed interval result. Introduction to interval arithmetic Cons and pros Assessing the numerical quality using IA Conclusions Claude-Pierre Jeannerod { Nathalie Revol Floating Point Arithmetic and Rounding Error Analysis Agenda Introduction to interval arithmetic Operations Vectors, matrices Comparisons Expressions and functions extensions Historical remarks Cons and pros Cons: overestimation, complexity Pros: automatic bounds, contractant iterations, Brouwer's theorem Assessing the numerical quality using IA Kahan's point of view Representation by midpoint and radius Variants of interval arithmetic Conclusions Introduction to interval arithmetic Cons and pros Assessing the numerical quality using IA Conclusions Claude-Pierre Jeannerod { Nathalie Revol Floating Point Arithmetic and Rounding Error Analysis Agenda Introduction to interval arithmetic Operations Vectors, matrices Comparisons Expressions and functions extensions Historical remarks Cons and pros Cons: overestimation, complexity Pros: automatic bounds, contractant iterations, Brouwer's theorem Assessing the numerical quality using IA Kahan's point of view Representation by midpoint and radius Variants of interval arithmetic Conclusions Operations Introduction to interval arithmetic Vectors, matrices Cons and pros Comparisons Assessing the numerical quality using IA Expressions and functions extensions Conclusions Historical remarks Claude-Pierre Jeannerod { Nathalie Revol Floating Point Arithmetic and Rounding Error Analysis A brief introduction Interval arithmetic: replace numbers by intervals and compute. Fundamental theorem of interval arithmetic: (or \Thou shalt not lie"): the exact result (number or set) is contained in the computed interval. No result is lost, the computed interval is guaranteed to contain every possible result. Operations Introduction to interval arithmetic Vectors, matrices Cons and pros Comparisons Assessing the numerical quality using IA Expressions and functions extensions Conclusions Historical remarks Claude-Pierre Jeannerod { Nathalie Revol Floating Point Arithmetic and Rounding Error Analysis A brief introduction Interval arithmetic: replace numbers by intervals and compute. Initially: introduced to take into account roundoff errors (Moore 1966) and also uncertainties (on the physical data. ). Later: computations \in the large", computations with sets. Interval analysis: develop algorithms for reliable (or verified, or guaranteed, or certified) computing, that are suited for interval arithmetic, i.e. different from the algorithms from classical numerical analysis. Operations Introduction to interval arithmetic Vectors, matrices Cons and pros Comparisons Assessing the numerical quality using IA Expressions and functions extensions Conclusions Historical remarks Claude-Pierre Jeannerod { Nathalie Revol Floating Point Arithmetic and Rounding Error Analysis A brief introduction: examples of applications I control the roundoff errors, cf. computational geometry I solve several problems with verified solutions: linear and nonlinear systems of equations and inequalities, constraints satisfaction, (non/convex, un/constrained) global optimization, integrate ODEs e.g. particules trajectories. I mathematical proofs: cf. Hales' proof of Kepler's conjecture or Tucker's proof that Lorenz system has a strange attractor or Helfgott's proof of the ternary Goldbach conjecture. Operations Cf. Introduction to interval arithmetic http://www.cs.utep.edu/interval-comp/Vectors, matrices Cons and pros Comparisons Assessing the numerical quality using IA Expressions and functions extensions Conclusions Historical remarks Claude-Pierre Jeannerod { Nathalie Revol Floating Point Arithmetic and Rounding Error Analysis Agenda Introduction to interval arithmetic Operations Vectors, matrices Comparisons Expressions and functions extensions Historical remarks Cons and pros Cons: overestimation, complexity Pros: automatic bounds, contractant iterations, Brouwer's theorem Assessing the numerical quality using IA Kahan's point of view Representation by midpoint and radius Variants of interval arithmetic Conclusions Operations Introduction to interval arithmetic Vectors, matrices Cons and pros Comparisons Assessing the numerical quality using IA Expressions and functions extensions Conclusions Historical remarks Claude-Pierre Jeannerod { Nathalie Revol Floating Point Arithmetic and Rounding Error Analysis A brief introduction Interval arithmetic: replace numbers by intervals and compute. Interval: closed connected subset of R. ;,[−1; 3], ] − 1; 2], [5; +1[ and R are intervals. ] − 1; 3], ]0; +1[ ou [1; 2] [ [3; 4] are not intervals. Operations Introduction to interval arithmetic Vectors, matrices Cons and pros Comparisons Assessing the numerical quality using IA Expressions and functions extensions Conclusions Historical remarks Claude-Pierre Jeannerod { Nathalie Revol Floating Point Arithmetic and Rounding Error Analysis Definitions: operations Fundamental theorem of interval arithmetic: (or \Thou shalt not lie"): the exact result (number or set) is contained in the computed interval. No result is lost, the computed interval is guaranteed to contain every possible result. Operations Introduction to interval arithmetic Vectors, matrices Cons and pros Comparisons Assessing the numerical quality using IA Expressions and functions extensions Conclusions Historical remarks Claude-Pierre Jeannerod { Nathalie Revol Floating Point Arithmetic and Rounding Error Analysis Definitions: operations x y = Hullfx y : x 2 x; y 2 yg Arithmetic and algebraic operations: use the monotonicity [x; x] + y; y = x + y; x + y [x; x] − y; y = x − y; x − y [x; x] × y; y = min(x × y; x × y; x × y; x × y); max(ibid.) [x; x]2 = min(x2; x2); max(x2; x2) if 0 62 [x; x] 0; max(x2; x2) otherwise Operations Introduction to interval arithmetic Vectors, matrices Cons and pros Comparisons Assessing the numerical quality using IA Expressions and functions extensions Conclusions Historical remarks Claude-Pierre Jeannerod { Nathalie Revol Floating Point Arithmetic and Rounding Error Analysis Interval arithmetic: implementation using floating-point arithmetic Implementation using floating-point arithmetic: use directed rounding modes (cf. IEEE 754 standard) p p p[2; 3] = [RD( 2); RU( 3)] Advantage: every result is guaranteed, in the sense that the exact, unknown result, belongs to the computed interval result. Operations Introduction to interval arithmetic Vectors, matrices Cons and pros Comparisons Assessing the numerical quality using IA Expressions and functions extensions Conclusions Historical remarks Claude-Pierre Jeannerod { Nathalie Revol Floating Point Arithmetic and Rounding Error Analysis Operations Algebraic properties: associativity, commutativity hold, some are lost: I subtraction is not the inverse of addition, in particular x − x 6= [0] I division is not the inverse of multiplication I squaring is tighter than multiplication by oneself I multiplication is only sub-distributive wrt addition I with floating-point implementation, operations are not associative either Operations Introduction to interval arithmetic Vectors, matrices Cons and pros Comparisons Assessing the numerical quality using IA Expressions and functions extensions Conclusions Historical remarks Claude-Pierre Jeannerod { Nathalie Revol Floating Point Arithmetic and Rounding Error Analysis Influence of the expression: first example [1; 1] + [2100; 2100] − [2100; 2100]? With these parentheses: ([1; 1]+[2100; 2100])−[2100; 2100] = [2100; succ(2100)]−[2100; 2100] = [0; ulp(2100)]: With those parentheses: [1; 1] + ([2100; 2100] − [2100; 2100]) = [1; 1] + [0; 0] = [1; 1]: Both include the results, one is more accurate than the other. Moral lesson: interval results are always guaranteed to include the exact result, whatever the chosen expression. However their accuracy strongly depends on the chosen expression, on the order of operations. Operations Introduction to interval arithmetic Vectors, matrices Cons and pros Comparisons Assessing the numerical quality using IA Expressions and functions extensions Conclusions Historical remarks Claude-Pierre Jeannerod { Nathalie Revol Floating Point Arithmetic and Rounding Error Analysis Agenda Introduction to interval arithmetic Operations Vectors, matrices Comparisons Expressions and functions extensions Historical remarks Cons and pros Cons: overestimation, complexity Pros: automatic bounds, contractant