Copyright by Ruben Antonio Gamboa 1999 Mechanically Verifying Real-Valued Algorithms in ACL2 by Ruben Antonio Gamboa, B.S., M.C.S. Dissertation Presented to the Faculty of the Graduate School of The University of Texas at Austin in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy The University of Texas at Austin May 1999 Mechanically Verifying Real-Valued Algorithms in ACL2 Approved by Dissertation Committee: This work is dedicated to my daughter, who inspired me to finish, and to my wife, who would not let me quit. Acknowledgments The members of my committee have all contributed greatly to my dissertation, either di- rectly with comments and ideas, or indirectly by providing me with the necessary back- ground. Special thanks are due to Robert Boyer and Matt Kaufmann, the former for sticking with me through the early years of learning ACL2 and choosing a topic, and the latter for serving as a bountiful source of ideas and enthusiasm. Many other people deserve recognition. Among them are my family for believing in me and supporting me, and my best friend and boss for giving me the time away from work to indulge my academic hobby. RUBEN ANTONIO GAMBOA The University of Texas at Austin May 1999 v Mechanically Verifying Real-Valued Algorithms in ACL2 Publication No. Ruben Antonio Gamboa, Ph.D. The University of Texas at Austin, 1999 Supervisor: Robert S. Boyer ACL2 is a theorem prover over a total, first-order, mostly quantifier-free logic, supporting defined and constrained functions, equality and congruence rewriting, induction, and other reasoning techniques. With its powerful induction engine and direct support for the rational and complex-rational numbers, ACL2 can be used to verify recursive rational algorithms. However, ACL2 can not be used to reason about real-valued algorithms that involve the irrational numbers. For example, there are elegant proofs of the correctness of the Fast Fourier Transform (FFT) which could be formulated in ACL2, but since ACL2’s sparse number system permits the proof that the square root of two, and hence the eighth principal root of one, does not exist, it is impossible to reason directly about the FFT in ACL2. This research extends ACL2 to allow reasoning about the real and complex irra- tional numbers. The modifications are based on non-standard analysis, since infinitesimals are more natural than limits in a quantifier-free context. It is also shown how the trigono- metric functions can be defined in the modified ACL2. These definitions are then used to prove that the FFT correctly implements the Fourier Transform. vi Contents Acknowledgments v Abstract vi List of Tables x Chapter 1 Introduction 1 1.1 Related Work ................................. 5 1.2 Outline .................................... 8 Chapter 2 Learning from the Square Root Function 11 2.1 The Non-Existence of the Square Root of Two ................ 12 2.2 Approximating the Square Root Function .................. 16 Chapter 3 Non-Standard Analysis in ACL2 24 3.1 An Introduction to Non-Standard Analysis .................. 24 3.2 Non-Standard Analysis Primitives in ACL2 ................. 32 3.3 The Non-Standard Analysis ACL2 Library .................. 42 Chapter 4 The Square Root Function Revisited 47 4.1 Defining the Square Root Function in ACL2 ................. 47 4.2 Properties of the Square Root Function .................... 51 vii Chapter 5 The Exponential Function 55 5.1 Defining the Exponential Function in ACL2 ................. 55 5.1.1 A Complex Norm ........................... 56 5.1.2 Geometric Series ........................... 58 5.1.3 The Definition of the Exponential Function ............. 64 5.2 The Exponent of a Sum: ex+y = ex ey ................... 70 · 5.2.1 The Binomial Theorem ........................ 71 5.2.2 Nested Summations ......................... 74 5.2.3 Proving ex+y = ex ey ........................ 77 · 5.3 The Continuity of the Exponential Function ................. 90 Chapter 6 Trigonometric Functions 95 6.1 Defining the Trigonometric Functions in ACL2 ............... 96 6.1.1 The Definition of Sine and Cosine .................. 96 6.1.2 Alternating Sequences ........................ 99 6.1.3 The Intermediate Value Theorem ................... 101 6.1.4 π for Dessert ............................. 111 6.2 Basic Trigonometric Identities ........................ 116 6.3 Computations with the Trigonometric Functions ............... 125 6.3.1 Estimating π ............................. 126 6.3.2 Approximating sine and cosine .................... 132 Chapter 7 Powerlists in ACL2 139 7.1 Regular Powerlists .............................. 139 7.2 Defining Powerlists in ACL2 ......................... 141 7.2.1 Similar Powerlists .......................... 150 7.2.2 Regular Powerlists .......................... 153 7.2.3 Functions on Powerlists ....................... 156 viii 7.3 Simple Examples ............................... 160 7.4 Sorting Powerlists ............................... 165 7.4.1 Merge Sorting ............................ 167 7.4.2 Batcher Sorting ............................ 171 7.4.3 A Comparison with the Hand-Proof ................. 184 7.5 Prefix Sums of Powerlists ........................... 186 7.5.1 Simple Prefix Sums .......................... 188 7.5.2 Ladner-Fischer Prefix Sums ..................... 199 7.5.3 Comparing with the Hand-Proof Again ............... 202 7.5.4 L’agniappe: A Carry-Lookahead Adder ............... 203 Chapter 8 The Fast Fourier Transform 212 8.1 The Fast Fourier Transform .......................... 212 8.2 Verifying the Fast Fourier Transform in ACL2 ................ 215 8.3 Specializing the ACL2 Proof ......................... 225 Chapter 9 Conclusion 232 Appendix A A Simple Introduction to ACL2 236 Bibliography 246 Vita 266 ix List of Tables 9.1 An Estimate of the Proof Effort ........................ 233 x Chapter 1 Introduction ACL2 is a total, first-order, mostly quantifier-free logic based on the programming language Common LISP. ACL2 is also an automated theorem prover for this logic. The theorem prover excels in using induction to prove theorems about recursive functions. At its heart is a rewriter, which uses a database of previously proved theorems to transform a term while maintaining an appropriate equivalence relation, e.g., equality or if-and-only-if. In addition, it supports many other inference mechanisms. For instance, numeric inequalities are automatically verified using a linear arithmetic decision procedure, and propositional tautologies can be proved using a decision procedure based on binary decision diagrams (BDDs). Moreover, ACL2 supports the introduction of constrained functions, allowing a limited amount of higher-order reasoning. ACL2 is a direct descendant of Nqthm, also known as the Boyer-Moore theorem prover. It is fair to say that ACL2 grew out of a desire to “do Nqthm, only better” [40, 12]. Among the enhancements of ACL2 over Nqthm is a richer number system. Nqthm had some native support for the integers, and it was primarily designed for working with the naturals. ACL2, on the other hand, introduced the rational and the complex-rational num- bers. However, the irrationals were deliberately excluded from the ACL2 number system. This exclusion stems from the desire to keep ACL2 as close to Common LISP as possible, 1 and Common LISP does not include the irrationals. Since the primary goal of ACL2 is to prove properties about Common LISP functions, it makes sense to exclude objects that do not exist in the Common LISP universe. However, the omission of the reals places an artificial limit on the theories that ACL2 can verify. For example, an important application of ACL2 is the proof of correctness of various micro-processor algorithms, such as floating-point arithmetic operations, roots, and other transcendental functions. In [54], Russinoff uses ACL2 to prove the correctness of the square root floating-point microcode in the AMD K5 processor. However, since the number px is possibly irrational for a given rational x, his theory “meticulously avoids any reference” to px. In particular, the correctness theorem has the (simplified) form l2 x h2 rnd(l) sqrt(x) rnd(h) ≤ ≤ ) ≤ ≤ where rnd rounds a number to its nearest floating-point representation and sqrt is the AMD K5 microcode algorithm for finding floating-point square roots. This theorem is equivalent to the desired statement of correctness: sqrt(x) = rnd(px) However, this equivalence can not be stated in ACL2. The difference between these two theorems is more than a matter of aesthetics. Consider the norm given by x = Re2(x) + Im2(x), where Re(x) and Im(x) are the jj jj p real and imaginary parts of x, respectively. An algorithm may require the value x y , jj · jj but it may compute it using x y because those values are already known. The results jj jj · jj jj are mathematically equivalent, even if it is false when the approximation sqrt(x) is used instead of the function px. This is a general phenomenon. The correctness of many algorithms depend on prop- erties that are true of real-valued functions, even though the algorithms are implemented entirely in terms of rational or floating-point values. That is, an algorithm may compute the value of the function f at a point x not directly by approximating the equation defining 2 f(x), but indirectly by approximating the equation defining g(x), where it can be proved that the functions f and g are equal. In the example above, the equation f(x; y) would suggest multiplying x and y and taking the norm of the product, while g(x; y) multiplies the norms x and y. The key fact is that in order to prove the correctness of the algorithm following the defining equation g(x), it may be necessary to prove that f(x) = g(x) and such arguments are often impossible without using the language of irrational numbers and irrational functions. Note, it is quite possible that the defining equation for g(x) is more amenable for ap- proximation using floating-point arithmetic than f(x). This happens when the computation of f(x) accumulates errors, whereas the errors tend to cancel out during the computation of g(x). In the language of numerical analysis, g(x) is called stable and f(x) unstable.