H-0140 October 10, 2002 Computer Science IBM Research Report FPgen - A Deep-Knowledge Test-Generator for Floating Point Verification Laurent Fournier, Sigal Asaf IBM Research Division Haifa Research Laboratory Haifa 31905, Israel Research Division Almaden - Austin - Beijing - Haifa - India - T. J. Watson - Tokyo - Zurich LIMITED DISTRIBUTION NOTICE: This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. I thas been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be limited to peer communications and specific requests. After outside publication, requests should be filled only by reprints or legally obtained copies of the article (e.g., payment of royalties). Copies may be requested from IBM T. J. Watson Research Center , P. O. Box 218, Yorktown Heights, NY 10598 USA (email: [email protected]). Some reports are available on the internet at http://domino.watson.ibm.com/library/CyberDig.nsf/home . FPgen – A Deep-Knowledge Test-Generator for Floating Point Verification Laurent Fournier and Sigal Asaf IBM Research Laboratory in Haifa [email protected], [email protected] correctness of all the cases. The difficult question is: “How should such a representative set of test cases be built?” Since Abstract both the architecture specification and the micro-architecture This paper describes FPgen, a test-program generator for implementation yield a myriad of special cases, pure (uniform) datapath verification of floating-point units in microproces- random generation of test cases would be largely inefficient. sors. FPgen is a convenient and powerful platform for gener- As described below, FPgen offers full control to bias test gen- ating interesting data combinations for floating-point eration so it reflects the underlying distribution of interesting instruction operands. Its main purpose is to provide a means cases. to fulfill floating-point test-plans which typically include a How does one know that a certain set of tests is sufficient? myriad of tasks that stem from both the architecture and the This question is related to the notion of coverage, which de- micro-architecture. FPgen focuses on the IEEE standard defi- fines the comprehensiveness of the set related to the verifica- nitions and therefore supports architectures that comply with tion target [6,7,8]. Usually, coverage models, which are sets of these definitions. related tasks, are defined and the set of tests should fulfill all 1. Introduction the defined tasks. For example, a common - albeit far from Traditionally, achieving IEEE compliance for floating-point trivial to fulfill - coverage model is one that requires enume- hardware in microprocessors has been a challenging task. ration of all major IEEE floating point types, simultaneously, Many escape bugs, including the infamous Pentium bug [16], for all operands of all floating-point instructions (the “All belong to the floating-point unit and reveal that the verification IEEE Types” model). For a given instruction with three oper- process in this area is still far from optimal. The growing de- ands, this potentially yields a thousand (10**3) cases that must mand for performance, quality, and faster time-to-market be covered, assuming 10 major floating-point types (+/- NaNs, causes the verification work to become increasingly difficult. +/- Infinity, +/- Zero, +/- Denorm, +/- Norm). Table 1 illus- Problems in the floating-point unit implementation have many trates this model, with a few common floating point instruc- sources. These problems range from data operations on indi- tions. This model can be further refined by including vidual instructions, to the correct handling of sequences of in- additional floating-point types, such as minimum and maxi- structions in which back-to-back events challenge superscalar mum Denorm, etc. Obviously, not all cases are possible (e.g., implementations. The complexity stems both from the richness the addition of two positive denorm numbers cannot reach in- of the specification (architecture) and from the peculiarities of finity), so that the actual number of cases is in fact lower than the implementation (micro-architecture). the size of the full Cartesian product. Verification has traditionally been handled by simulation of Instruction Op1 Op2 Output test-programs [3,5]. Recently, the area of formal methods has Fadd +/- NaN +/- NaN +/- NaN evolved significantly, especially for the floating-point data- Fsub +/- Infinity +/- Infinity +/- Infinity path [9,10,11,13]; however, it is nevertheless far from Fmul +/-0 +/-0 +/-0 providing a comprehensive answer to the problem. Fdiv +/-Denorm +/-Denorm +/-Denorm In most environments, the simulation of test cases is still a Fsqrt +/-Norm +/-Norm +/-Norm cornerstone of the verification process. This paper presents etc. FPgen, a new test generator developed in IBM, targeted to- Table 1: The “All IEEE Types” Coverage Model ward the functional verification of the datapath of floating A coverage model, or the set of all coverage models, is of- point units in microprocessors. FPgen is designed to yield a ten an attempt to partition the set of all the calculation cases in quasi-optimal framework for the generation of test cases in this such a way that the probability distribution is uniform over all area. subsets. As explained below, FPgen provides coverage by gen- When dealing with floating point verification by simulation, eration, i.e., it takes the request of a coverage model (such as there are an enormous, practically unbounded number of dif- the one in Table 1) as input, and outputs a set of tests that ferent calculation cases which need to be tested. In practice, cover all the achievable tasks of the model. simulation can be done on only a very small portion of the ex- The verification process is commonly defined and moni- isting space. The rationale behind verification by simulation is tored through a verification plan, which leads to multiple veri- that one acquires confidence in the design correctness by run- fication tasks. FPgen’s main purpose is to provide a ning a set of test cases that exercise a sufficiently large number convenient platform for performing all these tasks, both quali- of different cases, which in some sense are assumed to be a tatively and quantitatively. On the one hand, it should enable representative sample of the entire space. It is inferred that the the most complex verification requirements, involving intricate correct handling of the tested cases is a testimony to the design 1 sets of constraints, to be fulfilled. On the other hand, it should Other sources of tests widely used in the industry are the enable the generation of a practically unlimited number of dif- IEEE test suites. These test suites are an important quality ferent cases for each given set of constraints. FPgen provides threshold, and assist in reaching confidence in the design cor- these two properties in a context of randomness. This is mean- rectness. The tests provide good coverage of IEEE corner ingful since bug locations are mostly unpredictable. Thus, cases. However, the bug-free running of these suites is neces- FPgen provides a comprehensive, quasi-optimal simulation- sary, but far from sufficient, since their scope is confined to based solution for datapath verification of floating point imple- the IEEE standard and the implementation itself, with its host mentations. While its scope is not limited to a certain of specific cases, is not targeted. architecture or design, its first target is the IEEE Standard for The second technology, Formal Verification (FV), has Binary Floating Point Arithmetic [1], and architectures that evolved significantly during the last five to ten years. While comply with this standard. FV has traditionally been more focused on the verification of FPgen provides a convenient platform for biasing and gen- control paths, it has lately started to target floating-point data- erating operand data for floating point instructions. Simply path verification as well. Verification of control paths [20] put, a bias (or constraint) on an operand data, is a set of values [21], is improving, but still collides with the BDD size explo- to which the operand data is constrained. Resolving biases on sion problem relatively fast as the control block size increases. input operands is usually relatively straightforward, even Two new approaches, both relying on theorem proving [18] though uniformity among all the solutions is sometimes ex- technology, have emerged for datapaths. The first approach tremely hard to obtain. In contrast, resolving constraints on the translates the data-path circuit into the Theorem Prover’s lan- data, for both the intermediate result(s) and the output of in- guage. The proof of the circuit’s correctness is done entirely structions, adds a layer of complexity since it involves instruc- with the Theorem Prover, by proving a succession of manually tion semantics. FPgen’s scope, however, goes beyond the sin- proposed and deducted lemmas (e.g., the work in [9,10] with gle instruction domain, and includes the generation of se- ACL2). Because of the circuit’s complexity and the Theorem quences of instructions possibly driven by the definition of Prover’s limitations, parts of the proof are performed manually coverage models. [9,10]. The second approach combines traditional FV tools FPgen belongs to a family of Deep-Knowledge Test Gen- with the Theorem Prover. The circuit is split into blocks and erators. These generators, developed in IBM, focus on specific each block is verified by traditional FV tools. The Theorem areas not sufficiently stimulated by other means. This family Prover then combines the blocks into one unit [11]. Both ap- includes test generators for micro-architecture flow and for proaches require a significant amount of manual work by ex- Memory Management (Address Translation) Unit verification. perts in mathematics, floating-point, formal methods, and FP micro-architecture.