Posits: an Alternative to Floating Point Calculations

Total Page:16

File Type:pdf, Size:1020Kb

Posits: an Alternative to Floating Point Calculations Rochester Institute of Technology RIT Scholar Works Theses 5-2020 Posits: An Alternative to Floating Point Calculations Matt Wagner [email protected] Follow this and additional works at: https://scholarworks.rit.edu/theses Recommended Citation Wagner, Matt, "Posits: An Alternative to Floating Point Calculations" (2020). Thesis. Rochester Institute of Technology. Accessed from This Master's Project is brought to you for free and open access by RIT Scholar Works. It has been accepted for inclusion in Theses by an authorized administrator of RIT Scholar Works. For more information, please contact [email protected]. POSITS:AN ALTERNATIVE TO FLOATING POINT CALCULATIONS by MATT WAGNER GRADUATE PAPER Submitted in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE in Electrical Engineering Approved by: Mr. Mark A. Indovina, Senior Lecturer Graduate Research Advisor, Department of Electrical and Microelectronic Engineering Dr. Sohail A. Dianat, Professor Department Head, Department of Electrical and Microelectronic Engineering DEPARTMENT OF ELECTRICAL AND MICROELECTRONIC ENGINEERING KATE GLEASON COLLEGE OF ENGINEERING ROCHESTER INSTITUTE OF TECHNOLOGY ROCHESTER,NEW YORK MAY, 2020 I dedicate this paper to everyone along the way. Declaration Declaration I hereby declare that except where specific reference is made to the work of others, that all content of this Graduate Paper are original and have not been submitted in whole or in part for consideration for any other degree or qualification in this, or any other University. This Graduate Project is the result of my own work and includes nothing which is the outcome of work done in collaboration, except where specifically indicated in the text. Matt Wagner May, 2020 Acknowledgements I would like to thank everyone that has helped me get to where I am. I would like to specifically thank my advisor, Mark Indovina, for helping me put in the hours and always offering help. Abstract Floating point arithmetic is one of several methods of performing computations in digital de- signs; others include integer and fixed point computations. Fixed point utilizes a method com- parable to scientific notation in the binary domain. In terms of computations, floating pointis by far the most prevalent in today’s digital designs. Between the support offered by compilers, as well as for ready-to-use IP blocks, floating point units (FPU’s) are a de-facto standard for most processors. Despite its prevalence in modern designs, floating point has many flaws. One of the most common is the use of not-a-numbers (NaN’s). These are meant to provide a way of signaling invalid operation, however the excessive amount of them wastes usable bit patterns. As an alternative to floating point, a system named "Universal Numbers" or UNUMs wasde- veloped. This system consists of three different types, however for hardware compatibility, the Type III provides the best stand in for floating point. This system eliminates the NaN problem by only using one bit pattern, and also provides many other inherent benefits. Contents Contents v List of Figures ix List of Tablesx 1 Introduction1 1.1 Research Goals . .2 1.2 Organization . .2 2 Floating Point Representation4 2.1 Development . .4 2.2 Format . .5 2.3 Conversion . .6 2.3.1 Determine Sign . .6 2.3.2 Determine Exponent . .6 2.3.3 Determine Fractional Component . .7 2.4 Floating Point Exceptions . .8 2.4.1 Zero and Infinity . .8 2.4.2 Not a Number (NaN) . .9 2.4.3 Subnormal Numbers . .9 2.5 Floating Point Problems . .9 2.5.1 Meaningless Exceptions . 10 2.5.2 Unnecessary Hardware . 10 2.5.3 Overflow . 10 2.5.4 Round-off . 11 3 Unum Representation 12 3.1 Type I Unums . 12 3.1.1 Format . 12 3.1.1.1 U-Bit . 13 3.1.1.2 Exponent & Fraction Size Bits . 13 Contents vi 3.1.2 Type I Unum Characteristics . 14 3.2 Type II Unums . 15 3.2.1 Design . 15 3.2.2 Operations . 17 3.2.3 Accuracy . 17 3.2.4 Hardware Cost . 18 3.3 Interval Arithmetic . 18 4 Posit Representation 20 4.1 Background . 20 4.2 Format . 21 4.3 Regime . 21 4.4 Scaling Factor . 22 4.5 Exponent . 23 4.6 Fraction . 23 4.7 Exceptions . 24 4.8 Benefits . 24 5 Floating Point Core Design 26 5.1 Hierarchical Design . 26 5.2 Data Path . 27 5.2.1 Unpack Module . 28 5.2.2 Exception Handler . 28 5.2.3 Adder/Subtractor . 28 5.2.4 Multiplication . 29 5.2.4.1 Hardware Multiplier . 31 5.2.5 Divider . 31 5.2.5.1 Hardware Divider . 32 5.3 Post-Processor . 32 5.3.1 Rounding Block . 33 5.3.2 Packing Block . 34 5.4 Timing . 34 6 Posit Core Design 36 6.1 Hierarchical Design . 36 6.1.1 Control Logic . 37 6.1.2 Extraction Block . 37 6.1.3 Scaling Block . 37 6.1.3.1 Addition/Subtraction Scaling . 38 6.1.3.2 Multiplication/Division Scaling . 39 6.2 Operation Block . 39 Contents vii 6.3 Decoding Block . 40 6.4 Timing . 41 7 Testbench Design 42 7.1 Testbench . 42 7.1.1 UVM Testbench . 43 7.1.1.1 Top Level . 43 7.1.1.2 Test Case . 44 7.1.1.3 UVM Sequence . 44 7.1.1.4 UVM Environment . 45 7.1.1.5 UVM Scoreboard . 45 7.1.1.6 UVM Agent . 46 7.1.1.7 UVM Sequencer . 46 7.1.1.8 UVM Driver . 47 7.1.1.9 UVM Monitor . 47 7.2 Analysis . 47 8 Hardware Results 50 8.1 Hardware Synthesis . 50 8.2 DFT Insertion . 51 8.3 Analysis Results . 52 9 Conclusions and Further Research 54 9.1 Conclusions . 54 9.2 Future Research . 55 References 57 I FPU Source Code I-1 I.1 FPU Top Level . I-1 I.2 FPU Data Path . I-6 I.3 FPU Unpack . I-15 I.4 FPU Exception Handler . I-17 I.5 FPU Adder/Subtractor . I-22 I.6 FPU Multiplier . I-30 I.7 FPU Divider . I-38 I.8 FPU Post Processor . I-46 I.9 FPU Rounding . I-49 I.10 FPU Packing . I-51 I.11 FPU Test Frame . I-53 Contents viii II PAU Source Code II-1 II.1 PAU Top Level . II-1 II.2 PAU Extraction . II-9 II.3 PAU Scaler . II-12 II.4 PAU Operator . II-17 II.5 PAU Decoder . ..
Recommended publications
  • Nios II Custom Instruction User Guide
    Nios II Custom Instruction User Guide Subscribe UG-20286 | 2020.04.27 Send Feedback Latest document on the web: PDF | HTML Contents Contents 1. Nios II Custom Instruction Overview..............................................................................4 1.1. Custom Instruction Implementation......................................................................... 4 1.1.1. Custom Instruction Hardware Implementation............................................... 5 1.1.2. Custom Instruction Software Implementation................................................ 6 2. Custom Instruction Hardware Interface......................................................................... 7 2.1. Custom Instruction Types....................................................................................... 7 2.1.1. Combinational Custom Instructions.............................................................. 8 2.1.2. Multicycle Custom Instructions...................................................................10 2.1.3. Extended Custom Instructions................................................................... 11 2.1.4. Internal Register File Custom Instructions................................................... 13 2.1.5. External Interface Custom Instructions....................................................... 15 3. Custom Instruction Software Interface.........................................................................16 3.1. Custom Instruction Software Examples................................................................... 16
    [Show full text]
  • The MPFR Team [email protected] This Manual Documents How to Install and Use the Multiple Precision Floating-Point Reliable Library, Version 2.2.0
    MPFR The Multiple Precision Floating-Point Reliable Library Edition 2.2.0 September 2005 The MPFR team [email protected] This manual documents how to install and use the Multiple Precision Floating-Point Reliable Library, version 2.2.0. Copyright 1991, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005 Free Software Foundation, Inc. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with the Front-Cover Texts being “A GNU Manual”, and with the Back-Cover Texts being “You have freedom to copy and modify this GNU Manual, like GNU software”. A copy of the license is included in Appendix A [GNU Free Documentation License], page 30. i Table of Contents MPFR Copying Conditions ................................ 1 1 Introduction to MPFR ................................. 2 1.1 How to use this Manual ........................................................ 2 2 Installing MPFR ....................................... 3 2.1 How to install ................................................................. 3 2.2 Other make targets ............................................................ 3 2.3 Known Build Problems ........................................................ 3 2.4 Getting the Latest Version of MPFR ............................................ 4 3 Reporting Bugs ........................................ 5 4 MPFR Basics .........................................
    [Show full text]
  • IEEE Std 754-1985 Revision of Reaffirmed1990 IEEE Standard for Binary Floating-Point Arithmetic
    Recognized as an American National Standard (ANSI) IEEE Std 754-1985 An American National Standard IEEE Standard for Binary Floating-Point Arithmetic Sponsor Standards Committee of the IEEE Computer Society Approved March 21, 1985 Reaffirmed December 6, 1990 IEEE Standards Board Approved July 26, 1985 Reaffirmed May 21, 1991 American National Standards Institute © Copyright 1985 by The Institute of Electrical and Electronics Engineers, Inc 345 East 47th Street, New York, NY 10017, USA No part of this publication may be reproduced in any form, in an electronic retrieval system or otherwise, without the prior written permission of the publisher. IEEE Standards documents are developed within the Technical Committees of the IEEE Societies and the Standards Coordinating Committees of the IEEE Standards Board. Members of the committees serve voluntarily and without compensation. They are not necessarily members of the Institute. The standards developed within IEEE represent a consensus of the broad expertise on the subject within the Institute as well as those activities outside of IEEE which have expressed an interest in participating in the development of the standard. Use of an IEEE Standard is wholly voluntary. The existence of an IEEE Standard does not imply that there are no other ways to produce, test, measure, purchase, market, or provide other goods and services related to the scope of the IEEE Standard. Furthermore, the viewpoint expressed at the time a standard is approved and issued is subject to change brought about through developments in the state of the art and comments received from users of the standard. Every IEEE Standard is subjected to review at least once every five years for revision or reaffirmation.
    [Show full text]
  • FORTRAN 77 Language Reference
    FORTRAN 77 Language Reference FORTRAN 77 Version 5.0 901 San Antonio Road Palo Alto, , CA 94303-4900 USA 650 960-1300 fax 650 969-9131 Part No: 805-4939 Revision A, February 1999 Copyright Copyright 1999 Sun Microsystems, Inc. 901 San Antonio Road, Palo Alto, California 94303-4900 U.S.A. All rights reserved. All rights reserved. This product or document is protected by copyright and distributed under licenses restricting its use, copying, distribution, and decompilation. No part of this product or document may be reproduced in any form by any means without prior written authorization of Sun and its licensors, if any. Portions of this product may be derived from the UNIX® system, licensed from Novell, Inc., and from the Berkeley 4.3 BSD system, licensed from the University of California. UNIX is a registered trademark in the United States and in other countries and is exclusively licensed by X/Open Company Ltd. Third-party software, including font technology in this product, is protected by copyright and licensed from Sun’s suppliers. RESTRICTED RIGHTS: Use, duplication, or disclosure by the U.S. Government is subject to restrictions of FAR 52.227-14(g)(2)(6/87) and FAR 52.227-19(6/87), or DFAR 252.227-7015(b)(6/95) and DFAR 227.7202-3(a). Sun, Sun Microsystems, the Sun logo, SunDocs, SunExpress, Solaris, Sun Performance Library, Sun Performance WorkShop, Sun Visual WorkShop, Sun WorkShop, and Sun WorkShop Professional are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and in other countries.
    [Show full text]
  • Languages Fortran 90 and C / C+--L
    On the need for predictable floating-point arithmetic in the programming languages Fortran 90 and C / C+--l- DENNIS VERSCHAEREN, ANNIE CUYT AND BRIGITTE VERDONK University of Antwerp (Campus UIA) InfoProc Department of Mathematics and Computer Science Universiteitsplein 1 B-2610 Antwerp - Belgium E-mail: Dennis. Yerschaeren~uia. ua. ac. be E-mail: Annie. Cuyt@uia. ua. ac. be E-mail: Brigitte. Verdonk~uia. ua. ac. be Abstract. During the past decade the IEEE 754 standard for binary floating-point arithmetic has been very successful. Many of today's hardware platforms conform to this standard and compilers should make floating-point functionality available to programmers through high-level programming languages. They should also implement certain features of the IEEE 754 standard in software, such as the input and output conversions to and from the internal binary representation of numbers. In this report, a number of Fortran 90 and C / C++ compilers for workstations as well as personal computers will be screened with respect to their IEEE conformance. It will be shown that most of these compilers do not conform to the IEEE standard and that shortcomings are essentially due to their respective programming language standards which lack attention for the need of predictable floating-point arithmetic. Categories and subject descriptors: G.1.0 [Numerical Analysis]: General--Computer arithmetic; D.3.0 [Programming Languages]: General--Standards; D.3.4 [Programming Languages]: Processors--Compilers AMS subject classifications: 68N15, 68N20 General terms: floating-point, arithmetic Additional key words and phrases: IEEE floating-point standard, Fortran 90, C, C++ 1 - Introduction The IEEE standard for floating-point arithmetic describes how floating-point numbers should be handled in hardware and software.
    [Show full text]
  • Beal's Conjecture Vs. "Positive Zero", Fight
    Beal's Conjecture vs. "Positive Zero", Fight By Angela N. Moore Yale University [email protected] January 6, 2015 Abstract This article seeks to encourage a mathematical dialog regarding a possible solution to Beal’s Conjecture. It breaks down one of the world’s most difficult math problems into layman’s terms and encourages people to question some of the most fundamental rules of mathematics. More specifically; it reinforces basic algebra/critical thinking skills, makes use of properties attributed to the number one and reanalyzes the definition of a positive integer in order to provide a potential counterexample to Beal's Conjecture. I. THE UNDEFEATED CHAMPION (BEAL’S CONJECTURE) A^x + B^y = C^z Where A, B, C, x, y, and z are positive integers with x, y, z > 2, then A, B, and C have a common prime factor. II. NOW FOR THE COUNTEREXAMPLE OF DOOM Pre-Fight: Beal’s Conjecture is never true when (A^x= 1) + B^y = C^z. This is because 1 has no prime factors. Let the Games Begin: There are instances when positive 0 is not the same as zero. “Signed zero is zero with an associated sign. In ordinary arithmetic, −0 = +0 = 0. However, in computing, some number representations allow for the existence of two zeros, often denoted by −0 (negative zero) and +0 (positive zero).”i Furthermore, the same website proved that signed 0 sometimes produces different results than 0. “…the concept of signed zero runs contrary to the general assumption made in most mathematical fields (and in most mathematics courses) that negative zero is the same thing as zero.
    [Show full text]
  • Numerical Computation Guide
    Numerical Computation Guide A Sun Microsystems, Inc. Business 2550 Garcia Avenue Mountain View, CA 94043 U.S.A. Part No.: 801-7639-10 Revision A, August 1994 1994 Sun Microsystems, Inc. 2550 Garcia Avenue, Mountain View, California 94043-1100 U.S.A. All rights reserved. This product and related documentation are protected by copyright and distributed under licenses restricting its use, copying, distribution, and decompilation. No part of this product or related documentation may be reproduced in any form by any means without prior written authorization of Sun and its licensors, if any. Portions of this product may be derived from the UNIX® and Berkeley 4.3 BSD systems, licensed from UNIX System Laboratories, Inc., a wholly owned subsidiary of Novell, Inc., and the University of California, respectively. Third-party font software in this product is protected by copyright and licensed from Sun’s font suppliers. RESTRICTED RIGHTS LEGEND: Use, duplication, or disclosure by the United States Government is subject to the restrictions set forth in DFARS 252.227-7013 (c)(1)(ii) and FAR 52.227-19. The product described in this manual may be protected by one or more U.S. patents, foreign patents, or pending applications. TRADEMARKS Sun, the Sun logo, Sun Microsystems, Sun Microsystems Computer Corporation, Solaris, are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and certain other countries. UNIX is a registered trademark of Novell, Inc., in the United States and other countries; X/Open Company, Ltd., is the exclusive licensor of such trademark. OPEN LOOK® is a registered trademark of Novell, Inc.
    [Show full text]
  • Leveraging Posit Arithmetic in Deep Neural Networks
    Leveraging Posit Arithmetic in Deep Neural Networks Master’s Thesis Course 2020–2021 Author Raul Murillo Montero Advisors Dr. Alberto A. del Barrio García Dr. Guillermo Botella Juan Master in Computer Science Engineering Faculty of Computer Science Complutense University of Madrid This document is prepared for double-sided printing. Leveraging Posit Arithmetic in Deep Neural Networks Master’s Thesis in Computer Science Engineering Department of Computer Architecture and Automation Author Raul Murillo Montero Advisors Dr. Alberto A. del Barrio García Dr. Guillermo Botella Juan Call: January 2021 Grade: 10 Master in Computer Science Engineering Faculty of Computer Science Complutense University of Madrid Course 2020–2021 Copyright c Raul Murillo Montero Acknowledgements To my advisors, Alberto and Guillermo, thank you for giving me the opportunity to embark on this project, for all the corrections, the constant suggestions for improve- ment, and your support. Thanks to my family and to all of you who have accompanied me on this journey and have made it a little easier. v COMPLUTENSE UNIVERSITY OF MADRID Abstract Faculty of Computer Science Department of Computer Architecture and Automation Master in Computer Science Engineering Leveraging Posit Arithmetic in Deep Neural Networks by Raul Murillo Montero The IEEE 754 Standard for Floating-Point Arithmetic has been for decades imple- mented in the vast majority of modern computer systems to manipulate and com- pute real numbers. Recently, John L. Gustafson introduced a new data type called positTM to represent real numbers on computers. This emerging format was designed with the aim of replacing IEEE 754 floating-point numbers by providing certain ad- vantages over them, such as a larger dynamic range, higher accuracy, bitwise iden- tical results across systems, or simpler hardware, among others.
    [Show full text]
  • Floating Point
    Contents Articles Floating point 1 Positional notation 22 References Article Sources and Contributors 32 Image Sources, Licenses and Contributors 33 Article Licenses License 34 Floating point 1 Floating point In computing, floating point describes a method of representing an approximation of a real number in a way that can support a wide range of values. The numbers are, in general, represented approximately to a fixed number of significant digits (the significand) and scaled using an exponent. The base for the scaling is normally 2, 10 or 16. The typical number that can be represented exactly is of the form: Significant digits × baseexponent The idea of floating-point representation over intrinsically integer fixed-point numbers, which consist purely of significand, is that An early electromechanical programmable computer, expanding it with the exponent component achieves greater range. the Z3, included floating-point arithmetic (replica on display at Deutsches Museum in Munich). For instance, to represent large values, e.g. distances between galaxies, there is no need to keep all 39 decimal places down to femtometre-resolution (employed in particle physics). Assuming that the best resolution is in light years, only the 9 most significant decimal digits matter, whereas the remaining 30 digits carry pure noise, and thus can be safely dropped. This represents a savings of 100 bits of computer data storage. Instead of these 100 bits, much fewer are used to represent the scale (the exponent), e.g. 8 bits or 2 A diagram showing a representation of a decimal decimal digits. Given that one number can encode both astronomic floating-point number using a mantissa and an and subatomic distances with the same nine digits of accuracy, but exponent.
    [Show full text]
  • 5.3.1 Min and Max Operations
    The Removal/Demotion of MinNum and MaxNum Operations from IEEE 754™-2018 David H.C. Chen February 21, 2017 Acknowledgements In writing this report, I received help from many colleagues. A thank you to all my colleagues from Floating-Point Working Group for providing crucial insights and expertise. I would also like to express special appreciation to Bob Alverson, Steve Canon, Mike Cowlishaw, David Gay, Michel Hack, John Hauser, David Hough, William Huffman, Grant Martin, Terje Mathisen, Jason Riedy, and Lee Winter, whose valuable inputs, suggestions and comments have significantly improved this report. Any errors or omissions are mine and should not reflect on my colleagues. 1 Introduction IEEE Standard for Floating-Point Arithmetic 754™ 2008 version (IEEE std 754™-2008) [6] specifies four minimum and maximum (min-max) operations: minNum, maxNum, minNumMag, and maxNumMag, in sub-clause “5.3.1 General operations”. These four min-max operations are removed from or demoted in IEEE std 754™-2018 [7], due to their non-associativity. No existing implementations become non-conformant due to this removal/demotion. This report explains the technical reasons for the removal/demotion of the above four min-max operations, and reviews existing min-max implementations in some programming language standards and commercial hardware. Many implementations support several floating-point formats, but this report will only present one or two selected formats for an implementation when additional formats are supported in the same way. To be brief, this report focuses on treatments of Not a Number (NaN) by max operations. A given implementation may also provide max magnitude, min, and min magnitude operations treating signaling NaN (sNaN) and quiet NaN (qNaN) in the same way.
    [Show full text]
  • New Directions in Floating-Point Arithmetic
    New directions in floating-point arithmetic Nelson H. F. Beebe University of Utah Department of Mathematics, 110 LCB 155 S 1400 E RM 233 Salt Lake City, UT 84112-0090 USA Abstract. This article briefly describes the history of floating-point arithmetic, the development and features of IEEE standards for such arithmetic, desirable features of new implementations of floating-point hardware, and discusses work- in-progress aimed at making decimal floating-point arithmetic widely available across many architectures, operating systems, and programming languages. Keywords: binary arithmetic; decimal arithmetic; elementary functions; fixed-point arithmetic; floating-point arithmetic; interval arithmetic; mathcw library; range arithmetic; special functions; validated numerics PACS: 02.30.Gp Special functions DEDICATION This article is dedicated to N. Yngve Öhrn, my mentor, thesis co-advisor, and long-time friend, on the occasion of his retirement from academia. It is also dedicated to William Kahan, Michael Cowlishaw, and the late James Wilkinson, with much thanks for inspiration. WHAT IS FLOATING-POINT ARITHMETIC? Floating-point arithmetic is a technique for storing and operating on numbers in a computer where the base, range, and precision of the number system are usually fixed by the computer design. Conceptually, a floating-point number has a sign,anexponent,andasignificand (the older term mantissa is now deprecated), allowing a representation of the form .1/sign significand baseexponent.Thebase point in the significand may be at the left, or after the first digit, or at the right. The point and the base are implicit in the representation: neither is stored. The sign can be compactly represented by a single bit, and the exponent is most commonly a biased unsigned bitfield, although some historical architectures used a separate exponent sign and an unbiased exponent.
    [Show full text]
  • Catalogue of Common Sorts of Bug
    Warwick Research Software Engineering Catalogue of Common Sorts of Bug H. Ratcliffe and C.S. Brady Senior Research Software Engineers \The Angry Penguin", used under creative commons licence from Swantje Hess and Jannis Pohlmann. October 2, 2018 1 Bug Catalogue 1.1 Logic or algorithm bugs Logic bugs are a catch-all for when your program does what it ought to, but not what you wanted; effectively you have written the wrong correct program. They can be very tricky to find, because they often arise from some misunderstanding of what you are trying to achieve. You may find it helpful to look over your plan (you did make a plan, right?) Symptoms: • incorrect answers • poor performance Examples: • Finding the minimum of a list the wrong way: if your data is sorted, the minimum value must be the first in the list but sorting the data to just find the minimum item is silly.1 • Missing parts of a range: 1 INTEGER a , c 2 a = GET NEXT INTEGER( ) 3 IF ( a < 0 ) THEN 4 c = 1 5 ELSE IF ( a > 0 AND a < 5) THEN 6 c = 0 7 ELSE IF ( a >= 5) THEN 8 c = 2 9 END Notice that the case of a == 0 has been missed, and in this case c is undefined. • Single branch of sqrt: calculating a from something like a2 = 9, and forgetting that a can be either 3 or -3, introducing a sign error in further calculations • Most typos: mistyped names, function assignment rather than call (Python), missing semicolon (C). Mis-use of operators, for example using & and && (C).
    [Show full text]