DOCSLIB.ORG

Automatic Generation of High-Performance Quantized Machine Learning Kernels

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Automatic Generation of High-Performance Quantized Machine Learning Kernels Automatic Generation of High-Performance Quantized Machine Learning Kernels Meghan Cowan Thierry Moreau Tianqi Chen University of Washington University of Washington University of Washington Seattle, WA, USA Seattle, WA, USA Seattle, WA, USA [email protected] [email protected] [email protected] James Bornholt Luis Ceze University of Texas at Austin University of Washington Austin, TX, USA Seattle, WA, USA [email protected] [email protected] Abstract ACM Reference Format: Quantization optimizes machine learning inference for re- Meghan Cowan, Thierry Moreau, Tianqi Chen, James Bornholt, source constrained environments by reducing the precision and Luis Ceze. 2020. Automatic Generation of High-Performance Quantized Machine Learning Kernels. In Proceedings of the 18th of its computation. In the extreme, even single-bit computa- ACM/IEEE International Symposium on Code Generation and Op- tions can produce acceptable results at dramatically lower timization (CGO ’20), February 22ś26, 2020, San Diego, CA, USA. cost. But this ultra-low-precision quantization is difficult ACM, New York, NY, USA, 12 pages. https://doi.org/10.1145/3368826. to exploit because extracting optimal performance requires 3377912 hand-tuning both high-level scheduling decisions and low- level implementations. As a result, practitioners settle for a few predefined quantized kernels, sacrificing optimality and 1 Introduction restricting their ability to adapt to new hardware. Machine learning has seen successes in a wide variety of ap- This paper presents a new automated approach to imple- plication areas. These successes come at the cost of dramatic menting quantized inference for machine learning models. increases in the memory and compute demands of new mod- We integrate the choice of how to lay out quantized val- els such as convolutional neural networks. For example, per- ues into the scheduling phase of a machine learning com- forming a single prediction (an inference) with state-of-the- piler, allowing it to be optimized in concert with tiling and art image recognition models use tens of millions of parame- parallelization decisions. After scheduling, we use program ters and billions of floating-point operations [22, 26]. Quan- synthesis to automatically generate efficient low-level op- tization techniques reduce the scale of these models to make erator implementations for the desired precision and data them practical in resource-constrained environments such as layout. We scale up synthesis using a novel reduction sketch smartphones and edge devices [17]. Quantization reduces the that exploits the structure of matrix multiplication. On a bit-width of both the data (weights) and the computations ResNet18 model, our generated code outperforms an opti- (additions and multiplications) used to execute the model. In . mized floating-point baseline by up to 3 9×, and a state-of- the extreme, even single-bit weights and computations can . the-art quantized implementation by up to 16 6×. still yield an effective model (with under 10% accuracy loss) CCS Concepts · Software and its engineering → Com- while being orders of magnitude more efficient [12, 30]. This pilers. trade-off is appropriate for applications such as facial recog- Keywords quantization, machine learning, synthesis nition, which may use a lower-accuracy, on-device model as a low-pass filter to decide whether to offload to thecloud Permission to make digital or hard copies of all or part of this work for for more expensive, more accurate prediction [24]. personal or classroom use is granted without fee provided that copies Quantized inference evokes a classic accuracyśperformance are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights trade-off across a large space of potential configurations. At for components of this work owned by others than the author(s) must training time, users must select quantized bit-widths to meet be honored. Abstracting with credit is permitted. To copy otherwise, or their performance and accuracy targets (we do not consider republish, to post on servers or to redistribute to lists, requires prior specific training in this paper). They must then efficiently implement permission and/or a fee. Request permissions from [email protected]. inference at that chosen quantization level for the target CGO ’20, February 22ś26, 2020, San Diego, CA, USA platform. Commodity hardware rarely supports ultra-low- © 2020 Copyright held by the owner/author(s). Publication rights licensed to ACM. bitwidth operations, and so naive quantized implementations ACM ISBN 978-1-4503-7047-9/20/02...$15.00 can easily be slower than full-precision versions. These chal- https://doi.org/10.1145/3368826.3377912 lenges are exacerbated by pressures across the stack. Rapid 305 CGO ’20, February 22ś26, 2020, San Diego, CA, USA M. Cowan, T. Moreau, T. Chen, J. Bornholt, and L. Ceze progress in new machine learning models demands new spe- assembly code that produces the desired output while mini- cialized quantized operations (e.g., new matrix sizes or new mizing the cost function. We implement reduction sketches combinations of precisions). Meanwhile, a broad spectrum of and our synthesis engine in the Rosette solver-aided lan- hardware backends each require different implementations guage [37], and integrate the generated code into the TVM to best realize performance gains from quantization. machine learning compiler [4]. Using program synthesis In practice, users deploying quantized inference are re- further supports our goal of retargetabilityÐporting the op- stricted to a small class of pre-set configurations with cor- erators to a new hardware architecture (e.g., new vector responding hand-tuned inference implementations. Devel- extensions) requires only a specification of that hardware’s oping these hand-tuned implementations is a tedious and instruction set, from which the necessary implementations labor-intensive process that requires intimate knowledge can be automatically synthesized. of the target architecture. One developer estimated that im- We evaluate our approach by using it to implement quan- plementing and tuning a single quantization configuration tized ResNet-18 [15] image classification models. The re- on a single smartphone platform [42] required four person- sulting implementation on a low-power ARM CPU outper- months of work [40]. Similarly, the authors of another li- forms a floating-point version by 3.9× and is comparable brary [39] spent about a month implementing a single con- to a state-of-the-art hand-written quantized implementa- figuration [38]. These hand-tuned implementations are often tion [39] when configured in the same way. However, our brittle and difficult to extend with new optimizations; both automated approach allows us to apply optimizations such as authors reported difficulty with multi-threading and new parallelizationÐwhereas the hand-written implementation vector extensions. With the proliferation of new models and is single-threadedÐthat yield up to a 16.6× speedup over hardware backends, investing this level of effort for every the hand-written quantized implementation. We show that configuration is not a scalable approach. our quantized implementations shift the Pareto frontier of This paper presents a new automated approach to generat- the accuracyśperformance trade-off, offering 2.3× higher ing efficient code for quantized machine learning inference. throughput than existing implementations at the same 22% Our approach comprises two parts. First, we integrate the accuracy loss. in edge applications such as video process- choice of how to perform bit-slicing (i.e., how to arrange ing [24]. Automation also allows us to explore the space quantized values in memory) into the scheduling phase of of possible quantization configurations, performing a limit a machine learning compiler [4]. Our insight is that quanti- study of the potential speedups available from quantization. zation is best viewed as a scheduling decision, in the same In summary, this paper makes the following contributions: way that tensor libraries treat tiling and parallelism as part • Quantization-aware scheduling for compilation of ma- of the schedule space [29]. The bit-slicing choice is critical chine learning models to generating efficient code: different layouts influence both • An automated scheduling approach to tuning quan- spatial locality and the ability to apply different low-level tized implementation for new models and hardware operators for efficient computation (e.g., without scatter or • A new program synthesis approach to generating quan- gather operations on vector lanes). Making quantization a tized low-level tensor kernels scheduling decision also allows us to extract better perfor- • Results showing we can automatically generate code mance using holistic schedules that combine quantization that executes quantized models up to 12.4× faster than with other transformations such as loop tiling and paralleliza- state-of-the-art hand-written code. tion. Finally, integrating quantization into scheduling allows us to apply existing automated scheduling techniques [5] to optimize a model for a given quantization configuration and 2 Quantized Models hardware architecture. This automation supports flexible Quantized machine learning models promise to reduce the experimentation and retargeting to new hardware without compute and memory demands of inference while maintain- reengineering
Load more

Recommended publications
  • System Design for a Computational-RAM Logic-In-Memory Parailel-Processing Machine
    System Design for a Computational-RAM Logic-In-Memory ParaIlel-Processing Machine Peter M. Nyasulu, B .Sc., M.Eng. A thesis submitted to the Faculty of Graduate Studies and Research in partial fulfillment of the requirements for the degree of Doctor of Philosophy Ottaw a-Carleton Ins titute for Eleceical and Computer Engineering, Department of Electronics, Faculty of Engineering, Carleton University, Ottawa, Ontario, Canada May, 1999 O Peter M. Nyasulu, 1999 National Library Biôiiothkque nationale du Canada Acquisitions and Acquisitions et Bibliographie Services services bibliographiques 39S Weiiington Street 395. nie WeUingtm OnawaON KlAW Ottawa ON K1A ON4 Canada Canada The author has granted a non- L'auteur a accordé une licence non exclusive licence allowing the exclusive permettant à la National Library of Canada to Bibliothèque nationale du Canada de reproduce, ban, distribute or seU reproduire, prêter, distribuer ou copies of this thesis in microform, vendre des copies de cette thèse sous paper or electronic formats. la forme de microficbe/nlm, de reproduction sur papier ou sur format électronique. The author retains ownership of the L'auteur conserve la propriété du copyright in this thesis. Neither the droit d'auteur qui protège cette thèse. thesis nor substantial extracts fkom it Ni la thèse ni des extraits substantiels may be printed or otherwise de celle-ci ne doivent être imprimés reproduced without the author's ou autrement reproduits sans son permission. autorisation. Abstract Integrating several 1-bit processing elements at the sense amplifiers of a standard RAM improves the performance of massively-paralle1 applications because of the inherent parallelism and high data bandwidth inside the memory chip.
    [Show full text]
  • Block Ciphers: Fast Implementations on X86-64 Architecture
    Block Ciphers: Fast Implementations on x86-64 Architecture University of Oulu Department of Information Processing Science Master’s Thesis Jussi Kivilinna May 20, 2013 Abstract Encryption is being used more than ever before. It is used to prevent eavesdropping on our communications over cell phone calls and Internet, securing network connections, making e-commerce and e-banking possible and generally hiding information from unwanted eyes. The performance of encryption functions is therefore important as slow working implementation increases costs. At server side faster implementation can reduce the required capacity and on client side it can lower the power usage. Block ciphers are a class of encryption functions that are typically used to encrypt large bulk data, and thus make them a subject of many studies when endeavoring greater performance. The x86-64 architecture is the most dominant processor architecture in server and desktop computers; it has numerous different instruction set extensions, which make the architecture a target of constant new research on fast software implementations. The examined block ciphers – Blowfish, AES, Camellia, Serpent and Twofish – are widely used in various applications and their different designs make them interesting objects of investigation. Several optimization techniques to speed up implementations have been reported in previous research; such as the use of table look-ups, bit-slicing, byte-slicing and the utilization of “out-of-order” scheduling capabilities. We examine these different techniques and utilize them to construct new implementations of the selected block ciphers. Focus with these new implementations is in modes of operation which allow multiple blocks to be processed in parallel; such as the counter mode.
    [Show full text]
  • An FPGA-Accelerated Embedded Convolutional Neural Network
    Master Thesis Report ZynqNet: An FPGA-Accelerated Embedded Convolutional Neural Network (edit) (edit) 1000ch 1000ch FPGA 1000ch Network Analysis Network Analysis 2x512 > 1024 2x512 > 1024 David Gschwend [email protected] SqueezeNet v1.1 b2a ext7 conv10 2x416 > SqueezeNet SqueezeNet v1.1 b2a ext7 conv10 2x416 > SqueezeNet arXiv:2005.06892v1 [cs.CV] 14 May 2020 Supervisors: Emanuel Schmid Felix Eberli Professor: Prof. Dr. Anton Gunzinger August 2016, ETH Zürich, Department of Information Technology and Electrical Engineering Abstract Image Understanding is becoming a vital feature in ever more applications ranging from medical diagnostics to autonomous vehicles. Many applications demand for embedded solutions that integrate into existing systems with tight real-time and power constraints. Convolutional Neural Networks (CNNs) presently achieve record-breaking accuracies in all image understanding benchmarks, but have a very high computational complexity. Embedded CNNs thus call for small and efficient, yet very powerful computing platforms. This master thesis explores the potential of FPGA-based CNN acceleration and demonstrates a fully functional proof-of-concept CNN implementation on a Zynq System-on-Chip. The ZynqNet Embedded CNN is designed for image classification on ImageNet and consists of ZynqNet CNN, an optimized and customized CNN topology, and the ZynqNet FPGA Accelerator, an FPGA-based architecture for its evaluation. ZynqNet CNN is a highly efficient CNN topology. Detailed analysis and optimization of prior topologies using the custom-designed Netscope CNN Analyzer have enabled a CNN with 84.5 % top-5 accuracy at a computational complexity of only 530 million multiply- accumulate operations. The topology is highly regular and consists exclusively of convolu- tional layers, ReLU nonlinearities and one global pooling layer.
    [Show full text]
  • Advanced Synthesis Cookbook
    Advanced Synthesis Cookbook Advanced Synthesis Cookbook 101 Innovation Drive San Jose, CA 95134 www.altera.com MNL-01017-6.0 Document last updated for Altera Complete Design Suite version: 11.0 Document publication date: July 2011 © 2011 Altera Corporation. All rights reserved. ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS and STRATIX are Reg. U.S. Pat. & Tm. Off. and/or trademarks of Altera Corporation in the U.S. and other countries. All other trademarks and service marks are the property of their respective holders as described at www.altera.com/common/legal.html. Altera warrants performance of its semiconductor products to current specifications in accordance with Altera’s standard warranty, but reserves the right to make changes to any products and services at any time without notice. Altera assumes no responsibility or liability arising out of the application or use of any information, product, or service described herein except as expressly agreed to in writing by Altera. Altera customers are advised to obtain the latest version of device specifications before relying on any published information and before placing orders for products or services. Advanced Synthesis Cookbook July 2011 Altera Corporation Contents Chapter 1. Introduction Blocks and Techniques . 1–1 Simulating the Examples . 1–1 Using a C Compiler . 1–2 Chapter 2. Arithmetic Introduction . 2–1 Basic Addition . 2–2 Ternary Addition . 2–2 Grouping Ternary Adders . 2–3 Combinational Adders . 2–3 Double Addsub/ Basic Addsub . 2–3 Two’s Complement Arithmetic Review . 2–4 Traditional ADDSUB Unit . 2–4 Compressors (Carry Save Adders) . 2–5 Compressor Width 6:3 .
    [Show full text]
  • Usuba, Optimizing Bitslicing Compiler Darius Mercadier
    Usuba, Optimizing Bitslicing Compiler Darius Mercadier To cite this version: Darius Mercadier. Usuba, Optimizing Bitslicing Compiler. Programming Languages [cs.PL]. Sorbonne Université (France), 2020. English. tel-03133456 HAL Id: tel-03133456 https://tel.archives-ouvertes.fr/tel-03133456 Submitted on 6 Feb 2021 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. THESE` DE DOCTORAT DE SORBONNE UNIVERSITE´ Specialit´ e´ Informatique Ecole´ doctorale Informatique, Tel´ ecommunications´ et Electronique´ (Paris) Present´ ee´ par Darius MERCADIER Pour obtenir le grade de DOCTEUR de SORBONNE UNIVERSITE´ Sujet de la these` : Usuba, Optimizing Bitslicing Compiler soutenue le 20 novembre 2020 devant le jury compose´ de : M. Gilles MULLER Directeur de these` M. Pierre-Evariste´ DAGAND Encadrant de these` M. Karthik BHARGAVAN Rapporteur Mme. Sandrine BLAZY Rapporteur Mme. Caroline COLLANGE Examinateur M. Xavier LEROY Examinateur M. Thomas PORNIN Examinateur M. Damien VERGNAUD Examinateur Abstract Bitslicing is a technique commonly used in cryptography to implement high-throughput parallel and constant-time symmetric primitives. However, writing, optimizing and pro- tecting bitsliced implementations by hand are tedious tasks, requiring knowledge in cryptography, CPU microarchitectures and side-channel attacks. The resulting programs tend to be hard to maintain due to their high complexity.
    [Show full text]
  • Fpnew: an Open-Source Multi-Format Floating-Point Unit Architecture For
    1 FPnew: An Open-Source Multi-Format Floating-Point Unit Architecture for Energy-Proportional Transprecision Computing Stefan Mach, Fabian Schuiki, Florian Zaruba, Student Member, IEEE, and Luca Benini, Fellow, IEEE Abstract—The slowdown of Moore’s law and the power wall Internet of Things (IoT) domain. In this environment, achiev- necessitates a shift towards finely tunable precision (a.k.a. trans- ing high energy efficiency in numerical computations requires precision) computing to reduce energy footprint. Hence, we need architectures and circuits which are fine-tunable in terms of circuits capable of performing floating-point operations on a wide range of precisions with high energy-proportionality. We precision and performance. Such circuits can minimize the present FPnew, a highly configurable open-source transprecision energy cost per operation by adapting both performance and floating-point unit (TP-FPU) capable of supporting a wide precision to the application requirements in an agile way. The range of standard and custom FP formats. To demonstrate the paradigm of “transprecision computing” [1] aims at creating flexibility and efficiency of FPnew in general-purpose processor a holistic framework ranging from algorithms and software architectures, we extend the RISC-V ISA with operations on half-precision, bfloat16, and an 8bit FP format, as well as SIMD down to hardware and circuits which offer many knobs to vectors and multi-format operations. Integrated into a 32-bit fine-tune workloads. RISC-V core, our TP-FPU can speed up execution of mixed- The most flexible and dynamic way of performing nu- precision applications by 1.67x w.r.t.
    [Show full text]
  • Bit Slicing Repetitive Logic
    Bit Slicing Repetitive Logic Where logic blocks are duplicated within a system, there is much to gain from optimization. • Optimize single block for size. The effect is amplified by the number of identical blocks. • Optimize interconnect. With careful design, the requirement for routing channels between the blocks can be eliminated. Interconnection is by butting in 2 dimensions. 4001 Bit Slicing Instead of creating an ALU function by function, we create it slice by slice. A[7:0] B[7:0] A[7] B[7] A[0] B[0] Z[7] Z[7:0] Z[0] • Each bit slice is a full 1-bit ALU. • N are used to create an N–bit ALU. 4002 Bit Slicing FnAND FnOR FnXOR FnADD FnSHR FnSHL Cout SHRin A Z FA B A Cin A SHLin Z= A&B Z= A|B Z= A!=B Z= A+B Z= A>>1 Z= A<<1 • Simple 1–bit ALU. • The control lines select which function of A and B is fed to Z. • Some functions, e.g. Add, require extra data I/O. 4003 Bit Slicing FnAND FnAND Z Z Vdd GND B B A A • Data busses horizontal, control lines vertical. • Compact gate matrix implementation1. 1bit slice designs can also be built around standard cells although the full custom approach used here gives better results 4004 • Bit Sliced ALU (3-bits, scalable). CarryOut ShiftRightIn ShiftLeftOut FnAND FnOR FnXOR FnAdd FnSHR FnSHL Cout SHRin A Z[2] FA B[2] A[2] Cin A SHLin Cout SHRin A Z[1] FA B[1] A[1] Cin A SHLin Cout SHRin A Z[0] FA B[0] A[0] Cin A SHLin CarryIn ShiftRightOut ShiftLeftIn 4005 Bit Slicing We can extend this principle to the whole datapath.
    [Show full text]
  • Formally Verified Big Step Semantics out of X86-64 Binaries
    Formally Verified Big Step Semantics out of x86-64 Binaries Ian Roessle Freek Verbeek Binoy Ravindran Virginia Tech, USA Virginia Tech, USA Virginia Tech, USA [email protected] [email protected] [email protected] Abstract or systems where source code is unavailable due to propri- This paper presents a methodology for generating formally etary reasons. Various safety-critical systems in automotive, proven equivalence theorems between decompiled x86-64 aerospace, medical and military domains are built out of machine code and big step semantics. These proofs are built components where the source code is not available [38]. on top of two additional contributions. First, a robust and In such a context, certification plays a crucial role. Certi- tested formal x86-64 machine model containing small step fication can require a compliance proof based on formal semantics for 1625 instructions. Second, a decompilation- methods [32, 41]. Bottom-up formal verification can aid in into-logic methodology supporting both x86-64 assembly obtaining high levels of assurance for black-box components and machine code at large scale. This work enables black- running on commodity hardware, such as x86-64. box binary verification, i.e., formal verification of a binary A binary typically consists of blocks composed by control where source code is unavailable. As such, it can be applied flow. In this paper, a block is defined as a sequence of instruc- to safety-critical systems that consist of legacy components, tions that can be modeled with only if-then-else statements or components whose source code is unavailable due to (no loops).
    [Show full text]
  • Study of the Posit Number System: a Practical Approach
    STUDY OF THE POSIT NUMBER SYSTEM: A PRACTICAL APPROACH Raúl Murillo Montero DOBLE GRADO EN INGENIERÍA INFORMÁTICA – MATEMÁTICAS FACULTAD DE INFORMÁTICA UNIVERSIDAD COMPLUTENSE DE MADRID Trabajo Fin Grado en Ingeniería Informática – Matemáticas Curso 2018-2019 Directores: Alberto Antonio del Barrio García Guillermo Botella Juan Este documento está preparado para ser imprimido a doble cara. STUDY OF THE POSIT NUMBER SYSTEM: A PRACTICAL APPROACH Raúl Murillo Montero DOUBLE DEGREE IN COMPUTER SCIENCE – MATHEMATICS FACULTY OF COMPUTER SCIENCE COMPLUTENSE UNIVERSITY OF MADRID Bachelor’s Degree Final Project in Computer Science – Mathematics Academic year 2018-2019 Directors: Alberto Antonio del Barrio García Guillermo Botella Juan To me, mathematics, computer science, and the arts are insanely related. They’re all creative expressions. Sebastian Thrun Acknowledgements I have to start by thanking Alberto Antonio del Barrio García and Guillermo Botella Juan, the directors of this amazing project. Thank you for this opportunity, for your support, your help and for trusting me. Without you any of this would have ever been possible. Thanks to my family, who has always supported me and taught me that I am only limited by the limits I choose to put on myself. Finally, thanks to all of you who have accompanied me on this 5 year journey. Sharing classes, exams, comings and goings between the two faculties with you has been a wonderful experience I will always remember. vii Abstract The IEEE Standard for Floating-Point Arithmetic (IEEE 754) has been for decades the standard for floating-point arithmetic and is implemented in a vast majority of modern computer systems.
    [Show full text]
  • SSE Implementation of Multivariate Pkcs on Modern X86 Cpus
    SSE Implementation of Multivariate PKCs on Modern x86 CPUs Anna Inn-Tung Chen1, Ming-Shing Chen2, Tien-Ren Chen2, Chen-Mou Cheng1, Jintai Ding3, Eric Li-Hsiang Kuo2, Frost Yu-Shuang Lee1, and Bo-Yin Yang2 1 National Taiwan University, Taipei, Taiwan {anna1110,doug,frost}@crypto.tw 2 Academia Sinica, Taipei, Taiwan {mschen,trchen1103,lorderic,by}@crypto.tw 3 University of Cincinnati, Cincinnati, Ohio, USA [email protected] Abstract. Multivariate Public Key Cryptosystems (MPKCs) are often touted as future-proofing against Quantum Computers. It also has been known for efficiency compared to “traditional” alternatives. However, this advantage seems to erode with the increase of arithmetic resources in modern CPUs and improved algorithms, especially with respect to El- liptic Curve Cryptography (ECC). In this paper, we show that hard- ware advances do not just favor ECC. Modern commodity CPUs also have many small integer arithmetic/logic resources, embodied by SSE2 or other vector instruction sets, that are useful for MPKCs. In partic- ular, Intel’s SSSE3 instructions can speed up both public and private maps over prior software implementations of Rainbow-type systems up to 4×.Furthermore,MPKCs over fields of relatively small odd prime characteristics can exploit SSE2 instructions, supported by most mod- ern 64-bit Intel and AMD CPUs. For example, Rainbow over F31 can be up to 2× faster than prior implementations of similarly-sized systems over F16. Here a key advance is in using Wiedemann (as opposed to Gauss) solvers to invert the small linear systems in the central maps. We explain the techniques and design choices in implementing our chosen MPKC instances over fields such as F31, F16 and F256.
    [Show full text]
  • High-Throughput Elliptic Curve Cryptography Using AVX2 Vector Instructions
    High-Throughput Elliptic Curve Cryptography Using AVX2 Vector Instructions Hao Cheng, Johann Großsch¨adl,Jiaqi Tian, Peter B. Rønne, and Peter Y. A. Ryan DCS and SnT, University of Luxembourg 6, Avenue de la Fonte, L{4364 Esch-sur-Alzette, Luxembourg {hao.cheng,johann.groszschaedl,peter.roenne,peter.ryan}@uni.lu [email protected] Abstract. Single Instruction Multiple Data (SIMD) execution engines like Intel's Advanced Vector Extensions 2 (AVX2) offer a great potential to accelerate elliptic curve cryptography compared to implementations using only basic x64 instructions. All existing AVX2 implementations of scalar multiplication on e.g. Curve25519 (and alternative curves) are optimized for low latency. We argue in this paper that many real-world applications, such as server-side SSL/TLS handshake processing, would benefit more from throughput-optimized implementations than latency- optimized ones. To support this argument, we introduce a throughput- optimized AVX2 implementation of variable-base scalar multiplication on Curve25519 and fixed-base scalar multiplication on Ed25519. Both implementations perform four scalar multiplications in parallel, where each uses a 64-bit element of a 256-bit vector. The field arithmetic is based on a radix-229 representation of the field elements, which makes it possible to carry out four parallel multiplications modulo a multiple of p = 2255 − 19 in just 88 cycles on a Skylake CPU. Four variable-base scalar multiplications on Curve25519 require less than 250;000 Skylake cycles, which translates to a throughput of 32;318 scalar multiplications per second at a clock frequency of 2 GHz. For comparison, the to-date best latency-optimized AVX2 implementation has a throughput of some 21;000 scalar multiplications per second on the same Skylake CPU.
    [Show full text]
  • Floating-Point Formats.Pdf
    [Next] [Up/Previous] mrob Numbers Largenum Sequences Mandelbrot Men Core Values Search: • n particular: 0 00000000 00000000000000000000000 = 0 0 00000000 00000000000000000000001 = +1 * 2( -126) * 0.000000000000000000000012 = 2(-149) (Smallest positive value) ( -126) 0 00000000 10000000000000000000000 = +1 * 2 * 0.12 = 2**(-127) ( 1-127) 0 00000001 00000000000000000000000 = +1 * 2 * 1.02 = 2**(-126) (128-127) 0 10000000 00000000000000000000000 = +1 * 2 * 1.02 = 2 1 0 0 1 .5 .5 2^1 (129-127) 0 10000001 10100000000000000000000 = +1 * 2 * 1.1012 = 6.5 1.10 1 1 0 1 1 .5 .25 .125 1.624*2^2= 6.5 0 11111110 11111111111111111111111 = +1 * 2(254-127) * 1.111111111111111111111112 (Most positive finite value) 0 11111111 00000000000000000000000 = Infinity 0 11111111 00000100000000000000000 = NaN 1 00000000 00000000000000000000000 = -0 (128-127) 1 10000000 00000000000000000000000 = -1 * 2 * 1.02 = -2 (129-127) 1 10000001 10100000000000000000000 = -1 * 2 * 1.1012 = -6.5 1 11111110 11111111111111111111111 = -1 * 2(254-127) * 1.111111111111111111111112 (Most negative finite value) 1 11111111 00000000000000000000000 = -Infinity 1 11111111 00100010001001010101010 = NaNSearch • Site Map About PSC Contacts Employment RSS Feed PITTSBURGH SUPERCOMPUTING CENTER PSC Users Outreach & Training Services Research News Center Floating Point Introduction Int and unsigned int are approximations to the set of integers and the set of natural numbers. Unlike int and unsigned, the set of integers and the set of natural numbers is infinite. Because the set of int is finite, there is a maximum int and a minimum int. Ints are also contiguous. That is, between the minimum and maximum int, there are no missing values. To summarize: • The set of valid ints is finite.
    [Show full text]