Theoretical Peak FLOPS per instruction set on modern Intel CPUs Romain Dolbeau Bull – Center for Excellence in Parallel Programming Email: [email protected] Abstract—It used to be that evaluating the theoretical and potentially multiple threads per core. Vector of peak performance of a CPU in FLOPS (floating point varying sizes. And more sophisticated instructions. operations per seconds) was merely a matter of multiplying Equation2 describes a more realistic view, that we the frequency by the number of floating-point instructions will explain in details in the rest of the paper, first per cycles. Today however, CPUs have features such as vectorization, fused multiply-add, hyper-threading or in general in sectionII and then for the specific “turbo” mode. In this paper, we look into this theoretical cases of Intel CPUs: first a simple one from the peak for recent full-featured Intel CPUs., taking into Nehalem/Westmere era in section III and then the account not only the simple absolute peak, but also the full complexity of the Haswell family in sectionIV. relevant instruction sets and encoding and the frequency A complement to this paper titled “Theoretical Peak scaling behavior of current Intel CPUs. FLOPS per instruction set on less conventional Revision 1.41, 2016/10/04 08:49:16 Index Terms—FLOPS hardware” [1] covers other computing devices. flop 9 I. INTRODUCTION > operation> High performance computing thrives on fast com- > > putations and high memory bandwidth. But before > operations => any code or even benchmark is run, the very first × micro − architecture instruction number to evaluate a system is the theoretical peak > > - how many floating-point operations the system > can theoretically execute in a given time. The most instructions> × > common measurement is the FLOPS, floating-point ; flops cycle operations per second. The simple view is: the more = (2) node FLOPS, the better. cycles 9 However, evaluating the peak FLOPS is not as × > second> easy as it looks. It used to be that multiplying the > > number of floating-point operations per cycle by the cores => number of cycles per second was enough. It was the × machine architecture socket simple equation in equation1. A node is a single > > computer (perhaps part of a cluster), and it’s easy to > sockets> define it’s speed. It has only one super-scalar core, × ;> no fancy vector, and this is the early 1990s. node II. CONTRIBUTION TO THE PEAK flops flops cores = × (1) In this section, we will detail the meaning of node second node each part of equation2. Of course, they are not But since then, many level of parallelism have fully independent, as explained in each relevant been introduced. Multiple physical silicon devices section. As indicated in equation2, the first three per nodes (“socket”), each with multiple cores, are part of the micro-architecture (how the CPU cycles is designed), whereas the last three are part of D. =second: frequency the machine architecture (how many devices, which How many cycles in a second. While it sounds devices, what speed). as the easiest factor to compute, it can be quite flop A. =operation: instructions tricky as well. Modern CPUs feature “turbo” mode How many floating point “operation” (in the whereas some of the cores can have a higher speed mathematical sense of the word) are embedded in than nominal. Energy-saving mode lower the fre- the semantic of the instruction. Usually, most time- quency. Recent Intel CPUs have a different base consuming operations (such as division) are simply frequency when exploiting certain instructions, in not taken into account, and only the simplest and particular AVX instructions - the very instructions most relevant are counted: addition, subtraction, we are trying to count. cores sockets multiplication. They all count as one. So any in- E. =socket & =node: multi-core struction encoding one of those operations is repre- 1 How many cores in a node, computed with the sented by the ratio =1, whereas the quite common two factors. However, care should be taken: while fused multiply-add (or any variant) d = a × b + c is 2 hyper-threading (or simultaneous) adds additional represented by the ratio =1. resources to the processors seen as “cores” by the operations B. =instruction: vector operating system, it does not add execution units. So How many (group of) operations are executed the number of cores does not take hyper-threading simultaneously by an instruction. This is the so- into account. called “vector” or “SIMD” mode. If an instruction III. AN EASY INTEL CPU: uses vector of 4 elements as operands, then each NEHALEM/WESTMERE of the mathematical operation(s) are executed four The first case study are the older Intel CPUs from times, for a ratio of 4= . This can vary greatly, 1 the Nehalem or Westmere generation1. Those CPUs from 1= for “scalar” instructions, to 16= for single- 1 1 use the SSE2 SIMD instruction set for floating- precision floating-point in an AVX-512 vector of 64 point operations. This instruction set features 128 bytes. bits (16 bytes) registers, capable of holding either 2 instructions C. =cycle: super-scalar double-precision or 4 single-precision elements. The How many instructions can be executed simulta- instruction set does not support fused multiply-add. neously in a cycle. This is trickier than it sounds. But it does support a scalar mode, with only one For instance, do Intel processors take into account element used per vector register. TableI summarizes the legacy x87 Floating-Point Unit(FPU)? Or only the number of FLOPS per cycle one can expect from the SIMD units? The general question is, are those the Nehalem family in the three possible modes - instructions homogeneous enough - in terms of the scalar, vector single-precision and vector double- previous two factors - to be represented by a simple precision. The value for scalar applies to both single fraction ? Usually yes, as we can for instance simply and double precision. ignore the legacy FPU and the complex instructions Then for a given CPU model we can add the such as division. So if a CPU can do two fused specific data (frequency, core per socket, socket per 2 multiply-add at once, the ratio will be =1; for two node) and obtain the raw per-node theoretical peak, arbitrary single-operation instruction (add, sub, mul) as is done in figureII for the Xeon X5650[2]. This 2 again a ratio of =1; and a fused multiply-add in however is an approximation - as the description of parallel with a single-operation instruction can be the CPU signals, “turbo” mode allows some core to 3 encompassed by a ratio of =2. The real complexity 1Most processors even older than Nehalem but supporting SSE2 comes from CPU with non-scalar floating-point would fall into the same category. Strictly speaking, SSE only sup- units where less than one instruction per cycle is ports single-precision floating-point operations, and SSE2 supports possible - then more complex calculations have to double-precision. Processor without SSE2 have to rely on x87 for double-precision arithmetic, and are not considered. In the remainder be made. Fortunately, few modern CPUs are in that of this paper, the term SSE will be used to describe the SSE & SSE2 category. combination, since both are mandatory on all x86-64 processors. TABLE I operations in the following ways (always ignoring THEORETICAL PER-CYCLE PEAK FOR NEHALEM/WESTMERE the legacy x87 FPU): SSE (Scalar) SSE (DP) SSE (SP) 1) SSE-encoded scalar mode; flop =operation 1 1 1 2) SSE-encoded double-precision vector mode; operations × =instruction 1 2 4 3) SSE-encoded single-precision vector mode; ×instructions= 2 2 2 cycle 4) AVX-encoded scalar mode; =flop = 2 4 8 cycle 5) AVX-encoded 128 bits double-precision vector mode; go up to 3.06 GHz, something we have not taken 6) AVX-encoded 128 bits single-precision vector into account. mode; 7) AVX-encoded 256 bits double-precision vector TABLE II mode; THEORETICAL PER-NODE PEAK FOR X5650 8) AVX-encoded 256 bits single-precision vector mode; SSE (Scalar) SSE (DP) SSE (SP) flop 9) AVX-encoded scalar mode (with FMA) =cycle 2 4 8 cycles × =second 2:67G 2:67G 2:67G 10) AVX-encoded 128 bits double-precision vector cores × =socket 6 6 6 mode (with FMA); sockets × =node 2 2 2 11) AVX-encoded 128 bits single-precision vector flops = =node 64G 128G 256G mode (with FMA); 12) AVX-encoded 256 bits double-precision vector mode (with FMA); IV. THE COMPLEX CASE: THE HASWELL FAMILY 13) AVX-encoded 256 bits single-precision vector The Intel Haswell (a.k.a. Xeon E5 v3) family is mode (with FMA); much more complex to evaluate. First we will talk That is no less than thirteen different ways of about the various SIMD modes, and then about the computing on floating-point data. The first three (1– frequency of the cores. 3) and last two (12, 13) seems pretty obvious. The A. Instruction sets first three are mentioned in the previous section. The last two are the new 256 bits-wide instructions 1) Description: Up to and including the West- with FMA support from Haswell, which are well mere family described above, the only vector mode known. However, AVX-encoded instructions are not was SSE. However, Intel has since introduced a behaving well with the older, SSE-encoded 128 new encoding scheme and instruction set in the bits instructions and should not be mixed. Also, Sandy Bridge micro-architecture, known as AVX, SSE is a two-operand instruction set (the output which pushed vector register sizes to 256 bits (32 2 share a register with an input) whereas AVX is bytes) .