Topic Overview CZ4102 – High Performance Computing • Implicit Parallelism: Trends in Microprocessor Architectures Lectures 2 and 3: The Hardware • Limitations of Memory System Performance Considerations • Dichotomy of Parallel Computing Platforms • Communication Model of Parallel Platforms - A/Prof Tay Seng Chuan • Physical Organization of Parallel Platforms • Communication Costs in Parallel Machines • Messaging Cost Models and Routing Mechanisms Reference: ``Introduction to Parallel Computing'‘–Chapter 2. • Mapping Techniques 1 2 Implicit Parallelism: Trends in Scope of Parallelism Microprocessor Architectures • Different applications utilize different aspects of parallelism - e.g., • Current processors use these resources in multiple functional units data intensive applications utilize high aggregate throughput, server applications utilize high aggregate network bandwidth, and scientific E.g.: Add R1, R2 (i) Instruction Fetch, (ii) Instruction Decode, (iii) applications typically utilize high processing and memory system Instruction Execute performance. • It is important to understand each of these performance bottlenecks and their interacting effect. • Consider these two instructions: Add R1, R2 Add R2, R3 The precise manner in which these instructions are selected and executed provides impressive diversity in architectures with different performance and for different purpose. (Each architecture has its own merits and pitfalls.) 3 4 Pipelining and Superscalar Execution Pipelining and Superscalar Execution • Pipelining, however, has several limitations. • Pipelining overlaps various stages of instruction • The speed of a pipeline is eventually limited by the slowest stage. execution to achieve performance. • At a high level of abstraction, an instruction can be executed while the next one is being decoded and the next one is being fetched. 1 unit per • This is akin to an assembly line, e.g., for manufacture of 20 Second 20 Second 130 Second 20 Second 20 Second ?? second cars. • Conventional processors rely on very deep pipelines (20 stage pipelines in state-of-the-art Pentium processors). • However, in typical program traces, every 5th to 6th instruction is a conditional jump (such as in if-else, switch-case)! This requires very accurate branch prediction. • The penalty of a mis-prediction grows with the depth of the pipeline, 5 since a larger number of instructions will have to be flushed. 6 Superscalar Execution: An Example Pipelining and Superscalar Execution • One simple way of alleviating these bottlenecks is to use multiple pipelines. switch (…) { case 1: case 2: case 3: case 4: } • Selecting which pipeline for execution becomes a challenge. 7 Example of a two-way superscalar execution of instructions. 8 Superscalar Execution: An Example Superscalar Execution: An Example • In the above example, there is some wastage of resources due to data dependencies. • The example also illustrates that different instruction mixes with identical semantics can take significantly different execution time. • (i), (ii) and (iii) actually produce the same answer in 9 @2000. 10 Superscalar Execution: Superscalar Execution Issue Mechanisms • Scheduling of instructions is determined by a number of • In the simpler model, instructions can be issued only in factors: the order in which they are encountered. That is, if the – True Data Dependency: The result of one operation is an input to second instruction cannot be issued because it has a the next. E.g., A = B+C; D = A+E; data dependency with the first, only one instruction is – Resource Dependency: Two operations require the same issued in the cycle. This is called in-order issue resource. E.g., fprintf (indata, “%d”, a); fprintf (indata, “%d”, b); (sequentialization). If this model is preferred, we do not have to study CZ4102 any more. – Branch Dependency: Scheduling instructions across conditional branch statements cannot be done deterministically before hands. • In a more aggressive model, instructions can be issued – The scheduler, a piece of hardware looks at a large number of out of order. In this case, if the second instruction has instructions in an instruction queue and selects appropriate number data dependencies with the first, but the third instruction of instructions to execute concurrently based on these factors. does not, the first and third instructions can be co- scheduled. This is also called dynamic issue (speculation). • Performance of in-order issue is generally limited. Why? 11 12 Superscalar Execution: Very Long Instruction Word (VLIW) Efficiency Considerations Processors • Not all functional units can be kept busy at all times. • The hardware cost and complexity of the superscalar scheduler is a • If during a cycle, no functional units (FN) are utilized, this is referred to as major consideration in processor design. vertical waste. FN1 FN2 • To address this issues, VLIW processors rely on compile time analysis to identify and bundle together instructions that can be executed concurrently. • These instructions are packed and dispatched together, and thus the name very long instruction word is used. Add R1, R2 Sub R4, R3 Mul R2, R5 Div R1, R7 • If during a cycle, only some of the functional units are utilized, this is referred to as horizontal waste. • Due to limited parallelism in typical instruction traces, dependencies, or the 4-Way VLIW inability of the scheduler to extract parallelism, the performance of superscalar processors is eventually limited. • Conventional microprocessors typically support four-way superscalar execution. 13 14 How to bundle (pack) the Very Long Instruction Word (VLIW) instructions in VLIW? Processors: Considerations • Sequence of execution: • Hardware aspect is simpler. add R1, R2 // add R2 to R1 • Compiler has a bigger context from which to select co- sub R2, R1 // subtract R1 from R3 scheduled instructions. (More work for compiler.) add R3, R4 • Compilers, however, do not have runtime information such as cache misses. Scheduling is, therefore, sub R4, R3 inherently conservative. Solution: • Branch and memory prediction is more difficult. add R1, R2 sub R2,R1 • VLIW performance is highly dependent on the compiler. (1) add R3, R4 sub R4,R3 A number of techniques such as loop unrolling, speculative execution, branch prediction are critical. or • Typical VLIW processors are limited to 4-way to 8-way add R1, R2 add R3, R4 parallelism. (2) sub R2, R1 sub R4, R3 15 16 Limitations of Memory Latency: An Example Memory System Performance • Memory system, and not processor speed, is often the bottleneck for • Consider a processor operating at 1 GHz (1 ns clock) connected to many applications. a DRAM with a latency of 100 ns (no caches). Assume that the • Memory system performance is largely captured by two parameters, processor has two multiply-add units and is capable of executing latency and bandwidth. four instructions in each cycle of 1 ns. The following observations • Latency is the time from the issue of a memory request to the time follow: the data is available at the processor. (Waiting time until the first – Since the memory latency is equal to 100 cycles and block size is one data is received. Consider the example of a fire-hose. If the water word, every time a memory request is made, the processor must wait comes out of the hose two seconds after the hydrant is turned on, 100 cycles before it can start to process the data. This is a serious the latency of the system is two seconds. If you want immediate drawback. response from the hydrant, it is important to reduce latency. ) – The peak processor rating (assume no memory access) is computed as • Bandwidth is the rate at which data can be pumped to the processor follows: by the memory system. (Once the water starts flowing, if the hydrant Assume 4 instructions are executed on registers, peak processing rating delivers water at the rate of 5 gallons/second, the bandwidth of the = (4 Instructions)/(1 ns) = 4 GFLOPS. But this is not possible if system is 5 gallons/second. If you want to fight big fires, you need a memory access is needed. We will see it in the next slide. high bandwidth of water.) GFLOPS: Giga Floating Point Operations per Second 17 18 Improving Effective Memory Seriousness of Memory Latency Latency Using Caches • On the above architecture with memory latency of 100 ns, consider the • Caches are small and fast memory elements between the processor problem of computing a dot-product of two vectors. and DRAM. (a1, a2, a3) (b1, b2, b3) = a1xb1 + a2xb2 + a3xb3 • This memory acts as a low-latency high-bandwidth storage. • If a piece of data is repeatedly used, the effective latency of this – A dot-product computation performs one multiply-add on a single pair of memory system can be reduced by the cache. vector elements, i.e., each floating point operation requires one data fetch. • The fraction of data references satisfied by the cache is called the – It follows that the peak speed of this computation is limited to one floating point operation every 100 ns, ie, cache hit ratio of the computation on the system. (1 FLOP)/(100 ns) = 1/(100 x 10-9 sec) FLOPS = 107 FLOPS • Cache hit ratio achieved by a code on a memory system often = 10 x 106 FLOPS = 10 MFLOPS. determines its performance. Eg: For 100 attempts to access to data in cache, if 30 attempts are successful, what is the cache hit ratio? – The speed of 10 MFLOPS is a very small fraction of the peak processor What is the cache miss ratio? What is the impact if the cache miss rating (4 GFLOPS)! ratio is greater than the cache hit ratio? 19 20 Impact of Memory Bandwidth Impact of Memory Bandwidth • Memory bandwidth is determined by the bandwidth (no. • Increased bandwidth results can improve computation rates. of bytes per second) of the memory bus as well as the • The data layouts were assumed to be such that consecutive memory units. data words in memory were used by successive instructions • Memory bandwidth can be improved by increasing the (spatial locality of reference).