Flynn’s Taxonomy of Parallel Architectures Stefano Markidis, Erwin Laure, Niclas Jansson, Sergio Rivas-Gomez and Steven Wei Der Chien 1 Sequential Architecture • The von Neumann architecture was conceived in the mid-1940s • Arithmetic logic unit (ALU) is the heart of the computer, performing the actual computation. • Registers can be read and written to at the speed of the surrounding logic. A bank of several registers allow for simultaneous reads and writes • The processor accesses the main memory of the computer system to store and use the values of the program variables making up the state of the calculation. • The operation of the processor is managed by the controller, which creates a sequence of signals to the hardware. • Originally this was done as a series of phases: fetch instructions, execute operation, and write back to register (or memory). • This would be repeated for each succeeding operation of the instruction stream. 2 3/20/17 Flynn's Taxonomy • In the 1966, Michael Flynn proposed a Flynns taxonomy (1966) taxonomy that simplified categorization of distinct classes of parallel architecture and n {Single, Multiple} {Instructions, Data} control methods based on the relationships of data and instruction (control) • Comprising four characters, it divides the SISD SIMD world of computing structures into four Single Instruction, Single Data Single Instruction, Multiple Data classes in a 2D space. • One dimension concerns the data stream, D, • whether there is one such stream, S, or multiple MISD MIMD data streams, M. Multiple Instruction, Single Data Multiple Instruction, Multiple Data • The other dimension relates to the control or instruction stream, I • whether there is one instruction stream, S, or 3 multiple instructions streams, M. 3 SISD UNIVAC1 IBM360 Dell Laptop CDC7600 PDP1 SISD—single instruction stream, single data 4 stream 2 This represents the conventional sequential (serial) processor structure where a single thread of control, the instruction stream, guides the sequence of operations performed on a single set of data, one operand at a time 4 SIMD—single instruction stream, multiple data stream • The first form of parallelism conveyed within Flynn’s taxonomy is simultaneous operation on multiple datasets, controlled by the same set of instructions. • Thus each operation at any one time is the same performed on different data arguments. • Example of SIMD? 5 MIMD—multiple instruction stream, multiple data stream • Like SIMD, there are many sets of data but in this case each dataset has its own instruction stream associated with it. • At any one time there are many operations being performed, but they need not be the same and in fact are almost always different. • As will be seen, this is the most widely used form of parallel architecture, but the category has many different subclasses. • Example? 6 MISD—multiple instruction stream, single data stream Fourth of Flynn's categories is debated: • One possible interpretation is a coarse- grained pipeline where each pipe stage accepts data from the previous stage, performs a set of operations on these data stream elements, and then passes on the results to the next stage. • Another interpretation is a shared-memory multiprocessor where, as the name suggests, multiple processors each with its own instruction stream work on the same (therefore shared) data on which all the other processors operate. 7 SPMD • Not strictly part of Flynn's taxonomy but related to and inspired by it. We will use it later • SPMD stands for “single program, multiple data stream” and reflects a practical variation of the SIMD model. • Instead of issuing and broadcasting one instruction at a time to all the simple processing units of a SIMD-like machine, SPMD sends a function call of a coarse-grained procedure that is to be performed by all the processing units of the parallel machine. • The invocation of heavyweight tasks rather than lightweight instructions: • amortizes the overheads and latency times involved in system control 8 Conclusions • Flynn’s taxonomy categorizes different parallel architectures in terms of instruction and data streams that can be single or multiple • Still in use today to categorize parallel architectures. • Next week lectures you will use them 9.