Flynn’s Taxonomy of Parallel Architectures Stefano Markidis, Erwin Laure, Niclas Jansson, Sergio Rivas-Gomez and Steven Wei Der Chien

1

Sequential Architecture • The was conceived in the mid-1940s • (ALU) is the heart of the computer, performing the actual computation. • Registers can be read and written to at the speed of the surrounding logic. A bank of several registers allow for simultaneous reads and writes • The accesses the main memory of the computer system to store and use the values of the program variables making up the state of the calculation. • The operation of the processor is managed by the controller, which creates a sequence of signals to the hardware. • Originally this was done as a series of phases: fetch instructions, execute operation, and write back to register (or memory). • This would be repeated for each succeeding operation of the instruction stream. 2 3/20/17 Flynn's Taxonomy • In the 1966, Michael Flynn proposed a Flynns taxonomy (1966) taxonomy that simplified categorization of distinct classes of parallel architecture and n {Single, Multiple} {Instructions, Data} control methods based on the relationships of data and instruction (control) • Comprising four characters, it divides the SISD SIMD world of structures into four Single Instruction, Single Data Single Instruction, Multiple Data classes in a 2D space. • One dimension concerns the data stream, D, • whether there is one such stream, S, or multiple MISD MIMD data streams, M. Multiple Instruction, Single Data Multiple Instruction, Multiple Data • The other dimension relates to the control or instruction stream, I • whether there is one instruction stream, S, or 3 multiple instructions streams, M.

3 SISD

UNIVAC1 IBM360

Dell Laptop

CDC7600 PDP1

SISD—single instruction stream, single data 4 stream 2 This represents the conventional sequential (serial) processor structure where a single of control, the instruction stream, guides the sequence of operations performed on a single set of data, one operand at a time

4 SIMD—single instruction stream, multiple data stream

• The first form of parallelism conveyed within Flynn’s taxonomy is simultaneous operation on multiple datasets, controlled by the same set of instructions. • Thus each operation at any one time is the same performed on different data arguments. • Example of SIMD?

5

MIMD—multiple instruction stream, multiple data stream • Like SIMD, there are many sets of data but in this case each dataset has its own instruction stream associated with it. • At any one time there are many operations being performed, but they need not be the same and in fact are almost always different. • As will be seen, this is the most widely used form of parallel architecture, but the category has many different subclasses. • Example?

6 MISD—multiple instruction stream, single data stream Fourth of Flynn's categories is debated: • One possible interpretation is a coarse- grained where each pipe stage accepts data from the previous stage, performs a set of operations on these data stream elements, and then passes on the results to the next stage. • Another interpretation is a shared-memory multiprocessor where, as the name suggests, multiple processors each with its own instruction stream work on the same (therefore shared) data on which all the other processors operate. 7

SPMD

• Not strictly part of Flynn's taxonomy but related to and inspired by it. We will use it later • SPMD stands for “single program, multiple data stream” and reflects a practical variation of the SIMD model. • Instead of issuing and broadcasting one instruction at a time to all the simple processing units of a SIMD-like machine, SPMD sends a function call of a coarse-grained procedure that is to be performed by all the processing units of the parallel machine. • The invocation of heavyweight tasks rather than lightweight instructions: • amortizes the overheads and latency times involved in system control

8 Conclusions

• Flynn’s taxonomy categorizes different parallel architectures in terms of instruction and data streams that can be single or multiple • Still in use today to categorize parallel architectures. • Next week lectures you will use them

9