Distributed System: • Forms Parallelism

Home , Central processing unit, MIMD, MISD, Parallel computing, Pipeline (computing), Shared memory

Distributed system: • Forms parallelism.

• Classification architecture : A. Flynn Taxonomy:

1. SISD. 2. SIMD. 3. MISD. 4. MIMD.

B. According connection memory - processor :

1. Share memory: -According access” UMA, NUMA, COMA”. -According connection: buses, crossing switch, multistage network.

2. Message system: -according network topology.

3. Hybrid system (combination access).

4. Distributed share memory.

Form Parallelism 1. Processors:

-sequence: execute always only one instruction. -”look ahead”: looking also to next instruction: A technique for speeding up the process of fetching and decoding instructions in a computer program, A central processing unit wherein instruction fetch and execution is performed by a mechanism featuring an instruction look ahead mechanism whereby fetching and processing of the next software instruction is commenced as a last step of the currently executing software instruction, and the currently executing software instruction is terminated by the first portion of the next software instruction.

2. Overlapping phase read and decode with executing instruction.

3. Function parallelism: 1) Multi function units: Multifunctional unit (MFU) of the processor. This reconfigurable functional unit can hold several instructions or sequences of instructions that can be provided by a special compiler, and loaded by the processor when needed.

2) Chining-”pipeline”: Pipelining imparts an implicit execution parallelism in the different cycles of processing an instruction. Suppose execution of an instruction consists of the following stages: 1. Fetch – Get the instruction from memory. 2. Decode – Determine what the instruction is. 3. Execute – Perform the instruction decode. 4. Write – Store the results to memory.

4. Chining (Next parts function units). 1) Implicit processing vector operand. 2) Explicit vector operand.

5. Register –register. - Single Instruction Multiple Data “SIMD”: ex: processor array “EPIC” ( Explicitly Parallel Instruction Computing ), Very Long Instruction Word ”VLIW”:

Very long instruction word (VLIW) describes a computer processing architecture in which a language compiler or pre-processor breaks program instruction down into basic operations that can be performed by the processor in parallel (that is, at the same time). These operations are put into a very long instruction word which the processor can then take apart without further analysis, handing each operation to an appropriate functional unit.

“EPIC” ( Explicitly Parallel Instruction Computing ): Its architecture should enable the CISC processors to take a big enough step to overtake the RISC processors. By using a technique called VLIW, still experimental at the time, and creating the EPIC model, explicitly parallel. That is, multiple instructions can easily be executed at once, assuming that the hardware supports it.

6. Memory – memory. - Multiple Instructions Multiple Data “MIMD”: Ex: Multiprocessor, Multicomputer.

Flynn Taxonomy  Michael J. Flynn (1966).  How much instruction is in given moment running and over how much data elements?

Instruction Stream

Single Multiple

Single SISD MISD Data

Stream Multiple SIMD MIMD

FLYNN’S TAXONOMY OF COMPUTER ARCHITECTURE

The most popular taxonomy of computer architecture was defined by Flynn in 1966.

Flynn’s classification scheme is based on the notion of a stream of information. Two types of information flow into a processor: instructions and data. The instruction stream is defined as the sequence of instructions performed by the processing unit. The data stream is defined as the data traffic exchanged between the memory and the processing unit. According to Flynn’s classification, either of the 5

instruction or data streams can be single or multiple. Computer architecture can be classified into the following four distinct categories:

. single-instruction single-data streams (SISD); . single-instruction multiple-data streams (SIMD); . multiple-instruction single-data streams (MISD); and . multiple-instruction multiple-data streams (MIMD).

Conventional single-processor von Neumann computers are classified as SISD systems. Parallel computers are either SIMD or MIMD. When there is only one control unit and all processors execute the same instruction in a synchronized fashion, the parallel machine is classified as SIMD. In a MIMD machine, each processor has its own control unit and can execute different instructions on different data. In the MISD category, the same stream of data flows through a linear array of processors executing different instruction streams. In practice, there is no viable MISD machine; however, some authors have considered pipelined machines (and perhaps systolic-array computers) as examples for MISD. An extension of Flynn’s taxonomy was introduced by D. J. Kuck in 1978. In his classification, Kuck extended the instruction stream further to single (scalar and array) and multiple (scalar and array) streams. The data stream in Kuck’s classification is called the execution stream and is also extended to include single (scalar and array) and multiple (scalar and array) streams. The combination of these streams results in a total of 16 categories of architectures.

1) SISD ARCHITECTURE:

1. One control unit, one flow control. 2. Always read/write only one value data from memory. 3. Classical von Neumann architecture.

Today architecture of SISD:

2) SIMD ARCHITECTURE:

1. Single control unit (single instruction cache) control multi equal (synchronize) executed units. 2. Equal operation will execute with various data. 3. Vector processors: operate over vector execute in single instruction cycle. 4. (Increment over component scalar multiplies). 5. Work with special HW “VLIW, EPIC”. 6. Characteristic : 1) Count execution units (EU). 2) Count scalar instruction. 3) Semaphore. 4) Count bridge scheme. 5) Count vector instruction.

Note: VLIW (Very Long Instruction Word), EPIC ( Explicitly Parallel Instruction Computing ). Scalar (in computing), an atomic quantity that can hold only one value at a time.

The SIMD model of parallel computing consists of two parts: a front-end computer of the usual von Neumann style and a processor array as shown in Figure, The processor array is a set of identical synchronized processing elements capable of simultaneously performing the same operation on different data. Each processor in the array has a small amount of local memory where the distributed data resides while it is being processed in parallel. The processor array is connected to the memory bus of the front end so that the front end can randomly access the local processor memories as if it were another memory. Thus, the front end can issue special commands that cause parts of the memory to be operated on simultaneously or cause data to move around in the memory. A program can be developed and executed on the front end using a traditional serial programming language. The application program is executed by the front end in the usual serial way, but issues commands to the processor array to carry out SIMD operations in parallel. The similarity between serial and data parallel programming is one of the strong points of data parallelism. Synchronization is made irrelevant by the lock–step

synchronization of the processors. Processors either do nothing or exactly the same operations at the same time. In SIMD architecture, parallelism is exploited by applying simultaneous operations across large sets of data. This paradigm is most useful for solving problems that have lots of data that need to be updated on a wholesale basis. It is especially powerful in many regular numerical calculations.

There are two main configurations that have been used in SIMD machines. In the first scheme, each processor has its own local memory. Processors can communicate with each other through the interconnection network. If the interconnection network does not provide direct connection between a given pair of processors, then this pair can exchange data via an intermediate processor.

3) MISD ARCHITECTURE:

1. Multiple Various operation over equal data : -Data pass at once to all execution units (different parts single data flow). -Data pass to first execution unit and his output is input to next …”Micro- pipeline”.

2. Ex: Carnegie-Mellon: Multi mini processor, ”C.mmp” (16p,SIMD,MISD,MIMD)

3. “Data flow” architecture, systolic array.

4. Only for integrity, Practically unused.

4) MIMD ARCHITECTURE:

1. Multi flow “control and data” _ each processor can execute different program on itself data. 2. Can work asynchronous (on different from MISD). 3. Processors synchronous for access to share (resources) data. 4. Computer with multi processor (SM-MIMD). 5. Multi computer connected in network (DM-MIMD).

Multiple-instruction multiple-data streams (MIMD) parallel architectures are made Of multiple processors and multiple memory modules connected together via some interconnection network. They fall into two broad categories: shared memory or Message passing. the general architecture of these two categories. Processors exchange information through their central shared memory in shared memory systems, and exchange information through their interconnection network in message passing systems. A shared memory system typically accomplishes interprocessor coordination through a global memory shared by all processors. These are typically server systems that communicate through a bus and cache memory controller. The bus/

cache architecture alleviates the need for expensive multi ported memories and interface circuitry as well as the need to adopt a message-passing paradigm when developing application software. Because access to shared memory is balanced, these systems are also called SMP (symmetric multiprocessor) systems. Each processor has equal opportunity to read/write to memory, including equal access speed. A message passing system (also referred to as distributed memory) typically combines the local memory and processor at each node of the interconnection network. There is no global memory, so it is necessary to move data from one local memory to another by means of message passing. This is typically done by a Send/Receive pair of commands, which must be written into the application software by a programmer.

How can seem MIMD/SIMD?

 Multiprocessors: -Narrow band- communicates through share memory. -Single (share) addresses space. - Assigned data getting by searching program on Share memory locations. -Semaphore, mutex, barrier.  Multicomputer: -Free band-communication message system. -Each process have self memory and address space. -Located assigned data by effected step (instruction) in program.