Distributed Computing Lecture 4: Distributed Systems Taxonomy Distributed Computing Taxonomy • There are different ways in which we can connect CPUs together. • According to instruction and data stream, the most widely used classification Taxonomy - scheme (taxonomy) is that created by DC Flynn in 1972. MI MD 2 Distributed Computing Taxonomy, contd. • It classifies machines by the number of instruction streams and the number of data streams. Taxonomy • Multiple instruction streams means that - different instructions can be executed DC concurrently. 3 Flynn’s taxonomy • Flynn’s (1966) taxonomy was developed as a means to classify parallel computer architectures • Computer system can be fit into one of the following four forms: Single Data Taxonomy - DC Single Multiple Instruction Instruction Multiple Data 4 Single Instruction Single Data (SISD) • This is (obviously) the classic von Neumann Computer: serial (not parallel) computer, e.g.: • Old style single core PC CPUs, e.g. i486 • Single instruction Taxonomy • - One instruction stream acted on by DC the CPU during any one clock cycle • Single data • Only one input data stream for any one clock cycle • Deterministic execution 5 Single Instruction Multiple Data (SIMD) • A form of parallel computer • Early supercomputers used this model first • Nowadays it has become common – e.g., used in modern computers on GPUs • Single instruction Taxonomy - • All processing units execute DC the same instruction for any given clock cycle • Multiple data • Each processing unit (PU) can operate on a different data 6 element Single Instruction Multiple Data (SIMD), contd. • Runs in lockstep (i.e., all elements synchronized) • Works well for algorithms with a lot of regularity; e.g. graphics processing. Taxonomy • Two main types: - • Processor arrays DC • Vector pipelines • Still highly deterministic (know the same operation is applied to specific set of data – but 7 more data to keep track of per instruction) Single Instruction Multiple Data (SIMD) Examples • Vector pipelines Cray X-MP • IBM 9000, Cray X-MP, Fujitsu vector processor, NEC SX-2, Hitachi S820, ETA10 MasPar MP-1 Taxonomy • Processor arrays - DC • Thinking Machine CM2, MasPar MP-1 & MP-2, ILLIAC IV • Graphics processor units usually use SIMD 8 Multiple Instruction Single Data (MISD) • Single data stream fed into multiple processing units • Each processing unit works on data independently via independent instruction Taxonomy - streams DC • Few actual examples of this class of parallel computer have ever existed 9 Multiple Instruction Single Data (MISD) Example • Possible uses? Somewhat intellectual? • Maybe redundant! • Possible example application: • Different set of signal processing operations working the same signal stream Taxonomy - DC Simultaneously the min and max input, and do a sum of inputs. A = input x = +MAXINT x = -MAXINT x = 0 If A<x then x = A If A>x then x = A x = x + A … … … 10 CPU 1 CPU 2 CPU 3 Multiple Instruction Multiple Data (MIMD) • The most common type of parallel computer (most late model computers, e.g. Intel Core Duo, in this category) Taxonomy • Multiple Instruction - DC • Each processor can be executing a different instruction stream • Multiple Data • Every processor may be 11 working with a different data stream Multiple Instruction Multiple Data (MIMD), contd. • Execution can be asynchronous or synchronous; non-deterministic or deterministic Taxonomy - DC 12 Multiple Instruction Multiple Data (MIMD), contd. • Examples • Many of the current supercomputers • Networked parallel computer clusters • SMP computers AMD Opteron • multi-core PCs Taxonomy - • MIMD architectures could include DC all the other models. e.g., • SISD – just one CPU active, others running NOP • SIMD – all CPUs load the same instruction but apply to different data • 13 MISD – all CPUs load different instructions IBM BlueGene but apply it to the same data Distributed Computing Taxonomy, Contd. • Computers can perform various functions and each unit in a distributed system may be responsible for only a set number of Taxonomy functions in an organization. - DC We consider the concept of service models as a taxonomy of system configurations. 14 Distributed System Models • Distributed programming falls into one of several basic models: • Client-Server • Peer-to-Peer Taxonomy - • Services provided by multiple servers DC • Proxy servers and caches • Mobile code and mobile agents • Thin clients and mobile devices 15 Client-server model •Clients Invoke Individual Servers Taxonomy - DC • The easiest form of distributed systems • Resources are centralized on servers • Large number of clients access them through 16 request-reply interactions Peer-to-peer Systems • A peer-to-peer system is a collection of peer nodes . • Each peer is both a server and a client (“servant”). • Provides resources to other peers • Consumes resources from other peers. • Characteristics: Taxonomy • Put together resources at the - edge of the Internet DC • Share resources by direct exchange between nodes. • Perform critical functions in a decentralized manner. 17 Client-Server vs. Peer-to-Peer Computing Taxonomy - DC 18 A Service by Multiple Servers • Partition the set of objects and distribute them between themselves Taxonomy - • maintain DC replicated copies • Web -SUN Network Information 19 Service Web Proxy Server •A cache is a storage facility of recently used data objects that is closer to the objects themselves. •Web proxy servers provide a shared cache of web resources for the client machines at a site or across Taxonomy - several sites. DC • Increase availability • Increase performance • Reduce the load on the WAN and web server 20 Mobile Code - Web Applets • The service is provided using a procedure. • executed by a process in the server node. Taxonomy a) Client request results in the - • downloaded to the client and downloading of applet code DC executed locally • push service: the initiator is the server 21 b) Client interacts with the applet Thin Clients and Compute Servers Compute server Network computer or PC Thin network Application Client Process Taxonomy - DC • A thin client is a lightweight computer that is purpose-built for remoting into a server (typically cloud or desktop virtualization environments). • It depends heavily on another computer (its 22 server) to fulfill its computational roles. Mobile Agents • A mobile agent is a running program that travels from one computer to another in a network carrying out a task on someone’s behalf. Taxonomy - • It has the following features: DC • Autonomy • Cooperativeness • Mobility • Agents can continue to process tasks by migration with their 23 state. Mobile Agents, contd. • A mobile agent may make many invocations to local resources at each site it visits (e.g., access to individual database entries). • Might be used to: • Taxonomy install software on the computers of an organization - • compare the prices of products of vendors DC • Negative aspects • They are a potential security threat to the resources • they can be themselves vulnerable (they may not be able to complete their tasks if they are refused 24 access to the information needed) Client-Server System vs. Mobile Agent System Taxonomy - DC 25 .