UNIVERZA V LJUBLJANI FAKULTETA ZA RAČUNALNIŠTVO IN INFORMATIKO Janez Perme Simulacija izvajanja poslov na upravljanem porazdeljenem sistemu MAGISTRSKA NALOGA prof. dr. Nikolaj Zimic MENTOR dr. Andrej Brodnik SOMENTOR Ljubljana, 2011 ZAHVALA Zahvaljujem se mentorju, prof. dr. Nikolaju Zimicu, in somentorju, doc. dr. Andreju Brodniku, za strokovno vodenje, nasvete in pomoč pri nastajanju magistrske naloge. Posebna zahvala gre tudi Zoranu Nikolovskemu in dr. Radovanu Sernecu za konstruktivne diskusije. Janez Perme, Ljubljana, junij 2011. KAZALO 1 Uvod .................................................................................................................................... 5 1.1 Predstavitev problematike ............................................................................................. 5 1.2 Metodologija ................................................................................................................. 6 1.3 Prispevki magistrske naloge .......................................................................................... 7 2 Vzporedni in porazdeljeni sistemi .................................................................................... 9 2.1 Vzporedni sistemi .......................................................................................................... 9 2.1.1 Sistemi SISD ........................................................................................................ 12 2.1.2 Sistemi SIMD ....................................................................................................... 12 2.1.3 Sistemi MISD ....................................................................................................... 15 2.1.4 Sistemi MIMD ...................................................................................................... 15 2.2 Porazdeljeni sistemi .................................................................................................... 19 2.2.1 Sistemi GRID ....................................................................................................... 20 2.2.2 Sistemi v oblaku ................................................................................................... 22 2.2.3 Prostovoljni sistemi .............................................................................................. 23 2.2.4 Sistemi "vsak z vsakim" (P2P) ............................................................................. 24 2.3 Primerjava vzporednih in porazdeljenih sistemov ...................................................... 26 3 Porazdeljena sistema BOINC in Condor ....................................................................... 29 3.1 Sistem BOINC ............................................................................................................ 29 3.1.1 Arhitektura sistema BOINC ................................................................................. 30 3.1.2 Življenjski cikel opravila v sistemu BOINC ........................................................ 32 3.1.3 Večjedrne in večprocesorske arhitekture v sistemu BOINC ................................ 33 3.1.4 Varnost v sistemu BOINC .................................................................................... 34 3.2 Sistem Condor ............................................................................................................. 34 3.2.1 Arhitektura sistema Condor .................................................................................. 36 3.2.2 Življenjski cikel opravila v sistemu Condor ......................................................... 38 3.2.3 Stanja vozlišč in prehodi med stanji v sistemu Condor ........................................ 40 3.2.4 Delovna okolja v sistemu Condor ........................................................................ 41 3.2.5 Upravljanje z opravili s pomočjo orodij DAGMan in Master-Worker ................ 43 3.2.5.1 Orodje DAGMan ........................................................................................... 43 3.2.5.2 Orodje Master-Worker ................................................................................... 44 3.2.6 Dodeljevanje izvajalcev opravil v sistemu Condor .............................................. 45 3.3 Primerjava in abstrakcija sistemov BOINC in Condor ............................................... 46 4 Ovrednotenje zmogljivosti prostovoljnega sistema ...................................................... 49 4.1 Postopek simulacije ..................................................................................................... 49 4.2 Poenostavitev realnega prostovoljnega sistema .......................................................... 50 4.3 Simulacijski model ...................................................................................................... 54 4.4 Implementacija simulacijskega modela z orodjem ns-2 ............................................. 58 4.4.1 Življenjski cikel paketa UDP ................................................................................ 60 4.4.2 Opis parametrov implementacije ......................................................................... 60 4.5 Preverba in vrednotenje simulacijskega modela ......................................................... 61 4.5.1 Preverba implementacije simulacijskega modela ................................................ 62 4.5.2 Vrednotenje simulacijskega modela .................................................................... 63 4.6 Eksperimentalno delo in analiza rezultatov simulacije .............................................. 66 4.6.1 Določitev časa izvajanja simulacije ..................................................................... 68 4.6.2 Eksperimentalno delo in rezultati ........................................................................ 71 4.6.2.1 Eksperiment I ................................................................................................ 71 4.6.2.2 Eksperiment II ............................................................................................... 78 4.6.2.3 Eksperiment III .............................................................................................. 83 4.6.2.4 Eksperiment IV.............................................................................................. 89 4.6.2.5 Eksperiment V ............................................................................................... 93 4.6.2.6 Eksperiment VI.............................................................................................. 98 5 Zaključek ........................................................................................................................ 107 SEZNAM UPORABLJENIH KRATIC ALE Aritmetično logična enota API Application program interface ARC Advanced resource connector BOINC Berkeley open interface for network computing BSD Berkeley software design CERN Conseil européen pour la recherche nucléaire CGI Common gateway interface CPE Centralno procesna enota CPI Clocks per instruction DAG Directed acyclic graph DAGMan Directed acyclic graph manager DoS Denial of service DSL Digital subsciber line EGEE Enabling grids for e-science FC Fibre channel FIFO First in, first out FLOPS Floating point operations per second GPC Global positioning system GPU Graphics processing unit HPC High performance computing HTC High throughput computing HTTP Hyper text transfer protocol ILP Instruction level parallelism IP Internet protocol IPS Instructions per second IPTV Internet protocol television ISA Instruction set architecture JVM Java virtual machine LAN Local area network LB Load balancing MIMD Multiple instruction stream, multiple data stream MISD Multiple instruction stream, single data stream MMU Memory management unit MMX Multimedia extension ali Multiple math extension ali Matrix math extension MPI Message passing interface MPP Massively parallel processing NAM Network animator NAT Network address translation NFS Network file system NUMA Non-uniform memory access P2P Peer-to-peer PaaS Platform as a service PVM Parallel virtual machine RMComm Resource management and communication RPC Remote procedure call QoS Quality of service SaaS Software as a service SIMD Single instruction stream, multiple data stream SISD Single instruction stream, single data stream SLA Service level agreement SMP Symmetrical multiprocessing SSE Streaming SIMD extensions SSL Space science laboratory UMA Uniform memory access UNICORE Uniform interface to computing resources URL Uniform resource locator VLIW Very long instruction word VoIP Voice over IP WAN Wide area network XML Extensible markup language 1 Univerza v Ljubljani Fakulteta za računalništvo in informatiko Janez Perme Simulacija izvajanja poslov na upravljanem porazdeljenem sistemu POVZETEK Reševanje kompleksnih računskih problemov s pomočjo računalnikov je dandanes praktično povsod navzoče. Vse več je problemov, ki so časovno in prostorsko zahtevni. Rešujejo se s pomočjo namenskih vzporednih sistemov. Večino sodobnih sistemov te vrste umeščamo skoraj izključno v Flynnov razred MIMD. Primeri so predvsem tesno povezani sistemi SMP in ohlapno povezani sistemi MPP ter grozdi. Posebno na področju znanstvenega računanja pa so se, kot cenovno ugoden nadomestek vzporednih sistemov, uveljavili tudi porazdeljeni sistemi. Primerni so predvsem za reševanje problemov, pri katerih je dekompozicija na podprobleme trivialna. Pomembna razlika med obema pristopoma je predvsem v namembnosti. Vzporedni sistemi so namenjeni predvsem pohitritvi reševanja problemov (hitrejši odzivni