Algoritmos Evolutivos Multiobjetivo Para Despacho De Tareas En Redes Heterogéneas

Total Page:16

File Type:pdf, Size:1020Kb

Algoritmos Evolutivos Multiobjetivo Para Despacho De Tareas En Redes Heterogéneas Instituto de Computación – Facultad de Ingeniería Universidad de la República Montevideo, Uruguay Proyecto de Grado Algoritmos evolutivos multiobjetivo para despacho de tareas en redes heterogéneas Darío De León 2008 Supervisor: Sergio Nesmachnow Contenido Capítulo 1. Introducción....................................................................................7 Capítulo 2. Computación heterogénea...............................................................9 2.1 Conceptos de computación heterogénea..................................................9 2.1.1 Ventajas de los sistemas heterogéneos.............................................11 2.1.2 Tipos de sistemas heterogéneos ........................................................13 2.1.3 Perfiles de tareas ...............................................................................13 2.1.4 Técnicas de predicción......................................................................15 2.1.5 Mapeo................................................................................................16 2.1.6 Entornos de programación ................................................................18 2.1.7 Evaluación de desempeño .................................................................19 2.1.8 Algunas medidas de performance .....................................................20 2.1.9 Algunas heurísticas propuestas .........................................................23 2.2 Aplicaciones de la computación heterogénea........................................28 2.2.1 Berkeley Open Infrastructure for Network Computing (BOINC) ....28 2.2.2 World Community Grid (WCG) .......................................................31 2.3 Modelo ETC ..........................................................................................32 2.4 Trabajos relacionados............................................................................33 2.4.1 Estimating the Execution Time Distribution for a Task Graph in a Heterogeneous Computing System.................................................................33 2.4.2 A Comparison of Eleven Static Heuristics for Mapping a Class of Independent Tasks onto Heterogeneous Distributed Computing Systems .....34 2.4.3 The Relative Performance of Various Algorithms is Independent of Sizable Variances in Run-time Predictions.....................................................36 2.4.4 A Taxonomy for describing Matching and Scheduling Heuristics for Mixed-Machine Heterogeneous Computing Systems.....................................36 2.4.5 Efficient Batch Job Scheduling in Grids using Cellular Memetic Algorithms ......................................................................................................37 2.5 Conclusiones..........................................................................................38 Capítulo 3. Computación evolutiva.................................................................39 3.1 Técnicas de computación evolutiva.......................................................39 3.2 Algoritmos genéticos.............................................................................41 3.2.1 Representación de soluciones............................................................42 3.2.2 Función de fitness .............................................................................42 3.2.3 Operadores evolutivos.......................................................................43 3.3 Modelos de evolución............................................................................44 3.3.1 Modelo de estado estacionario ..........................................................44 3.3.2 Otros modelos de evolución..............................................................45 3.4 Algoritmos evolutivos multiobjetivos ...................................................46 3.4.1 Problemas de optimización multiobjetivo.........................................46 3.4.2 AE para optimización multiobjetivo (MOEA)..................................47 • Non Pareto based MOEAs ................................................................48 • Pareto based MOEAs ........................................................................48 3.4.3 MOEAs de primera generación.........................................................49 • VEGA - Vector Evaluated Genetic Algorithms................................49 • NSGA - Non-dominated Sorting Genetic Algorithm........................49 • NPGA - Niched Pareto Genetic Algorithm.......................................50 • MOGA - MultiObjective Genetic Algorithm....................................51 3.4.4 MOEAs de segunda generación ........................................................52 • SPEA - Strength Pareto Evolutionary Algorithm..............................52 • SPEA-2..............................................................................................53 • PAES - Pareto Archived Evolution Strategy.....................................54 3 4 CONTENIDO • NSGA-II............................................................................................55 • NPGA-2.............................................................................................56 • MICRO GA.......................................................................................57 3.4.5 Métricas de evaluación de MOEAs...................................................57 • Número de Puntos no Dominados.....................................................58 • Tasa de Error.....................................................................................58 • Distancia Generacional .....................................................................58 • Spacing..............................................................................................58 • Spread................................................................................................59 3.5 Algoritmos evolutivos paralelos............................................................59 3.5.1 Modelo maestro-esclavo ...................................................................60 3.5.2 Modelo de subpoblaciones distribuidas ............................................61 3.5.3 Modelo celular ..................................................................................62 Capítulo 4. MOEAs para planificación en entornos HC .................................65 4.1 Descripción del problema......................................................................65 4.2 Trabajo previo. Despacho de tareas en redes heterogéneas: enfoque multiobjetivo.......................................................................................................67 4.3 Diseño del algoritmo .............................................................................68 4.3.1 Codificación ......................................................................................69 4.3.2 Inicialización de la población............................................................69 4.3.3 Selección ...........................................................................................70 4.3.4 Cruzamiento ......................................................................................70 4.3.5 Migración ..........................................................................................71 4.3.6 Mutación ...........................................................................................71 4.4 Frameworks utilizados...........................................................................72 4.4.1 MOE Framework ..............................................................................73 4.4.2 JMetal................................................................................................76 4.4.3 Sparrow .............................................................................................78 4.5 Detalles de la implementación...............................................................80 4.5.1 NSGAII .............................................................................................80 4.5.2 SPEA-2..............................................................................................81 Capítulo 5. Evaluación experimental...............................................................85 5.1 Casos de prueba.....................................................................................85 5.2 Plataforma computacional .....................................................................85 5.3 Parámetros de Ejecución .......................................................................86 5.4 Análisis de resultados experimentales...................................................86 5.5 Evolución de los frentes de Pareto ........................................................88 5.6 Frentes de Pareto obtenidos...................................................................92 5.7 Análisis de métricas de calidad .............................................................95 5.8 Análisis de la eficiencia computacional ................................................95 Capítulo 6. Conclusiones y trabajo futuro .......................................................99 Lista de Algoritmos Algoritmo 1 – Algoritmo Evolutivo Genérico........................................................40 Algoritmo 2 – Esquema de un Algoritmo Genético................................................41 Algoritmo 3 – Esquema de un MOEA....................................................................47 Algoritmo 4 – Esquema del NSGA.........................................................................50 Algoritmo 5 – Esquema de NPGA..........................................................................51
Recommended publications
  • Volunteer Computing Different Grids for Different Needs
    From the Web to the Grid How did the Grid start? • Name “Grid” chosen by analogy with electric power grid (Foster and Kesselman 1997) • Vision: plug-in computer for processing power just like plugging in toaster for electricity. • Concept has been around for decades (distributed computing, metacomputing) • Key difference with the Grid is to realise the vision on a global scale. From the Web to the Grid – 2007 HST 2011: Grids and Volunteer Computing Different Grids for different needs There is as yet no unified Grid, like there is a single web. Rather there are many Grids for many applications: • Enterprise Grids link together PCs within one company. • Volunteer computing links together public computers. • Scientific Grids link together major computing centres. • Latest trend federates national Grids into global Grid infrastructure. • High Energy Physics is a driving force for this. HST 2011: Grids and Volunteer Computing The LHC data challenge 1 Megabyte (1MB) • 40 million bunch collisions per second A digital photo 1 Gigabyte (1GB) = 1000MB • After filtering, ~100 collisions of 5GB = A DVD movie interest per second per detector 1 Terabyte (1TB) = 1000GB World annual book production • > 1 Megabyte of data per collision 1 Petabyte (1PB) recording rate > 1 Gigabyte/sec = 1000TB Annual production of one LHC experiment 10 • 10 collisions recorded each year 1 Exabyte (1EB) stored data ~15 Petabytes/year = 1000 PB 3EB = World annual information production …for more than 10 years HST 2011: Grids and Volunteer Computing Data Storage for the LHC Balloon (30 Km) • LHC data correspond to about 20 million CDs each year! CD stack with 1 year LHC data! (~ 20 Km) Concorde Where will the (15 Km) experiments store all of these data? Mt.
    [Show full text]
  • An Investigation Into Citizen Cyberscience
    When Scientists Meet the Public: An Investigation into Citizen Cyberscience Peter Darch Corpus Christi College University of Oxford A dissertation submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy Michaelmas Term 2011 Peter Darch Corpus Christi College When Scientists Meet the Public: An Investigation into Citizen Cyberscience Abstract Citizen Cyberscience Projects (CCPs) are projects mediated through the Internet, in which teams of scientists recruit members of the public (volunteers) to assist in scientific research, typically through the processing of large quantities of data. This thesis presents qualitative ethnographic case studies of the communities that have formed around two such projects, climateprediction.net and Galaxy Zoo. By considering these social actors in the broader contexts in which they are situated (historical, institutional, social, scientific), I discuss the co-shaping of the interests of these actors, the nature of the relationships amongst these actors, and the infrastructure of the projects and the purposes and nature of the scientific work performed. The thesis focusses on two relationships in particular. The first is that between scientists and volunteers, finding that, although scientists in both projects are concerned with treating volunteers with respect, there are nevertheless considerable differences between the projects. These are related to a number of interconnecting factors, including the particular contexts in which each project is embedded, the nature of the scientific work that volunteers are asked to undertake, the possibilities and challenges for the future development of the projects as perceived by the scientists, and the tools at the disposal of the respective teams of scientists for mediating relationships with volunteers.
    [Show full text]
  • Integrated Service and Desktop Grids for Scientific Computing
    Integrated Service and Desktop Grids for Scientific Computing Robert Lovas Computer and Automation Research Institute, Hungarian Academy of Sciences, Budapest, Hungary [email protected] Ad Emmen AlmereGrid, Almere, The Nederlands [email protected] RI-261561 GRID 2010, DUBNA Why Desktop Grids are important? http://knowledgebase.ehttp://knowledgebase.e--irg.euirg.eu RI-261561 GRID 2010, DUBNA Introduction RI-261561 WP4 Author: Robert Lovas, Ad Emmen version: 1.0 Prelude - what do people at home and SME’s think about grid computing Survey of EDGeS project Questionnaires all across Europe Get an idea of the interest in people and SMEs to donate computing time for science to a Grid Get an idea of the interest in running a Grid inside an SME RI-261561 GRID 2010, DUBNA Survey amongst the General Public and SME’s RI-261561 GRID 2010, DUBNA Opinions about Grid computing RI-261561 GRID 2010, DUBNA Survey - Conclusions Overall: there is interest in Desktop Grid computing in Europe. However, that people are willing to change their current practice and say that they want to participate in Grid efforts does not mean that they are actually going to do that. Need to generate trust in the organisation that manages the Grid. People want to donate computing time for scientific applications, especially medical applications. They do not like to donate computing time to commercial or defense applications. People want feedback on the application they are running. No clear technical barriers perceived by the respondents: so this does not need much attention. Overall the respondents were rather positive about donating computing time for a Grid or about running applications on a Grid.
    [Show full text]
  • Projects on BONIC?
    PRESENTED BY: SAJIN GEORGE PRIYANKA ANGADI Overview of the talk ● GRID and Cluster computing and what are their differences. ● What is BOINC and how it works ? ● Key features of BOINC. ● Power consumption and Energy consumption. ● Projects that are currently using BOINC ● BONIC interface and Client Software. ● How to setup your own BOINC. GRID COMPUTING A distributed computing environment that uses its own resources to handle computational tasks. Grid VS Cluster Loosely coupled Tightly coupled systems (Decentralization) Single system image Diversity and Dynamism Centralized Job Distributed Job management & scheduling Management & scheduling system Volunteer computing. Volunteer computing is an arrangement in which people (volunteers) provide computing resources to projects, which use the resources to do distributed computing and/or storage. ● Volunteers are typically members of the general public who own Internet-connected PCs. Organizations such as schools and businesses may also volunteer the use of their computers. ● Projects are typically academic (university-based) and do scientific research. But there are exceptions; for example, GIMPS and distributed.net (two major projects) are not academic www.bonic.com What is B.O.I.N.C ? is a massive open-source grid computing tool by the university of berkeley. How BOINC works ? How the software works CREDIT The project's server keeps track of how much work your computer has done; this is called credit. To ensure that credit is granted fairly, most BOINC projects work as follows: ● Each task may be sent to two computers. ● When a computer reports a result, it claims a certain amount of credit, based on how much CPU time was used.
    [Show full text]
  • Matemáticas En Gidgrids De E-Cienc Ias
    Aplicaciones Matemáticas en GidGrids de e-Cienc ias “Son las Grid el Nuevo Paradigma de la Computación” Felipe Rolando Menchaca García CIC-IPN GRIDS Las Grid han alcanzado un nivel de madurez tal que se están convirtiendo en la infraestructura de cómputo idónea para la investigación científica y el desarrollo de soluciones de ingeniería de alto nivel Felipe Rolando Menchaca García CIC-IPN GRIDS • La computación distribuida ha sido diseñada para resolver problemas que exceden la capacidad de cualquier supercomputadora, mientras se mantiene la flexibilidad de trabajar en múltiples problemas más ppqequeños. • En particular la computación de ciclos redundantes aprovecha liddlas capacidades no utilizadas de las máquinas para realizar procesamiento de muy altos requerimientos. Felipe Rolando Menchaca García CIC-IPN GRIDS El constante incremento de la capacidad del hdhardware y e ldl decremen tdlto de los prec ios dlde los componentes informáticos, ha permitido que los clusters se hayan convertido en una alternativa muy atractiva en el campo de la computación paralela y distribuida. No obstante,,g la gran demanda tanto de computación como de espacio de almacenamiento requeridos por un diversas aplicaciones que manejan grandes cantidades de datos y deben hacerlo de forma eficiente, exige el uso de nuevas tecnologías, como es el caso de la computación GRID. Felipe Rolando Menchaca García CIC-IPN GRIDS • ¿Qué es la e-Ciencia? • ¿La e-Ciencia requiere del cómputo GRID? • ¿En qué tecnologías se apoyan las GRID? • ¿Qué es el middleware GRID? • ¿Cómo es infraestructura física de la GRID? • ¿Hacia dónde se dirigen las GRID? • ¿Qué son los servicios GRID? • ¿Qué aplicaciones hay en los servicios GRID? Felipe Rolando Menchaca García CIC-IPN Los Sistemas más Poderosos • Folding@Home opera a 8.1 PetaFlops.
    [Show full text]
  • Connecting Citizens to Science
    ConnectingConnecting CitizensCitizens toto ScienceScience AA LearningLearning EE--volutionvolution MiguelMiguel AngelAngel MarquinaMarquina CERN,CERN, SwitzerlandSwitzerland CERN University of Geneva UN Institute for Training and Research “All for science, science for all” 45th EUCEN Conference Connecting Citizens to Science… Charmey - May 31, 2013 Miguel Angel Marquina ScienceScience andand InnovationInnovation ResearchResearch vsvs HEHE structuresstructures ResearchResearch environments,environments, purepure joyjoy ButBut sometimes,sometimes, IvoryIvory TowersTowers MajorMajor societalsocietal investmentinvestment onon formingforming suchsuch ““eliteelite”” ReturnReturn toto Society,Society, eveneven ifif justjust aa token,token, withwith educationeducation andand learninglearning opportunities?opportunities? 45th EUCEN Conference Connecting Citizens to Science… Charmey - May 31, 2013 Miguel Angel Marquina ScienceScience andand InnovationInnovation GlobalGlobal ChallengesChallenges ToughTough timetime forfor Researchers,Researchers, competingcompeting for:for: ResourcesResources –– Funding,Funding, ICTICT BigBig DataData challengeschallenges Non/semiNon/semi--automatableautomatable analysisanalysis ImpactImpact onon Society,Society, reachingreaching thethe publicpublic 45th EUCEN Conference Connecting Citizens to Science… Charmey - May 31, 2013 Miguel Angel Marquina ScienceScience andand InnovationInnovation CitizenCitizen InvolvementInvolvement CitizensCitizens havehave beenbeen passivepassive watcherswatchers soso farfar
    [Show full text]
  • Volunteer Computing and Virtualization
    Volunteer Computing and Virtualization Ben Segal Holger Schulz and: Predrag Buncic, David Garcia, Jakob Blomer, Pere Mato, Carlos Aguado / CERN Artem Harutyunyan / Yerevan Physics Institute Jarno Rantala / Tampere University of Technology David Weir / Imperial College, London Yushu Yao / Lawrence Berkeley Laboratory Rohit Yadav / Bnares Hindu University 2nd Workshop on Multi-Core and Virtualization CERN June 21-22, 2010 1 What is BOINC? • “Berkeley Open Infrastructure for Network Computing” • Software platform for distributed computing using volunteered computer resources • http://boinc.berkeley.edu • Uses a volunteer PC’s unused CPU cycles to analyse scientific data • Client-server architecture • Free and Open-source • Also handles DESKTOP GRIDS Some volunteer computing projects LIFE SCIENCES MATHEMATICS & INTERNET SCIENCE Parabon Computation CRYPTOGRAPHY PERFORMANCE SETI@home (BOINC) Folding@home Great Internet Mersenne Prime Search Gómez Performance ($) evolution@home FightAIDS@home Proth Prime Search Network Peer eOn Übero ECMNET NETI@home climateprediction.net (BOINC) Drug Design Optimization Lab (D2OL) Minimal Equal Sums of Like Powers dCrawl Muon1 The Virtual Laboratory Project MM61 Project DIMES LHC@home (BOINC) Community TSC 3x + 1 Problem Red Library DLV Einstein@Home(BOINC) Predictor@home (BOINC) Distributed Search for Fermat Majestic-12 BBC Climate Change XGrid@Stanford Number Divisors Boitho Experiment (BOINC) Human Proteome Folding (WCG) PCP@Home PeerFactor Leiden Classical (BOINC) CHRONOS (BOINC) Generalized Fermat Prime
    [Show full text]
  • The 10Th BOINC Workshop
    The 10th BOINC Workshop David P. Anderson Space Sciences Lab University of California, Berkeley 29 Sept. 2014 1985 ● Wisconsin UC Berkeley ● Internet as backplane 1987 ● Marionette 1992 ● Industry 1995 ● David Gedye: SETI@home idea 1998 ● SETI@home development – Eric Korpela – Jeff Cobb – Matt Lebofsky 1999 ● SETI@home launch 2000 ● Infrastructure issues ● United Devices 2001 ● United Devices falling-out 2002 ● ClimatePrediction.net: Myles Allen ● BOINC computing power Scientists Volunteers education/outreach 2002 ● Open source software ● Credit ● Replication and validation ● Client job buffer ● Code signing 2002 ● Hiram Clawson, Eric Heien ● NSF proposal – Mari Maeda, Kevin Thompson ● Visit Climateprediction – Carl Christensen, Tolu Aina ● Derrick Kondo ● Vijay Pande 2003 ● UD lawsuit ● Undergrads, PHP code ● Karl Chen, Mr. Python ● Oct: LIGO, Bruce Allen ● Nov: CERN – Francois Grey, Ben Segal ● Nov: WCG kicks tires 2003 job creation MySQL assimilator server scheduler validator transitioner 2004 ● Rom Walton ● Charlie Fenton 2004 ● Anonymous platform ● Separate GUI ● Cross-project ID and credit ● Preemptive scheduling ● Sticky files ● Upload/download hierarchies ● DB as buffer 2004 ● Predictor@home, Michela Taufer – homogeneous redundancy ● SETI@home: Eric Korpela ● BURP: Janus Kristensen ● Climateprediction.net launch ● LHC@home launch 2004 2004 ● Supercomputer 04 talk ● Matt Blumberg, account manager design 2005 ● Einstein@home – Reinhard Prix, Bernd Machenschalk, Oliver Bock ● Primegrid – Rytis Slatkevičius ● Rosetta@home ● IBM World
    [Show full text]
  • Distributed Computing Systems As Project Learning Environment for ”Generation NET ∗
    Distributed computing systems as project learning environment for ”Generation NET ∗ Pavel Rabinovich Ilya Kurochkin Kirill Zavedensky MSRU,FIED,Moscow, Russia IITP RAS, Moscow, Russia Fablab Factory of ideas [email protected] [email protected] and innovations, Moscow, Russia [email protected] Abstract The article is devoted to applying desktop grid systems in educa- tion. The proposed solution for integration of the technologies of dis- tributed computing in school education through children’s practice- oriented projects. The article contains early results of implementation of practice-oriented project learning in the Russian schools. Keywords: distributed computing, desktop grid, BOINC, education, children’s project, CosmOdis, project learning, research in school. 1 Introduction This work tries to find solution for contemporary task of staffingin the distributed computing area and the educational task of training modern students called the NET Generation. Let us take a brief look at both aspects. Creation and development distributed computingsystems (DCS) is a strategic way of intensive tasks pro- cessing in medicine, chemistry, biology, space exploration etc. [Isaev & Kornilov, 2013], [Radchenko, 2012], [Knushov at al., 2014].The development and functioning of DCS requires sufficient number of professionals with high level of proficiency in mathematics, competencies in algorithms, programming, user interfaces, adminis- tration, testing and debugging, project management, etc. [Mishchenko & Gubarev , 2015]. Russian Federation raises this professionals in top engineering universities: MSU, MIPT, ITMO, Bauman MSTU, MEPHI, RSUH, HSE, MISIS, PetrSU, SWSU, ISU, BSU, MAI, MSP etc. However, there is an essential task in DCS development area.This task is attracting and retaining minimal required number of volunteers-participants in virtual com- munities and maintaining interest to ongoing DCS projects [Bobrov at al., 2015], [Kurochkin 2016].The virtual volunteer community is difficult to join.
    [Show full text]
  • Efficient Scheduling Heuristics for Independent Tasks In
    CORE Metadata, citation and similar papers at core.ac.uk Provided by ethesis@nitr Efficient Scheduling Heuristics for Independent Tasks in Computational Grids Sanjaya Kumar Panda Department of Computer Science and Engineering National Institute of Technology Rourkela Rourkela - 769008, Odisha, India Efficient Scheduling Heuristics for Independent Tasks in Computational Grids Thesis submitted in partial fulfillment of the requirements for the degree of Master of Technology in Computer Science and Engineering by Sanjaya Kumar Panda (Roll No.: 211CS2285) under the supervision of Prof. Pabitra Mohan Khilar Assistant Professor Department of Computer Science and Engineering, NIT Rourkela Department of Computer Science and Engineering National Institute of Technology Rourkela Rourkela - 769008, Odisha, India Department of Computer Science and Engineering National Institute of Technology Rourkela Rourkela - 769008, Odisha, India. Rourkela 22nd May 2013 Certificate This is to certify that the work in the thesis entitled Efficient Scheduling Heuristics for Independent Tasks in Computational Grids by Sanjaya Kumar Panda, bearing Roll Number 211CS2285, is a record of an original research work carried out by him under my supervision and guidance in partial fulfillment of the requirements for the award of the degree of Master of Technology in Computer Science and Engineering with specialization in Information Security. Neither this thesis nor any part of it has been submitted for any degree or academic award elsewhere. Prof. Pabitra Mohan Khilar Acknowledgment I owe deep gratitude to the ones who have contributed greatly in completion of this thesis. Foremost, I would like to express my sincere gratitude to my supervisor Prof. Pabitra Mohan Khilar for providing me with a platform to work on challenging areas of grid computing and fault tolerance.
    [Show full text]
  • The Planetary Report — May-June 2007
    The PLANETARYPLANETARY REPORT REPORT Volume XXVII Number 3 May/June 2007 Examining Mars From The Editor On the Cover: admit it—I’m not objective about SETI@home. The Mars Exploration Rovers, Spirit and Opportunity, were designed to last three I My feelings for this scientific research project are months on the surface of the Red Planet. Three years and more than 180,000 close to maternal. images of Mars later, these hardworking robotic explorers are still making thrilling When David Anderson and Dan Werthimer, the proj- discoveries. Opportunity’s Panoramic Camera (Pancam) imaged these sand dunes ect’s leaders, came to us with their proposal, not every- at the bottom of Endurance crater on sol (Martian day) 211. Some of these dunes one here could see what to me was glaringly obvious: are more than a meter tall from crest to trough. The image is processed in false even if it was audacious and revolutionary, SETI@home color for mineral analysis. Image: NASA/JPL/Cornell University could involve the public in scientific research on the widest scale ever. So, when the response was leaning toward “No,” Background: I declared that “No” was not an appropriate answer, Once the Mars Exploration Rover Opportunity finished exploring Endurance crater, twisted arms every which way, and with a few stout- it drove south to examine how the discarded heat shield it had used during land- hearted cohorts found the start-up money. The SETI@ ing had weathered. Next to the heat shield was a rock—the only one for kilome- home project now acknowledges The Planetary Society’s ters around.
    [Show full text]