DOCSLIB.ORG

Automated Optimization Methods for Scientific Workflows in E-Science Infrastructures

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Automated Optimization Methods for Scientific Workflows in E-Science Infrastructures Scientific workflows have emerged as a key technology that assists scientists with the design, management, execution, sharing and reuse of in silico experiments. Workflow management systems simplify the management of scientific workflows by providing graphical interfaces for their development, monitoring and analysis. Nowadays, e-Science combines such workflow man- agement systems with large-scale data and computing resources into complex research infra- structures. For instance, e-Science allows the conveyance of best practice research in collab- orations by providing workflow repositories, which facilitate the sharing and reuse of scientific workflows. However, scientists are still faced with different limitations while reusing workflows. One of the most common challenges they meet is the need to select appropriate applications and their individual execution parameters. If scientists do not want to rely on default or experi- ence-based parameters, the best-effort option is to test different workflow set-ups using either trial and error approaches or parameter sweeps. Both methods may be inefficient or time con- suming respectively, especially when tuning a large number of parameters. Therefore, scientists require an effective and efficient mechanism that automatically tests different workflow set-ups in an intelligent way and will help them to improve their scientific results. This thesis addresses the limitation described above by defining and implementing an approach for the optimization of scientific workflows. In the course of this work, scientists’ needs are investigated and requirements are formulated resulting in an appropriate optimization concept. This concept is prototypically implemented by extending a workflow management system with an Workflows of Scientific Optimization Automated optimization framework. This implementation and therewith the general approach of workflow optimization is experimentally verified by four use cases in the life science domain. Finally, a new collaboration-based approach is introduced that harnesses optimization provenance to make optimization faster and more robust in the future. This publication was written at the Jülich Supercomputing Centre (JSC) which is an integral part of the Institute for Advanced Simulation (IAS). The IAS combines the Jülich simulation sciences and the supercomputer facility in one organizational unit. It includes those parts of the scientific institutes at Forschungszentrum Jülich which use simulation on supercomputers as their main Sonja Holl research methodology. IAS Series IAS Series Automated Optimization Methods for Scientific Workflows in e-Science Infrastructures Sonja Holl IAS Series Volume 24 ISBN 978-3-89336-949-2 24 Member of the Helmholtz Association Member of the Automated Optimization Methods for Scientific Workflows in e-Science Infrastructures Sonja Holl Member of the Helmholtz Association Member of the Schriften des Forschungszentrums Jülich IAS Series Volume 24 Forschungszentrum Jülich GmbH Institute for Advanced Simulation (IAS) Jülich Supercomputing Centre (JSC) Automated Optimization Methods for Scientific Workflows in e-Science Infrastructures Sonja Holl Schriften des Forschungszentrums Jülich IAS Series Volume 24 ISSN 1868-8489 ISBN 978-3-89336-949-2 Bibliographic information published by the Deutsche Nationalbibliothek. The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data are available in the Internet at http://dnb.d-nb.de. Publisher and Forschungszentrum Jülich GmbH Distributor: Zentralbibliothek 52425 Jülich Phone +49 (0) 24 61 61-53 68 · Fax +49 (0) 24 61 61-61 03 e-mail: [email protected] Internet: http://www.fz-juelich.de/zb Cover Design: Jülich Supercomputing Centre, Forschungszentrum Jülich GmbH Printer: Grafische Medien, Forschungszentrum Jülich GmbH Copyright: Forschungszentrum Jülich 2014 Schriften des Forschungszentrums Jülich IAS Series Volume 24 D 5 (Diss., Bonn, Univ., 2014) ISSN 1868-8489 ISBN 978-3-89336-949-2 Persistent Identifier: urn:nbn:de:0001-2014022000 Resolving URL: http://www.persistent-identifier.de/?link=610 Neither this book nor any part of it may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, microfilming, and recording, or by any information storage and retrieval system, without permission in writing from the publisher. Abstract Scientific workflows have emerged as a key technology that assists scientists with the design, management, execution, sharing and reuse of in silico experiments. Workflow man- agement systems simplify the management of scientific workflows by providing graphical interfaces for their development, monitoring and analysis. Nowadays, e-Science combines such workflow management systems with large-scale data and computing resources into complex research infrastructures. For instance, e-Science allows the conveyance of best practice research in collaborations by providing workflow repositories, which facilitate the sharing and reuse of scientific workflows. However, scientists are still faced with different limitations while reusing workflows. One of the most common challenges they meet is the need to select appropriate applications and their individual execution parameters. If scien- tists do not want to rely on default or experience-based parameters, the best-effort option is to test different workflow set-ups using either trial and error approaches or parameter sweeps. Both methods may be inefficient or time consuming respectively, especially when tuning a large number of parameters. Therefore, scientists require an effective and efficient mechanism that automatically tests different workflow set-ups in an intelligent way and will help them to improve their scientific results. This thesis addresses the limitation described above by defining and implementing an approach for the optimization of scientific workflows. In the course of this work, scien- tists’ needs are investigated and requirements are formulated resulting in an appropriate optimization concept. In a following step, this concept is prototypically implemented by extending a workflow management system with an optimization framework, including general mechanisms required to conduct workflow optimization. As optimization is an ongoing research topic, different algorithms are provided by pluggable extensions (plugins) that can be loosely coupled with the framework, resulting in a generic and quickly extend- able system. In this thesis, an exemplary plugin is introduced which applies a Genetic Algorithm for parameter optimization. In order to accelerate and therefore make workflow optimization feasible at all, e-Science infrastructures are utilized for the parallel execution of scientific workflows. This is empowered by additional extensions enabling the execution of applications and workflows on distributed computing resources. The actual implementation and therewith the general approach of workflow optimization is experimentally verified by four use cases in the life science domain. All workflows were significantly improved, which demonstrates the advantage of the proposed workflow optimization. Finally, a new collaboration-based approach is introduced that harnesses optimization provenance to make optimization faster and more robust in the future. Acknowledgements I would like to express my gratitude to all the people who supported me in any way during my PhD project. First and foremost, I would like to thank Prof. Martin Hofmann-Apitius for his advice and guidance though such a versatile research project. Visiting his research department was a wonderful working experience, especially collaborating with such a friendly and open minded group. I would like to thank Prof. Thomas Lippert and Daniel Mallmann for the opportunity to conduct my PhD research within the Federated Systems and Data group at the Jülich Supercomputing Center in the Forschungszentrum Jülich. It was a great workplace offering both an excellent technical infrastructure and social environment. I am also very thankful to my supervisor Olav Zimmermann for his continuous support, fruitful discussions and numerous comments helping me sharpen my doctoral studies. Moreover, I thank Prof. Heiko Schoof to act as a very excited co-referee as well as further referees, Prof. Rainer Manthey and Prof. Diana Imhof. I would like to acknowledge all my colleagues at the Jülich Supercomputing Center for providing technical background during the implementation, enormous efforts to create a stable and reliable infrastructure, proof-reading parts of this thesis, and offering a warm and cheerful office atmosphere in the coldest office in JSC. Additional thanks go to all the Studium Universale members offering friendly meetings as a welcome alternation to the PhD work. A special thanks goes to Prof. Carole Goble for hosting me at the University of Manchester and her team, in person Dr. Khalid Belhajjame, Alan Williams, Stian Soiland- Reyes, Dr. Alexandra Nenadic and Dr. Katy Wolstencroft for providing excellent support and assistance during the implementation phase on extensions to the Taverna Workflow Management System and Research Objects. Many thanks are due to my collaboration partners namely Prof. Magnus Palmblad, Dr. Yassene Mohammed, Daniel Garijo, Dr. Matthias Obst, Renato De Giovanni, Shweta Bagewadi, and Dr. Philipp Senger for their fruitful discussions and great assistance with biological, medical or workflow provenance issues. Finally, I am deeply grateful to
Load more

Recommended publications
  • UNICORE OPTIMIZATION William Jalby
    UNICORE OPTIMIZATION William Jalby LRC ITA@CA (CEA DAM/ University of Versailles St-Quentin-en-Yvelines) FRANCE 1 Outline The stage Key unicore performance limitations (excluding caches) Multimedia Extensions Compiler Optimizations 2 Abstraction Layers in Modern Systems Application Algorithm/Libraries CS Programming Language Original Compilers/Interpreters domain of Operating System/Virtual Machines Domain of the computer recent architect Instruction Set Architecture (ISA) computer architecture (‘50s-’80s) Microarchitecture (‘90s) Gates/Register-Transfer Level (RTL) Circuits EE Devices Physics Key issue Application Algorithm/Libraries Understand the We have to take into relationship/interaction account the between Architecture intermediate layers Microarchitecture and Applications/Algorithms Microarchitecture KEY TECHNOLOGY: Don’t forget also the lowest layers Performance Measurement and Analysis Performance Measurement and Analysis AN OVERLOOKED ISSUE HARDWARE VIEW: mechanism description and a few portions of codes where it works well (positive view) COMPILER VIEW: aggregate performance number (SPEC), little correlation with hardware Lack of guidelines for writing efficient programs Uniprocessor Performance From Hennessy and Patterson, Computer Architecture: A Quantitative Approach, 4th edition, October, 2006 - VAX: 25%/year 1978 to 1986 - RISC + x86: 52%/year 1986 to 2002 RISC + x86: ??%/year 2002 to present Trends Unicore Performance REF: Mikko Lipasti-University of [source: Intel] Wisconsin Modern Unicore Stage KEY PERFORMANCE
    [Show full text]
  • Microsoft Office Sharepoint Server Custom Application Development: Document Workflow Management Project
    Microsoft Office SharePoint Server Custom Application Development: Document Workflow Management Project Author: Eric Charran, Microsoft Corporation Date published: February 2009 Applies to: Microsoft® Office SharePoint® Server 2007 VSTS Rangers: This content was created in a VSTS Rangers project. “Our mission is to accelerate the adoption of Microsoft® Visual Studio® Team System by delivering out-of-band solutions for missing features or guidance. We work closely with members of Microsoft Services and MVP communities to make sure that their solutions address real-world blockers.” Bijan Javidi, VSTS Ranger Lead Summary: This white paper reviews the successful Microsoft Consulting Services (MCS) design, construction, and deployment of a custom Microsoft® Office SharePoint® Server 2007 application to a mid-market enterprise customer. This documentation will serve to educate and familiarize customers who are planning to undertake similar custom Office SharePoint Server 2007 application projects within their own organizations. The guidance and experiences come directly from a real-world field implementation and contain patterns and practices in addition to descriptions of the problem, business challenge, the solution's technical structure, and the staffing model that was used to implement the application. The audience for this document includes customers who are interested in conducting a software development life cycle for a custom application that will be built on the Microsoft SharePoint Products and Technologies platform. This document not only discusses the project's goals and vision from inception, but also follows the project through staffing by identifying successful team models. The solution's physical design and best practice considerations are documented and provide background and context for the presented lessons learned.
    [Show full text]
  • Workflow Definition, Benefits, and Preparation
    4 Country View Road Malvern, PA 19355 800.223.7036 610.647.5930 S www.sct.com Workflow definition, benefits, and preparation An SCT White Paper EWHT-005 (02/03) SCT, Banner, PowerCAMPUS, and the circle P logo are registered trademarks; and the SCT logo, SCT Plus, SCT Matrix, SCT Campus Pipeline, and Luminis are trademarks, of Systems & Computer Technology Corporation or one or more of its wholly owned subsidiaries. All other products and company names referenced herein are used for identification purposes only and may be trademarks of their respective owners. Copyright © 2003 Systems & Computer Technology Corporation. All rights reserved. Workflow (cont.) What is Workflow? Most colleges and universities today practice workflow of some kind, either manually or with software applications. Admitting a student, for example, requires more than one person (and more than one department) to find the prospect, review the application, compile the financial aid package, and send the offer letter. The process of passing information along and responding to it is a type of workflow. However, more often than not, the process grinds to a halt when one of those papers languishes in a participant’s in-box. A software-based workflow management system would speed this process—and hundreds of others like it—and elevate its quality by automating, simplifying, measuring, directing, and managing the flow of information from department to department across the enterprise. There are many different definitions of workflow. For consistency, SCT follows the definition put forth by the respected Workflow Management Coalition (WfMC): The coalition also states that a true workflow management system must have the following criteria: • Define, create, and manage the execution of workflows with software • Run on one or more workflow engines • Interpret the process definition • Interact with workflow participants • Invoke IT tools and applications where required In addition, workflow solutions comprise data, people, tools, and activities.
    [Show full text]
  • Introduction to Workflows April 6, 2020
    Tutorial An Introduction to Workflows April 6, 2020 Sample to Insight QIAGEN Aarhus Silkeborgvej 2 Prismet 8000 Aarhus C Denmark digitalinsights.qiagen.com [email protected] An Introduction to Workflows 2 Tutorial An Introduction to Workflows A workflow consists of a series of connected tools where the output of one tool is used as input for another tool, making it possible to analyze many samples using a standardized pipeline. This tutorial covers: • Setting up a workflow that maps reads to a reference sequence, detects variants in the data relative to the reference, and filters away variants that are also present in a database of common variants • Launching a workflow in batch mode to analyze multiple samples • Workflow management, including creating a workflow installer, for installing the workflow on your own or another CLC Workbench, or on a CLC Server, where all or a selected group of server users could make use of it Steps where action should be taken are numbered. Prerequisites: We recommend using CLC Genomics Workbench 20.0 or higher when working through this tutorial. Some functionality referred to is not available in earlier versions. Download and import data for the tutorial The example data for this tutorial includes two files of sequencing reads and several reference tracks: the human mitochondrial genome from the hg18 build, NC_0O1807 (Genome), corre- sponding CDS and Gene tracks, and a chrMdbSNPCommon track containing the dbSNP common variants for this mitochondrial sequence. 1. Download the example data from http://resources.qiagenbioinformatics.com/ testdata/chrM-tutorial-data.zip. 2. Start the CLC Genomics Workbench.
    [Show full text]
  • Computer Architecture Techniques for Power-Efficiency
    MOCL005-FM MOCL005-FM.cls June 27, 2008 8:35 COMPUTER ARCHITECTURE TECHNIQUES FOR POWER-EFFICIENCY i MOCL005-FM MOCL005-FM.cls June 27, 2008 8:35 ii MOCL005-FM MOCL005-FM.cls June 27, 2008 8:35 iii Synthesis Lectures on Computer Architecture Editor Mark D. Hill, University of Wisconsin, Madison Synthesis Lectures on Computer Architecture publishes 50 to 150 page publications on topics pertaining to the science and art of designing, analyzing, selecting and interconnecting hardware components to create computers that meet functional, performance and cost goals. Computer Architecture Techniques for Power-Efficiency Stefanos Kaxiras and Margaret Martonosi 2008 Chip Mutiprocessor Architecture: Techniques to Improve Throughput and Latency Kunle Olukotun, Lance Hammond, James Laudon 2007 Transactional Memory James R. Larus, Ravi Rajwar 2007 Quantum Computing for Computer Architects Tzvetan S. Metodi, Frederic T. Chong 2006 MOCL005-FM MOCL005-FM.cls June 27, 2008 8:35 Copyright © 2008 by Morgan & Claypool All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means—electronic, mechanical, photocopy, recording, or any other except for brief quotations in printed reviews, without the prior permission of the publisher. Computer Architecture Techniques for Power-Efficiency Stefanos Kaxiras and Margaret Martonosi www.morganclaypool.com ISBN: 9781598292084 paper ISBN: 9781598292091 ebook DOI: 10.2200/S00119ED1V01Y200805CAC004 A Publication in the Morgan & Claypool Publishers
    [Show full text]
  • Installation Guide for UNICORE Server Components
    Installation Guide for UNICORE Server Components Installation Guide for UNICORE Server Components UNICORE Team July 2015, UNICORE version 7.3.0 Installation Guide for UNICORE Server Components Contents 1 Introduction1 1.1 Purpose and Target Audience of this Document.................1 1.2 Overview of the UNICORE Servers and some Terminology...........1 1.3 Overview of this Document............................2 2 Installation of Core Services for a Single Site3 2.1 Basic Scenarios..................................3 2.2 Preparation....................................5 2.3 Installation....................................6 2.4 Security Settings................................. 15 2.5 Installation of the Perl TSI and TSI-related Configuration of the UNICORE/X server....................................... 18 2.6 The Connections Between the UNICORE Components............. 20 3 Operation of a UNICORE Installation 22 3.1 Starting...................................... 22 3.2 Stopping...................................... 22 3.3 Monitoring.................................... 22 3.4 User Management................................. 22 3.5 Testing your Installation............................. 23 4 Integration of Another Target System 24 4.1 Configuration of the UNICORE/X Service.................... 24 4.2 Configuration of Target System Interface..................... 25 4.3 Addition of Users to the XUUDB........................ 26 4.4 Additions to the Gateway............................. 26 5 Multi-Site Installation Options 26 5.1 Multiple Registries...............................
    [Show full text]
  • A Adapting Scientific Workflow Structures Using Multi-Objective
    A Adapting Scientific Workflow Structures Using Multi-Objective Optimisation Strategies IRFAN HABIB, Centre for Complex Cooperative Systems, FET, University of the West of England, Bristol, UK ASHIQ ANJUM, School of Computing and Mathematics, University of Derby, Derby, UK RICHARD MCCLATCHEY, Centre for Complex Cooperative Systems, FET, University of the West of England, Bristol, UK OMER RANA, School of Computer Science & Informatics, Cardiff University, UK Scientific workflows have become the primary mechanism for conducting analyses on distributed comput- ing infrastructures such as grids and clouds. In recent years, the focus of optimisation within scientific workflows has primarily been on computational tasks and workflow makespan. However, as workflow-based analysis becomes ever more data intensive, data optimisation is becoming a prime concern. Moreover, scien- tific workflows can scale along several dimensions: (i) number of computational tasks, (ii) heterogeneity of computational resources, and the (iii) size and type (static vs. streamed) of data involved. Adapting workflow structure in response to these scalability challenges remains an important research objective. Understand- ing how a workflow graph can be restructured in an automated manner (through task merge for instance), to address constraints of a particular execution environment is explored in this work, using a multi-objective evolutionary approach. Our approach attempts to adapt the workflow structure to achieve both compute and data optimisation. The question of when to terminate the evolutionary search in order to conserve compu- tations is tackled with a novel termination criterion. The results presented in this paper, demonstrate the feasibility of the termination criterion and demonstrate that significant optimisation can be achieved with a multi-objective approach.
    [Show full text]
  • Intel's High-Performance Computing Technologies
    Intel’s High-Performance Computing Technologies 11th ECMWF Workshop Use of HIgh Performance Computing in Meteorology Reading, UK 26-Oct-2004 Dr. Herbert Cornelius Advanced Computing Center Intel EMEA Advanced Computing on Intel® Architecture Intel HPC Technologies October 2004 HPC continues to change … *Other brands and names are the property of their respective owners •2• Advanced Computing on Intel® Architecture Intel HPC Technologies October 2004 Some HPC History 1960s 1970s 1980s 1990s 2000s HPC Systems 1970s 1980s 1990s 2000s Processor proprietary proprietary COTS COTS Memory proprietary proprietary COTS COTS Motherboard proprietary proprietary proprietary COTS Interconnect proprietary proprietary proprietary COTS OS, SW Tools proprietary proprietary proprietary mixed COTS: Commercial off the Shelf (industry standard) *Other brands and names are the property of their respective owners •3• Advanced Computing on Intel® Architecture Intel HPC Technologies October 2004 High-Performance Computing with IA Source: http://www.top500.org/lists/2004/06/2/ Source: http://www.top500.org/lists/2004/06/5/ 4096 (1024x4) Intel® Itanium® 2 processor based system 2500 (1250x2) Intel® Xeon™ processor based system 22.9 TFLOPS peak performance 15.3 TFLOPS peak performance PNNL RIKEN 9 1936 Intel® Itanium® 2 processor cluster 7 2048 Intel® Xeon™ processor cluster 11.6 / 8.6 TFLOPS Rpeak/Rmax 12.5 / 8.7 TFLOPS Rpeak/Rmax *Other brands and names are the property of their respective owners •4• Advanced Computing on Intel® Architecture Intel HPC Technologies
    [Show full text]
  • How to Describe Workflow Information Systems to Support Business Process
    How to Describe Workflow Information Systems to Support Business Process Josefina Guerrero García, Jean Vanderdonckt, Christophe Lemaige, Juan M. González Calleros 1Université catholique de Louvain, Louvain School of Management Place des Doyens, 1 – B-1348 Louvain-la-Neuve (Belgium) Abstract ticipant to another for action, according to a set of pro- cedural rules” [26]. Workflow Information Systems (WIS) cover the application of information technology This paper addresses a methodology for developing to business problems. Its primary characteristic is the the various user interfaces (UI) of a workflow informa- automation of processes involving combinations of tion system (WIS), which are advocated to automate human activities with information technology applica- business processes, following a model-centric ap- tions. Owing to the fact that the users of a IS interact proach based on the requirements and processes of the with it through its user interfaces (UIs) in the pursuit organization. The methodology applies to: 1) integrate of organizational goals, flexibility in creating them is human and machines based activities, in particular therefore important. We will explore a systematic way those involving interaction with IT applications and to define UIs for a WIS. The workflow model defines tools, 2) to identify how tasks are structured, who per- what processes and tasks need to be fulfilled and their form them, what their relative order is, how they are possible ordering; hence the workflow model is a offered or assigned, and how tasks are being tracked. ‘framework’ for creating task model, hence the task For this purpose, workflow is recursively decomposed model is suitable for developing UIs.
    [Show full text]
  • Automatic Configuration of Multi-Objective Optimizers and Multi-Objective Configuration
    Chapter 1 Automatic Configuration of Multi-objective Optimizers and Multi-objective Configuration Leonardo C. T. Bezerra1, Manuel López-Ibáñez2, and Thomas Stützle3 Abstract Heuristic optimizers are an important tool in academia and industry, and their performance-optimizing configuration requires a significant amount of expertise. As the proper configuration of algorithms is a crucial aspect in the engineering of heuristic algorithms, a significant research effort has been dedicated over the last years towards moving this step to the computer and, thus, make it automatic. These research efforts go way beyond tuning only numerical parameters of already fully defined algorithms, but exploit automatic configuration as a means for automatic algorithm design. In this chapter, we review two main aspects where the research on automatic configuration and multi-objective optimization intersect. The first is the automatic configuration of multi-objective optimizers, where we discuss means and specific approaches. In addition, we detail a case study that shows how these approaches can be used to design new, high-performing multi-objective evolutionary algorithms. The second aspect is the research on multi-objective configuration, that is, the possibility of using multiple performance metrics for the evaluation of algorithm configurations. We highlight some few examples in this direction. Key words: multi-objective optimization, automatic algorithm configuration, automatic configuration of multi-objective optimizers, automatic configuration considering multiple objectives Instituto Metrópole Digital (IMD), Universidade Federal do Rio Grande do Norte (UFRN), Natal, RN, Brazil ([email protected]) · Alliance Manchester Business School, University of Manchester, UK ([email protected]) · IRIDIA, Université Libre de Bruxelles (ULB), Brussels, Belgium ([email protected]).
    [Show full text]
  • Application of Computational Intelligence to Explore and Analyze System Architecture and Design Alternatives
    Scholars' Mine Doctoral Dissertations Student Theses and Dissertations Fall 2019 Application of computational intelligence to explore and analyze system architecture and design alternatives Gene Lesinski Follow this and additional works at: https://scholarsmine.mst.edu/doctoral_dissertations Part of the Systems Engineering Commons Department: Engineering Management and Systems Engineering Recommended Citation Lesinski, Gene, "Application of computational intelligence to explore and analyze system architecture and design alternatives" (2019). Doctoral Dissertations. 2837. https://scholarsmine.mst.edu/doctoral_dissertations/2837 This thesis is brought to you by Scholars' Mine, a service of the Missouri S&T Library and Learning Resources. This work is protected by U. S. Copyright Law. Unauthorized use including reproduction for redistribution requires the permission of the copyright holder. For more information, please contact [email protected]. APPLICATION OF COMPUTATIONAL INTELLIGENCE TO EXPLORE AND ANALYZE SYSTEM ARCHITECTURE AND DESIGN ALTERNATIVES by EUGENE JOSEPH LESINSKI III A DISSERTATION Presented to the Faculty of the Graduate Faculty of the MISSOURI UNIVERSITY OF SCIENCE AND TECHNOLOGY In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY in SYSTEMS ENGINEERING 2019 Approved by: Steven Corns, PhD, Advisor Suzanna Long, PhD Cihan Dagli, PhD Stephen Raper, PhD Donald Wunsch, PhD John Farr, PhD 2019 Eugene Joseph Lesinski III All Rights Reserved iii PUBLICATION DISSERTATION OPTION This dissertation consists of four peer reviewed publications that have been published, accepted, or that will be submitted, which have been formatted in the style used by the Missouri University of Science & Technology. Paper I: Pages 13-29 is a paper that has been published in IEEE Conference on Evolutionary Computation.
    [Show full text]
  • Implementing Production Grids William E
    Implementing Production Grids William E. Johnston a, The NASA IPG Engineering Team b, and The DOE Science Grid Team c Contents 1 Introduction: Lessons Learned for Building Large-Scale Grids ...................................... 3 5 2 The Grid Context .................................................................................................................. 3 The Anticipated Grid Usage Model Will Determine What Gets Deployed, and When. 7 3.1 Grid Computing Models ............................................................................................................ 7 3 1.1 Export Existing Services .......................................................................................................... 7 3 1.2 Loosely Coupled Processes ..................................................................................................... 7 3.1 3 WorlqTow Managed Processes .............................................................................. 8 3.1 4 Distributed-Pipelined / Coupled processes ............................................................................. 9 3.1 5 Tightly Coupled Processes ........................................................................ 9 3.2 Grid Data Models ..................................................................................................................... 10 3.2.1 Occasional Access to Multiple Tertiary Storage Systems ..................................................... 11 3.2.2 Distributed Analysis of Massive Datasets Followed by Cataloguing and Archiving ...........
    [Show full text]