DOCSLIB.ORG

Contributions for Resource and Job Management in High Performance Computing Yiannis Georgiou

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Contributions for Resource and Job Management in High Performance Computing Yiannis Georgiou Contributions for Resource and Job Management in High Performance Computing Yiannis Georgiou To cite this version: Yiannis Georgiou. Contributions for Resource and Job Management in High Performance Computing. Distributed, Parallel, and Cluster Computing [cs.DC]. Université de Grenoble, 2010. English. tel- 01499598 HAL Id: tel-01499598 https://hal-auf.archives-ouvertes.fr/tel-01499598 Submitted on 31 Mar 2017 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Public Domain THÈSE Pour obtenir le grade de DOCTEUR DE L’UNIVERSITÉ DE GRENOBLE Spécialité : Informatique Arrêté ministériel : 7 août 2006 Présentée par « Yiannis GEORGIOU » Thèse dirigée par « Olivier RICHARD » et codirigée par « Jean-Francois MEHAUT » préparée au sein du Laboratoire d'Informatique de Grenoble dans l'École Doctorale de Mathématiques, Sciences et Technologies de l'Information, Informatique Contributions for Resource and Job Management in High Performance Computing Thèse soutenue publiquement le « 5 novembre 2010 », devant le jury composé de : M. Daniel, HAGIMONT Professeur à INPT/ENSEEIHT, France, Président M. Franck, CAPPELLO Directeur de Recherche à INRIA, France, Rapporteur M. William T.C., KRAMER Directeur de Recherche à NCSA, USA, Rapporteur M. Morris, JETTE Informaticien au LLNL, USA, Membre Mme. Pascale, ROSSE-LAURENT Architecte Informatique à BULL, France, Membre M. Jean-Francois, MEHAUT Professeur à l'UJF, France, Membre M. Olivier, RICHARD Maître de Conférence à l'UJF, France, Membre CONTRIBUTIONS FOR RESOURCE AND JOB MANAGEMENT IN HIGH PERFORMANCE COMPUTING Yiannis Georgiou November 2010 Sth γιαγιά mou KrustallÐa Abstract High Performance Computing is characterized by the latest technological evolutions in com- puting architectures and by the increasing needs of applications for computing power. A particular middleware called Resource and Job Management System (RJMS), is responsible for delivering computing power to applications. The RJMS plays an important role in HPC since it has a strategic place in the whole software stack because it stands between the above two layers. However the latest evolutions in hardware and applications layers have provided new levels of complexities to this middleware. Issues like scalability, management of topological constraints, energy efficiency and fault tolerance have to be particularly considered, among others, in order to provide a better system exploitation from both the system and user point of view. This dissertation provides a state of the art upon the fundamental concepts and research issues of Resources and Jobs Management Systems. It provides a multi-level comparison (concepts, functionalities, performance) of some of the most widely used Resource and Jobs Management Systems in High Performance Computing. An important metric to evaluate the work of a RJMS on a platform is the observed system utilization. However studies and logs of production platforms show that HPC systems in general suffer of significant unutilization rates. Our study deals with these clusters’ unutilization periods by proposing methods to aggregate otherwise unutilized resources for the benefit of the system or the application. More particularly this thesis explores RJMS level mechanisms: 1) for increasing the jobs valuable computation rates in the high volatile environments of a lightweight grid context, 2) for improving system utilization with malleability techniques and 3) providing energy efficient system management through the exploitation of idle computing machines. The experimentation and evaluation in this type of contexts provide important complexities due to the inter-dependency of multiple parameters that have to be taken into control. In our study we have developed a methodology based upon real-scale controlled experimentation with submission of synthetic or real workload traces. Acknowledgements First of all I would like to thank my advisor Olivier Richard who accepted me as I am, back in the days when my main motivation for an internship in the laboratory was the proximity of ski resorts. Thank you for giving me the opportunity to discover the world of research in computer science. Thank you for your patience and continuous support during my internship, master’s and PhD. The writing of this PhD would never have been possible without your guidance and motivation throughout all these years. Then I would like to express my gratitude to my second advisor Pascale Rosse-Laurent for her mentorship within the enterprise environment. Thank you for all the time, interesting discussions and precious advices. Thank you for showing me the way and supporting me during my PhD and afterwards. Besides my advisors, I would like to thank my colleague and friend Marcia Cristina Cera with whom I collaborated during our PhDs. She was a very nice person always cheerful and motivated. It was a pleasure working with her and it is so difficult for me to believe that she is not here anymore. I also thank Prof. Jean-Francois Mehaut for his time and exchanges and Andry Razafinjatovo for the motivation and support throughout the final stage of my thesis. My sincere thanks goes to the reporters of my thesis Dr. Franck Cappello, Dr. William T.C. Kramer and Mr. Morris Jette for their precious comments and corrections. I would also like to thank my colleagues from the laboratory and enterprise for the nice working environment. A special thanks goes to all my friends and flat-mates particularly Eleni, Lauriane, Vasilis and Alexandros. Thanks for all the fun and for standing by me during the difficult moments of this thesis. Last but not least, I would like to thank my family for their spiritual and financial support throughout my academic studies all these years. Contents List of Tables . 7 List of Figures . 9 1 Introduction 13 1.1 Motivation . 14 1.2 Research Subjects . 15 1.2.1 Scheduling and Management of Resources and Jobs . 15 1.2.2 Techniques for efficient resources exploitation . 16 1.2.3 Experimental Methodology based on workload traces . 19 1.3 Thesis Overview . 20 2 Evaluation Methodology for Resource and Job Management Systems 21 2.1 Related Work . 22 2.1.1 Experimentation methodologies . 23 2.1.2 Workload modelling, characterization and benchmarks . 26 2.2 Real-scale experimentation upon Grid5000 platform . 28 2.2.1 Grid5000 Design and Architecture . 28 2.2.2 Automatization and Reproduction techniques . 33 2.3 Evaluation Methodology based upon Replay of Workloads . 34 2.3.1 System factors . 35 2.3.2 Workloads . 36 2.3.3 Executed Applications . 41 2.3.4 Evaluation Metrics . 45 2.3.5 Xionee: Workload Trace Analysis, Visualization and Automatic Injection Toolkit . 46 2.4 Conclusions . 49 3 3 Comparisons of Resource and Job Management Systems 51 3.1 Background and Related Work . 52 3.2 RJMS Principles . 54 3.2.1 Resource Management . 57 3.2.2 Job Management . 59 3.2.3 Scheduling . 61 3.3 Research challenges . 64 3.3.1 Topology aware Placement . 64 3.3.2 High Availability . 67 3.3.3 Energy Consumption . 68 3.3.4 Launcher and Scheduler . 69 3.4 Survey of Resource and Job Management Systems . 73 3.4.1 Commercial RJMS . 73 3.4.2 Opensource RJMS . 78 3.4.3 SLURM . 81 3.4.4 OAR . 86 3.4.5 Synthesis . 94 3.5 Performance Evaluation of opensource RJMS . 96 3.5.1 Launching and Scheduling Evaluation . 97 3.5.2 Network Topology Aware placement Evaluation . 102 3.5.3 Scalability and Efficiency Evaluation . 107 3.6 Conclusions . 114 4 Improving system utilization in a lightweight grid context 116 4.1 Background Information and Related Work . 118 4.1.1 Grid and alternative grid technologies . 118 4.1.2 Fault Treatment upon grid technologies . 119 4.1.3 Scheduling for bag-of-tasks applications . 120 4.1.4 Checkpoint-Restart Recovery technique . 121 4.2 An alternative lightweight grid computing approach for bag-of-tasks applications . 122 4.2.1 Lightweight grid and cluster utilization policy . 122 4.2.2 Global Architecture . 123 4.3 Enhancements for turnaround time optimization . 127 4.3.1 Transparent Checkpoint/Restart mechanism for a lightweight grid . 128 4 4.3.2 Periodic and Triggered checkpoints strategies . 129 4.4 Experimentation Results . 130 4.4.1 Deploying a lightweight grid upon Grid5000 platform . 131 4.4.2 Performance Evaluation . 133 4.5 Conclusions . 137 5 Improving resources exploitation with Malleability techniques 139 5.1 Background and Related Work . 140 5.1.1 Dynamic Applications and Programming . 141 5.1.2 Resource Management and Dynamicity . 143 5.1.3 RJMS and Applications Communication protocols . 144 5.1.4 Scheduling with Dynamic Applications . 146 5.2 Malleability techniques: Implementation and Experimental Testbed . 148 5.2.1 Implementation upon OAR resource manager . 148 5.2.2 Experimentation through controlled platform and real traces . 149 5.3 Malleability on Clusters with Dynamic MPI . 151 5.3.1 Dynamic MPI mechanism Requirements . 151 5.3.2 Evaluating Dynamic MPI technique . 153 5.4 Malleability on Multicore Nodes with Dynamic CPUSET Mapping . 157 5.4.1 Dynamic CPUSETs Mapping Requirements . 158 5.4.2 Evaluating Dynamic CPUSETs Mapping technique . 160 5.5 Improving Resource Utilization using Malleability . 165 5.6 Conclusions . 169 6 Energy Efficient Management Techniques 171 6.1 Related Work . 172 6.2 Measuring Energy Consumption upon Grid5000 . 174 6.2.1 The Hardware Energy-Meters . 174 6.2.2 A Library to Interface with Energy-Meters . 175 6.3 Energy Reduction through idle resources manipulation . 176 6.3.1 Adapting OAR to exploit idle resources for energy conservations .
Load more

Recommended publications
  • Operating System Support for Parallel Processes
    Operating System Support for Parallel Processes Barret Rhoden Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS-2014-223 http://www.eecs.berkeley.edu/Pubs/TechRpts/2014/EECS-2014-223.html December 18, 2014 Copyright © 2014, by the author(s). All rights reserved. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission. Operating System Support for Parallel Processes by Barret Joseph Rhoden A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate Division of the University of California, Berkeley Committee in charge: Professor Eric Brewer, Chair Professor Krste Asanovi´c Professor David Culler Professor John Chuang Fall 2014 Operating System Support for Parallel Processes Copyright 2014 by Barret Joseph Rhoden 1 Abstract Operating System Support for Parallel Processes by Barret Joseph Rhoden Doctor of Philosophy in Computer Science University of California, Berkeley Professor Eric Brewer, Chair High-performance, parallel programs want uninterrupted access to physical resources. This characterization is true not only for traditional scientific computing, but also for high- priority data center applications that run on parallel processors. These applications require high, predictable performance and low latency, and they are important enough to warrant engineering effort at all levels of the software stack.
    [Show full text]
  • Gforge-Lists-Mailman Apache2 0. Mailman 0. Gforge-Db-Postgresql
    llvm-2.8-dev mongodb-clients libfpdi-php lazarus-ide-gtk2 libecpg6 libtulip-ogl-dev gnome-mime-data 0. 0. 0. libparrot3.3.0 libwnck-3-0 0. 0. libjaxp1.3-java-gcj 0. libusbmuxd1 erlang-public-key 0. 1.93236714976 0. llvm-2.8 mongodb-dev libfpdf-tpl-php 0. libecpg-compat3 0. lazarus-ide libgemplugin-ruby1.8 0. 0. 2.24719101124 0. libtulip-dev 3.33333333333 0. 0. 0. 0. 0. 0. 0. 0. libgnomevfs2-0 gnome-desktop3-data 0. 0. 0. 1.68269230769 libopenr2-3 gir1.2-wnck-3.0 0. libxalan2-java-gcj 2.22222222222 libimobiledevice1 3.22580645161 erlang-inets 0. libgpg-error0 libblkid1 0.24154589372 libparrot-dev parrot llvm-2.8-runtime mongodb-server pnp4nagios-web libpgtypes3 libtulip-qt4-dev libnet-daemon-perl lazarus-src 0.0. 0. 0. 0. libgemplugin-ruby kdebase-workspace-data libecore-input1 0. libbft-dev libgtk2.0-common 0. 0. 0. 0.70949940875 0.749506903353 libeet-dev python-lazr.uri 0. 4.54674623473 0. cl-alexandria libxau6 0. pia freespacenotifier libksieve4 libtasn1-3 libgnutls26 0.45045045045 libwnck-3-common usbmuxd erlang-ssl libxerces2-java-gcj libgnomevfs2-common 0.0426257459506 0. gir1.2-gnomedesktop-3.0 fxload 0. dahdi-linux 0. 0. libcgns-dev 0. 0. 0. 0. mount initscripts sysvinit-utils 0. 0. reportbug libdbi-perl geant321-data 0.0394477317554 0.355029585799 0. 0. 4.07786302927 llvm-2.9 libecore-evas1 libecore-fb1 0. 0. 0.1574803149611.06709047565 libatk1.0-0 0. sugar-presence-service-0.90 0. 0. 0. 0. parrot-minimal mongrel 1.41899881751.45956607495 libgcrypt11 0. ksysguard kdebase-workspace-bin 0.
    [Show full text]
  • Gestión De Proyectos Software
    Proyecto Fin de Carrera AITForge: Gestión de Proyectos Software Autor: Antonio Domingo Lagares Alfaro Titulación: Ingeniero de Telecomunicación (Plan 98) Especialidad: Telemática Año: 2005 Tutor: Antonio Estepa Alonso AITForge: Gestión de Proyectos Software Índice de contenido 1 Prefacio..................................................................................................................6 2 Portales de Desarrollo Colaborativo......................................................................7 2.1 Introducción a los Entornos Colaborativos....................................................8 2.1.1 Hosting de Proyectos de Software Libre (FOSPHost)...........................9 2.1.2 ¿Software Libre y Software de Fuentes Abiertas?...............................10 2.1.3 Prácticas deseables en Software Libre................................................11 2.1.4 El nacimiento de una nueva filosofía de trabajo...................................13 2.1.5 Objetivos de los sistemas libres de FOSPHost....................................14 2.1.6 Principales características de los sistemas FOSPHost........................16 Características intrínsecas..........................................................................16 Características de utilidad...........................................................................17 Características de usabilidad......................................................................18 Características contextuales.......................................................................19 2.1.7
    [Show full text]
  • Sched-ITS: an Interactive Tutoring System to Teach CPU Scheduling Concepts in an Operating Systems Course
    Wright State University CORE Scholar Browse all Theses and Dissertations Theses and Dissertations 2017 Sched-ITS: An Interactive Tutoring System to Teach CPU Scheduling Concepts in an Operating Systems Course Bharath Kumar Koya Wright State University Follow this and additional works at: https://corescholar.libraries.wright.edu/etd_all Part of the Computer Engineering Commons, and the Computer Sciences Commons Repository Citation Koya, Bharath Kumar, "Sched-ITS: An Interactive Tutoring System to Teach CPU Scheduling Concepts in an Operating Systems Course" (2017). Browse all Theses and Dissertations. 1745. https://corescholar.libraries.wright.edu/etd_all/1745 This Thesis is brought to you for free and open access by the Theses and Dissertations at CORE Scholar. It has been accepted for inclusion in Browse all Theses and Dissertations by an authorized administrator of CORE Scholar. For more information, please contact [email protected]. SCHED – ITS: AN INTERACTIVE TUTORING SYSTEM TO TEACH CPU SCHEDULING CONCEPTS IN AN OPERATING SYSTEMS COURSE A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science By BHARATH KUMAR KOYA B.E, Andhra University, India, 2015 2017 Wright State University WRIGHT STATE UNIVERSITY GRADUATE SCHOOL April 24, 2017 I HEREBY RECOMMEND THAT THE THESIS PREPARED UNDER MY SUPERVISION BY Bharath Kumar Koya ENTITLED SCHED-ITS: An Interactive Tutoring System to Teach CPU Scheduling Concepts in an Operating System Course BE ACCEPTED IN PARTIAL FULFILLMENT OF THE REQIREMENTS FOR THE DEGREE OF Master of Science. _____________________________________ Adam R. Bryant, Ph.D. Thesis Director _____________________________________ Mateen M. Rizki, Ph.D. Chair, Department of Computer Science and Engineering Committee on Final Examination _____________________________________ Adam R.
    [Show full text]
  • Scheduling: Introduction
    7 Scheduling: Introduction By now low-level mechanisms of running processes (e.g., context switch- ing) should be clear; if they are not, go back a chapter or two, and read the description of how that stuff works again. However, we have yet to un- derstand the high-level policies that an OS scheduler employs. We will now do just that, presenting a series of scheduling policies (sometimes called disciplines) that various smart and hard-working people have de- veloped over the years. The origins of scheduling, in fact, predate computer systems; early approaches were taken from the field of operations management and ap- plied to computers. This reality should be no surprise: assembly lines and many other human endeavors also require scheduling, and many of the same concerns exist therein, including a laser-like desire for efficiency. And thus, our problem: THE CRUX: HOW TO DEVELOP SCHEDULING POLICY How should we develop a basic framework for thinking about scheduling policies? What are the key assumptions? What metrics are important? What basic approaches have been used in the earliest of com- puter systems? 7.1 Workload Assumptions Before getting into the range of possible policies, let us first make a number of simplifying assumptions about the processes running in the system, sometimes collectively called the workload. Determining the workload is a critical part of building policies, and the more you know about workload, the more fine-tuned your policy can be. The workload assumptions we make here are mostly unrealistic, but that is alright (for now), because we will relax them as we go, and even- tually develop what we will refer to as ..
    [Show full text]
  • This Book Doesn't Tell You How to Write Faster Code, Or How to Write Code with Fewer Memory Leaks, Or Even How to Debug Code at All
    Practical Development Environments By Matthew B. Doar ............................................... Publisher: O'Reilly Pub Date: September 2005 ISBN: 0-596-00796-5 Pages: 328 Table of Contents | Index This book doesn't tell you how to write faster code, or how to write code with fewer memory leaks, or even how to debug code at all. What it does tell you is how to build your product in better ways, how to keep track of the code that you write, and how to track the bugs in your code. Plus some more things you'll wish you had known before starting a project. Practical Development Environments is a guide, a collection of advice about real development environments for small to medium-sized projects and groups. Each of the chapters considers a different kind of tool - tools for tracking versions of files, build tools, testing tools, bug-tracking tools, tools for creating documentation, and tools for creating packaged releases. Each chapter discusses what you should look for in that kind of tool and what to avoid, and also describes some good ideas, bad ideas, and annoying experiences for each area. Specific instances of each type of tool are described in enough detail so that you can decide which ones you want to investigate further. Developers want to write code, not maintain makefiles. Writers want to write content instead of manage templates. IT provides machines, but doesn't have time to maintain all the different tools. Managers want the product to move smoothly from development to release, and are interested in tools to help this happen more often.
    [Show full text]
  • Cross-Platform Environment for Application Life Cycle Management
    International Journal “Information Theories and Applications”, Vol. 24, Number 2, © 2017 177 CROSS-PLATFORM ENVIRONMENT FOR APPLICATION LIFE CYCLE MANAGEMENT Elena Chebanyuk, Oleksii Hlukhov Abstract: “Application Lifecycle Management (ALM) integrates and governs the planning, definition, design, development, testing, deployment, and management phases throughout the application lifecycle” [OMG, 2006]. This paper is devoted to designing of ALM for supporting all software development processes. A review of papers, making strong contribution for improving software development life cycle processes is represented. This review touches three branches of investigation, namely papers, related to: (1) improving of communication processes between stakeholders; (2) increasing effectiveness of some operations in software development life cycle processes; (3) developing fundamental methods and tools for performing different operations related to several software development life cycle. Then comparative analysis of such ALM environments as Visual Studio, Team Foundation Server, FusionForge, TeamForge, IBM Rational Team Concert, IBM Rational Software Architect, and IBM Rational Functional Tester, is performed. Comparison of different ALM environments’ functionality lets to formulate requirements for designing cross-platform ALM environment. Then the conceptual schema of cross-platform ALM based on Eclipse environment is proposed. All plugins’ functionalities were properly tested. Collaboration of plugins for supporting several software development tasks
    [Show full text]
  • Download, Copy, Distribute, Print, Search, Or Link to the Full Texts of Articles
    Interdisciplinary Journal of Problem-Based Learning Volume 4 | Issue 2 Article 6 Published online: 9-19-2010 An Investigation of an Open-Source Software Development Environment in a Software Engineering Graduate Course Xun Ge University of Oklahoma, [email protected] Kun Huang University of Oklahoma, [email protected] Yifei Dong Openseed Institute, [email protected] IJPBL is Published in Open Access Format through the Generous Support of the Teaching Academy at Purdue University, the School of Education at Indiana University, and the Jeannine Rainbolt College of Education at the University of Oklahoma. Recommended Citation Ge, X. , Huang, K. , & Dong, Y. (2010). An Investigation of an Open-Source Software Development Environment in a Software Engineering Graduate Course. Interdisciplinary Journal of Problem-Based Learning, 4(2). Available at: https://doi.org/10.7771/1541-5015.1120 This document has been made available through Purdue e-Pubs, a service of the Purdue University Libraries. Please contact [email protected] for additional information. This is an Open Access journal. This means that it uses a funding model that does not charge readers or their institutions for access. Readers may freely read, download, copy, distribute, print, search, or link to the full texts of articles. This journal is covered under the CC BY-NC-ND license. An Investigation of an Open-Source Software Development Environment in a Software Engineering Course Xun Ge, Kun Huang, and Yifei Dong Abstract A semester-long ethnography study was carried out to investigate project-based learning in a graduate software engineering course through the implementation of an Open-Source Software Development (OSSD) learning environment, which featured authentic projects, learning community, cognitive apprenticeship, and technology affordances.
    [Show full text]
  • Githosting Public Git Hosting Sites
    GitHosting From Git SCM Wiki Public Git hosting sites Here are some places that provide free Git hosting. Check on GitServer if you want to host your own repository. List is limited to sites that provide explicit Git hosting, not including generic hosting sites that can be used to host Git repositories. Framework is Support for Open-source Space Free private Provider open-source? other SCM repositories (GB) repositories Assembla (http://www.assembla.com/catalog/51-free-private- No SVN/Hg/P4 Yes 0.15 1 project, 3 users git-repository-package?type=private&ad=git-wiki) Beanstalk (http://beanstalkapp.com/?ad=git-wiki) No SVN No 0.1 1 projects, 1 user Unlimited projects, 5 bitbucket.org (http://bitbucket.org/) No Mercurial Yes Unlimited collaborators 5 repositories, 5 Codetidy (https://codetidy.net) No No No 0.1 collaborators Public access 1 project (unlimited Codebase (http://codebasehq.com) No Mercurial/SVN 0.05 available repos), 2 collaborators CloudForge (http://www.cloudforge.com/) No CVS/SVN Yes 0.2 1 user only Unlimited projects, 6 Deveo (https://deveo.com/) No Mercurial/SVN No Unlimited collaborators Unlimited projects, 10 GitEnterprise (http://www.gitenterprise.com/) No No No 1 collaborators GitHub (http://github.com/) No SVN Yes Unlimited No Unlimited projects, GitLab.com (https://about.gitlab.com/gitlab-com/) Yes No Yes Unlimited unlimited collaborators Pikacode (http://pikacode.com/) No Mercurial Yes 1 No 1 project, 2 ProjectLocker (http://www.projectlocker.com) No SVN Read-only http 0.2 collaborators repo.or.cz (http://repo.or.cz/)
    [Show full text]
  • Collaborative Software Development Using R-Forge
    INVITED SECTION:THE FUTURE OF R 9 Collaborative Software Development Using R-Forge Special invited paper on “The Future of R” Forge, host a project-specific website, and how pack- age maintainers submit a package to the Compre- by Stefan Theußl and Achim Zeileis hensive R Archive Network (CRAN, http://CRAN. R-project.org/). Finally, we summarize recent de- velopments and give a brief outlook to future work. Introduction Open source software (OSS) is typically created in a R-Forge decentralized self-organizing process by a commu- nity of developers having the same or similar inter- R-Forge offers a central platform for the develop- ests (see the famous essay by Raymond, 1999). A ment of R packages, R-related software and other key factor for the success of OSS over the last two projects. decades is the Internet: Developers who rarely meet R-Forge is based on GForge (Copeland et al., face-to-face can employ new means of communica- 2006) which is an open source fork of the 2.61 Source- tion, both for rapidly writing and deploying software Forge code maintained by Tim Perdue, one of the (in the spirit of Linus Torvald’s “release early, release original SourceForge authors. GForge has been mod- often paradigm”). Therefore, many tools emerged ified to provide additional features for the R commu- that assist a collaborative software development pro- nity, namely a CRAN-style repository for hosting de- cess, including in particular tools for source code velopment releases of R packages as well as a quality management (SCM) and version control.
    [Show full text]
  • Managing Large-Scale, Distributed Systems Research Experiments with Control-Flows Tomasz Buchert
    Managing large-scale, distributed systems research experiments with control-flows Tomasz Buchert To cite this version: Tomasz Buchert. Managing large-scale, distributed systems research experiments with control-flows. Distributed, Parallel, and Cluster Computing [cs.DC]. Université de Lorraine, 2016. English. tel- 01273964 HAL Id: tel-01273964 https://tel.archives-ouvertes.fr/tel-01273964 Submitted on 15 Feb 2016 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. École doctorale IAEM Lorraine MANAGINGLARGE-SCALE,DISTRIBUTED SYSTEMSRESEARCHEXPERIMENTSWITH CONTROL-FLOWS THÈSE présentée et soutenue publiquement le 6 janvier 2016 pour l’obtention du DOCTORATDEL’UNIVERSITÉDELORRAINE (mention informatique) par TOMASZBUCHERT Composition du jury : Rapporteurs Christian Pérez, Directeur de recherche Inria, LIP, Lyon Eric Eide, Research assistant professor, Univ. of Utah Examinateurs François Charoy, Professeur, Université de Lorraine Olivier Richard, Maître de conférences, Univ. Joseph Fourier, Grenoble Directeurs de thèse Lucas Nussbaum, Maître de conférences, Université de Lorraine Jens Gustedt, Directeur de recherche Inria, ICUBE, Strasbourg Laboratoire Lorrain de Recherche en Informatique et ses Applications – UMR 7503 ABSTRACT(ENGLISH) Keywords: distributed systems, large-scale experiments, experiment control, busi- ness processes, experiment provenance Running experiments on modern systems such as supercomputers, cloud infrastructures or P2P networks became very complex, both technically and methodologically.
    [Show full text]
  • (RES) Architecture Study
    Reuse Enablement System (RES) Architecture Study Prepared by: NASA Earth Science Data Systems – Software Reuse Working Group July 23, 2007 Updated February 14, 2008 Earth Science Data Systems Software Reuse Working Group Editor: James Marshall (Innovim / NASA Goddard Space Flight Center) Contributing Working Group Members: Angelo Bertolli (Innovim / NASA Goddard Space Flight Center) Nancy Casey (Science Systems and Applications, Inc. / NASA Goddard Space Flight Center) Bradford Castalia (University of Arizona) Victor Delnore * (NASA Langley Research Center) Robert R. Downs (Columbia University / NASA Socioeconomic Data and Applications Center) Ryan Gerard (Innovim / NASA Goddard Space Flight Center) Mary Hunter (Innovim / NASA Goddard Space Flight Center) Shahin Samadi (Innovim / NASA Goddard Space Flight Center) Mark Sherman (SGT Inc. / NASA Goddard Space Flight Center) Ross Swick (National Snow and Ice Data Center, University of Colorado – Boulder) Curt Tilmes (NASA Goddard Space Flight Center) Robert Wolfe * (NASA Goddard Space Flight Center) * Co-chair Working Group Participants: Nadine Alameh (MobiLaps LLC) Howard Burrows (Autonomous Undersea Systems Institute / National Science Digital Library) Yonsook Enloe (SGT Inc. / NASA Goddard Space Flight Center) Stefan Falke (Washington University in St. Louis) Michael Folk (National Center for Supercomputing Applications) Emily Greene (Raytheon Company) Tommy Jasmin (University of Wisconsin-Madison, Space Science and Engineering Center) Steve Olding (Everware / NASA Goddard Space Flight Center) Bill Teng (Science Systems and Applications, Inc. / NASA Goddard Space Flight Center) Fred Watson (California State University, Monterey Bay) Acknowledgements: Michael Chyatte (Computer Science Corporation / NASA Goddard Space Flight Center) – SourceMotel site running GForge Thomas Clune (NASA Goddard Space Flight Center) – SourceMotel site running GForge Kathy Fontaine (NASA Goddard Space Flight Center) – Earth Science Data System Working Group Coordinator Gene Major (Science Systems and Applications, Inc.
    [Show full text]