DOCSLIB.ORG

Administration Syst`Eme

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Administration Syst`Eme Administration Syst emeÁ Remy Card ([email protected]) Fr ed eric Gilbert ([email protected]) - 3 janvier 2014 - Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 1 Bibliographie ± ºUnix System Administration Handbookº, 4th Ed, E. Nemeth, G. Snyder, S. Seebass, T. Hein, Prentice Hall, 2010 ± ºUnix Administrationº, 3 emeÁ Ed, J-M Moreno, Dunod, 2003 (en francËais) ± ºUnix Administrationº, J.-F. Bouchaudy, G. Goubet, Eyrolles, 2007 (en francËais) ± ºTCP/IP Network Administrationº, 2nd Ed, C. Hunt, O'Reilly, 1998 ± ºPractical Unix and Internet Securityº, 2nd Ed, S. Gar®nkel, G. Spafford, 1996 ± ºLe Guide du Rootardº, Eric Dumas, 1998, http : //www.linux-france.org/article/grl/Guide Rootard.html ± ºThe Network Administrator's Guideº, 2nd Ed., Olaf Kirch et Terry Dawson, 2000, http : //www.tldp.org/guides.html (The Linux Documentation Project, voir ce site pour d'autres r ef erences) ± Le support de ce cours : http : //www-roc.inria.fr/ ∼fgilbert/uvsq/poly.pdf Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 2 Th emesÁ abord es GEN ERALIT ES 4 DEMARRAGE D'UNIX 16 GESTION DES UTILISATEURS 35 LE SYST EMEÁ DE FICHIERS 48 LES SAUVEGARDES 96 LA JOURNALISATION (LES ªLOGSº) 121 LE NOYAU 138 LES PROCESSUS P ERIODIQUES 179 L'IMPRESSION 185 LA GESTION DES TERMINAUX 202 RESEAU ET SERVICES R ESEAU 222 SECURIT E ET CRYPTOGRAPHIE 317 Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 3 GEN ERALIT ES Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 4 Bref historique d'Unix ± Fin des ann ees 60 : Ken Thompson, Bell Labs, syst emeÁ personnel pour PDP-7 Denis Ritchie, syst emeÁ multi-utilisateur pour PDP-11 ± 1975-1980 : division en trois branches ± Bell Labs : groupe Unix Ajout du ®le system switch, des streams. Developpent 9 versions puis passent aÁ Plan 9. ± Berkeley University : Berkeley Software Distribution Ajout de la m emoire virtuelle (4.1), du r eseau (sockets, TCP-IP ; 4.2). ± Branche s epar ee chez Bell Labs : SYS III, puis SYS V, d etenteurs de la marque. Ajout de la m emoire partag ee, des IPC. ± Nombreuses versions selon constructeurs : ± Sun : BSD 4.2 + compatibilit e et utilitaires SYS V (SunOS), puis SYS V (Solaris); ªOracle Solarisº depuis le rachat de Sun en 2010. ± HP (HP-UX), IBM (AIX), Silicon Graphics (Irix), . ± Sur PC : SCO (SYS V commercial); FreeBSD, NetBSD; Minix puis Linux (POSIX). Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 5 Syst emesÁ etudi es ± Unix System V ± HP-UX ± Solaris 2 ± Unix BSD ± SunOS (Solaris 1) ± FreeBSD ± Linux Diff erentes distributions : RedHat et ses d eriv es (Fedora, RedHat Enterprise, CentOS, Mandriva, ...), Debian et ses d eriv es (Ubuntu, Xandros, . .), Suse, . Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 6 Roleà d'un syst emeÁ d'exploitation ± Machine virtuelle ± abstractions de haut niveau ± plus facile aÁ manipuler que la machine physique ± Gestion des ressources mat erielles (processeur, m emoire, p eriph eriques, ...) ± Partage de ressources ± Contr olesà Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 7 Notions de base sur Unix ± Multi-t achesà multi-utilisateurs ± Un utilisateur a des droits restreints. ± root a tous les droits. ± Processus : chaque t acheà a un environnement, un espace m emoire avec un adressage qui lui sont propres . Un s equenceur g ereÁ la transition et la priorit e entre les processus. ± M emoire virtuelle ± L'espace d'adresse d'un processus est ind ependant de la m emoire physique. ± Quand il n'y a plus de ªpagesº libres on utilise le swap . ± Pages effectivement utilis ees <= mem. physique + swap ± M ecanismes simples et puissants ± en particulier pour la gestion des entr ees/sorties et des processus. ± Interface standard : POSIX ± Un grand nombre de logiciels libres disponibles. Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 8 Tachesà de l'administrateur ± Ajout et suppression d'utilisateurs ± Ajout et suppression de mat eriel, recon®guration ± Sauvegardes et restaurations ± Installation de logiciels ± Surveillance du syst emeÁ ± s ecurit e ± monitoring ± Gestion de la documentation locale ± Aide aux utilisateurs Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 9 Principes de base de l'administration ± Tout syst emeÁ n ecessite un administrateur ± Complexit e de l'administration accrue par l'informatique omnipr esente : ± de plus de plus de machines ± syst emesÁ h et erog enesÁ ± nomadisme ± Pas de modi®cation du syst emeÁ lui-m emeà ± modi®cation de ®chiers de con®guration ± lancement de services ± Pas de recon®guration ± le vendredi soir ou avant de partir en cong es ± apr esÁ un ºpotº Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 10 Le compte 'root' ± Unix surveille le comportement de tous les utilisateurs ... ± ... sauf 'root' qui a tous les droits ± Connexion sous 'root' : ± connexion normale (login), d econseill ee ± su ± sudo , calife , ... Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 11 Commandes d'administration ± Les commandes pour les utilisateurs sont contenues dans les r epertoires /bin et /usr/bin ± Il existe des r epertoires sp eci®ques pour les commandes d'administration : ± /sbin ± /usr/sbin ± /etc ± /usr/etc ± Les ®chiers de con®guration du syst emeÁ sont g en eralement plac es dans le r epertoire /etc Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 12 Arborescence Unix ± Les r epertoires classiques sous Unix sont : ± bin , sbin : binaires et binaires syst emeÁ ± lib : les librairies, partag ees ou non ± include : les ent etesà d ecrivant la librairie C ± etc : ®chiers syst emeÁ ± share : resources ind ependantes de l'architecture ± man (ou share/man ) : le manuel ± var : les donn ees ªvariablesº ± tmp : les ®chiers (en principe) temporaires ± Ces r epertoires peuvent se retrouver aÁ plusieurs niveaux du syst emeÁ de ®chier : ± / : racine du syst eme,Á tout ce qui y est directement li e ± /usr : tout ce qui est li e aÁ l'utilisation du syst emeÁ par les utilisateurs ± /usr/local : tout ce qui ªlocalº aÁ un site ± /usr/local/samba : tout ce qui est li e aÁ un produit logiciel donn e ± ... Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 13 Les processus ± Leur structure : Structures en m emoire identi® ee par un num ero unique, le PID Arborescence, partant du processus init , de PID 1 (commande pstree ). Autres informations : PPID , propri etaire, commande, r epertoire courant, priorit e,. . ± Les voir : la commande ps , typiquement ps auxww USER PID VSZ RSS TTY STAT START TIME COMMAND ......... root 1273 1376 424 ? S Apr26 0:00 /usr/sbin/apmd-p 10 -w 5 -W -P /etc/sysconfig/apm-scripts/apmscript root 1299 1632 1632 ? SL Apr26 0:00 xntpd -A -c /etc/ntp/ntp.conf root 1310 2076 160 ? S Apr26 0:00 xinetd -stayalive -pidfile /var/run/xinetd.pid root 1371 1424 52 ? S Apr26 0:00 gpm -t imps2 -m /dev/mouse root 1389 1616 484? S Apr26 0:01crond ......... ± Les g erer : envoi d'un signal avec la commande kill kill -STOP pid kill -CONT pid kill -HUP pid kill -KILL pid Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 14 La Documentation ± Le manuel : man ! ± man commande ± man man ± man 1 kill : dans la section 1 (commandes utilisateur) ± man 2 kill : dans la section 2 (appels syst eme)Á ± man -k kill : -k comme keyword, a.k.a. apropos Plus sp eci®que Linux : ± Documents dans /usr/doc , /usr/share/doc ± Sites Web : http : //www.lea-linux.org/ ± Google! (les messages d'erreur par exemple) Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 15 DEMARRAGE D'UNIX Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 16 Demarrage d'Unix ± Ex ecution d'un chargeur primaire ± Chargement et ex ecution d'un chargeur secondaire ± Chargement du noyau ± Ex ecution du noyau ± d etection et initialisation du mat eriel ± lancement des processus syst emeÁ ± swapper (0), pagedaemon (2) sous BSD ± sched (0) sous System V ± Ex ecution du processus init (processus num ero 1) Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 17 Roleà de init ± Ex ecution des scripts d'initialisation ± Gestion des connexions sur terminaux ± Anc etreÁ de tous les processus ± Adoption des processus orphelins ± Deux types de programmes init : ± BSD ± System V Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 18 init BSD ± Deux niveaux d'ex ecution : ± mono-utilisateur ± multi-utilisateurs ± Lors du d emarrage, ex ecution de : ± /etc/rc.boot (SunOS) ± /etc/rc ± /etc/rc.single (SunOS en mode mono-utilisateur) ± /etc/rc.local ± Gestion des connexions sur terminaux : ± /etc/ttys ± /etc/ttytab Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 19 init System V ± Con®guration dans /etc/inittab : label : niveaux : action : commande ± Action : ± respawn : relanc e par init apr esÁ terminaison ± wait : lancement et attente ± once : lancement une seule fois ± boot : lancement lors de la premi ereÁ lecture de /etc/inittab ± bootwait : idem et attente ± off ± initdefault : niveau d'ex ecution par d efaut ± sysinit : lancement avant l'acc esÁ aÁ la console ± powerfail : coupure d'alimentation Administration Syst emeÁ - R emy Card (UVSQ), Fr ed eric Gilbert (INRIA) 20 Niveaux de init
Load more

Recommended publications
  • Guide Avancé D'écriture Des Scripts Bash
    Guide avancé d'écriture des scripts Bash Une exploration en profondeur de l'art de la pro- grammation shell Mendel Cooper Guide avancé d'écriture des scripts Bash: Une exploration en profondeur de l'art de la programmation shell Mendel Cooper 5.3 Publié le 11 mai 2008 Résumé Ce tutoriel ne suppose aucune connaissance de la programmation de scripts, mais permet une progression rapide vers un niveau in- termédiaire/avancé d'instructions tout en se plongeant dans de petites astuces du royaume d'UNIX®. Il est utile comme livre, comme manuel permettant d'étudier seul, et comme référence et source de connaissance sur les techniques de programmation de scripts. Les exercices et les exemples grandement commentés invitent à une participation active du lecteur avec en tête l'idée que la seule façon pour vraiment apprendre la programmation de scripts est d'écrire des scripts. Ce livre est adapté à une utilisation en classe en tant qu'introduction générale aux concepts de la programmation. La dernière mise à jour de ce document, comme une « archive tar » compressée avec bzip2 incluant à la fois le source SGML et le HTML généré, peut être téléchargée à partir du site personnel de l'auteur. Une version PDF est aussi disponible (site miroir du PDF). Voir le journal des modifications pour un historique des révisions. Dédicace Pour Anita, la source de toute magie i Part 1. Introduction .......................................................................................................................................... 1 1. Pourquoi la programmation
    [Show full text]
  • Sustaining the Montage Image Mosaic Engine Since 2002
    PROCEEDINGS OF SPIE SPIEDigitalLibrary.org/conference-proceedings-of-spie 6XVWDLQLQJWKH0RQWDJHLPDJH PRVDLFHQJLQHVLQFH *%UXFH%HUULPDQ-RKQ&*RRG *%UXFH%HUULPDQ-RKQ&*RRG6XVWDLQLQJWKH0RQWDJHLPDJHPRVDLF HQJLQHVLQFH3URF63,(6RIWZDUHDQG&\EHULQIUDVWUXFWXUHIRU $VWURQRP\9 -XO\ GRL (YHQW63,($VWURQRPLFDO7HOHVFRSHV,QVWUXPHQWDWLRQ$XVWLQ7H[DV 8QLWHG6WDWHV 'RZQORDGHG)URPKWWSVZZZVSLHGLJLWDOOLEUDU\RUJFRQIHUHQFHSURFHHGLQJVRIVSLHRQ7HUPVRI8VHKWWSVZZZVSLHGLJLWDOOLEUDU\RUJWHUPVRIXVH Sustaining the Montage Image Mosaic Engine Since 2002 G. Bruce Berriman*a, John C. Gooda a Caltech/IPAC-NExScI, 1201 East California Blvd., Pasadena, CA 91125, USA ABSTRACT This paper describes how we have sustained the Montage image mosaic engine (http://montage.ipac.caltech.edu) first released in 2002, to support the ever-growing scale and complexity of modern data sets. The key to its longevity has been its design as a toolkit written in ANSI-C, with each tool performing one distinct task, for easy integration into scripts, pipelines and workflows. The same code base now supports Windows, JavaScript and Python by taking advantage of recent advances in compilers. The design has led to applicability of Montage far beyond what was anticipated when Montage was first built, such as supporting observation planning for the JWST. Moreover, Montage is highly scalable and is in wide use within the IT community to develop advanced, fault-tolerant cyber-infrastructure, such as job schedulers for grids, workflow orchestration, and restructuring techniques for processing complex workflows and pipelines. Keywords: image processing, software toolkits, software engineering, software sustainability, image mosaics, astronomy imaging. 1. INTRODUCTION The Montage image mosaic engine is entering its sixteenth year of support to the astrophysics and IT communities. The mosaics created by Montage are intended for scientific analysis: they preserve the calibration and astrometric fidelity of the original images, and rectify the highly variable sky background to a common level across all the original images1.
    [Show full text]
  • HPSEC-05.Pdf
    A COMPARISON OF TWO METHODS FOR BUILDING ASTRONOMICAL IMAGE MOSAICS ON A GRID∗ DANIEL S. KATZ,† G. BRUCE BERRIMAN,‡ EWA DEELMAN,§ JOHN GOOD,‡ JOSEPH C. JACOB,† CARL KESSELMAN,§ ANASTASIA C. LAITY,‡ THOMAS A. PRINCE,¶ GURMEET SINGH,§ MEI-HUI SU§ Abstract. This paper compares two methods for running an application composed of a set of modules on a grid. The set of modules generates large astronomical image mosaics by composing multiple small images. This software suite is called Montage (http://montage.ipac.caltech.edu/). The workflow that describes a particular run of Montage can be expressed as a directed acyclic graph (DAG), or as a short sequence of parallel (MPI) and sequential programs. In the first case, Pegasus can be used to run the workflow. In the second case, a short shell script that calls each program can be run. In this paper, we discuss the Montage modules, the workflow run for a sample job, and the two methods of actually running the workflow. We examine the run time for each method and compare the portions that differ between the two methods. Key words. grid applications, performance, astronomy, mosaics Technical Areas. Network-Based/Grid Computing, Cluster Computing, Algorithms & Applications 1. Introduction. Many science data processing applications can be expressed as a sequence of tasks to be performed. In astronomy, one such application is building science-grade mosaics from multiple image data sets as if they were single images with a common coordinate system, map projection, etc. Here, the software must preserve the astrometric and photometric integrity of the original data, and rectify background emission from the sky or from the instrument using physically based models.
    [Show full text]
  • Astronomical Image Mosaicking on a Grid: Initial Experiences∗ Daniel S
    ASTRONOMICAL IMAGE MOSAICKING ON A GRID: INITIAL EXPERIENCES∗ DANIEL S. KATZ,† NATHANIEL ANAGNOSTOU,‡ G. BRUCE BERRIMAN,‡ EWA DEELMAN,§ JOHN GOOD,‡ JOSEPH C. JACOB,† CARL KESSELMAN,§ ANASTASIA LAITY,‡ THOMAS A. PRINCE,¶ GURMEET SINGH,§ MEI-HUI SU,§ ROY WILLIAMSk Abstract. This chapter discusses some grid experiences in solving the problem of generat- ing large astronomical image mosaics by composing multiple small images, from the team that has developed Montage (http://montage.ipac.caltech.edu/). The problem of generating these mosaics is complex in that individual images must be projected into a common coordinate space, overlaps between images calculated, the images processed so that the backgrounds match, and images com- posed while using a variety of techniques to handle the presence of multiple pixels in the same output space. To accomplish these tasks, a suite of software tools called Montage has been developed. The modules in this suite can be run on a single processor computer using a simple shell script, and can additionally be run using a combination of parallel approaches. These include running MPI versions of some modules, and using standard grid tools. In the latter case, processing workflows are automatically generated, and appropriate data sources are located and transferred to a variety of parallel processing environments for execution. As a result, it is now possible to generate large-scale mosaics on-demand in timescales that support iterative, scientific exploration. In this chapter, we describe Montage, how it was modified to execute in the grid environment, the tools that were used to support its execution, as well as performance results. Key words.
    [Show full text]
  • Administration D'oracle Solaris : Systèmes De Fichiers ZFS • Février 2012 Table Des Matières
    Administration d'Oracle® Solaris : Systèmes de fichiers ZFS Référence : E25822–03 Février 2012 Copyright © 2006, 2012, Oracle et/ou ses affiliés. Tous droits réservés. Ce logiciel et la documentation qui l'accompagne sont protégés par les lois sur la propriété intellectuelle. Ils sont concédés sous licence et soumis à des restrictions d'utilisation et de divulgation. Sauf disposition de votre contrat de licence ou de la loi, vous ne pouvez pas copier, reproduire, traduire, diffuser, modifier, breveter, transmettre, distribuer, exposer, exécuter, publier ou afficher le logiciel, même partiellement, sous quelque forme et par quelque procédé quecesoit. Par ailleurs, il est interdit de procéder à toute ingénierie inverse du logiciel, de le désassembler ou de le décompiler, excepté à des fins d'interopérabilité avec des logiciels tiers ou tel que prescrit par la loi. Les informations fournies dans ce document sont susceptibles de modification sans préavis. Par ailleurs, Oracle Corporation ne garantit pas qu'elles soient exemptes d'erreurs et vous invite, le cas échéant, à lui en faire part par écrit. Si ce logiciel, ou la documentation qui l'accompagne, est concédé sous licence au Gouvernement des Etats-Unis, ou à toute entité qui délivre la licence de ce logiciel ou l'utilise pour le compte du Gouvernement des Etats-Unis, la notice suivante s'applique : U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations.
    [Show full text]
  • Creating Updated, Scientifically-Calibrated Mosaic Images for the Rc3 Catalogue
    Draft version June 11, 2018 Preprint typeset using LATEX style emulateapj v. 01/23/15 CREATING UPDATED, SCIENTIFICALLY-CALIBRATED MOSAIC IMAGES FOR THE RC3 CATALOGUE Jung Lin Lee1, Robert J. Brunner2,3,4,5, Draft version June 11, 2018 ABSTRACT The Third Reference Catalogue of Bright Galaxies (RC3) is a reasonably complete listing of 23,011 nearby, large, bright galaxies. By using the final imaging data release from the Sloan Digital Sky Survey, we generate scientifically-calibrated FITS mosaics by using the montage program for all SDSS imaging bands for all RC3 galaxies that lie within the survey footprint. We further combine the SDSS g, r, and i band FITS mosaics for these galaxies to create color-composite images by using the STIFF program. We generalized this software framework to make FITS mosaics and color-composite images for an arbitrary catalog and imaging data set. Due to positional inaccuracies inherent in the RC3 catalog, we employ a recursive algorithm in our mosaicking pipeline that first determines the correct location for each galaxy, and subsequently applies the mosaicking procedure. As an additional test of this new software pipeline and to obtain mosaic images of a larger sample of RC3 galaxies, we also applied this pipeline to photographic data taken by the Second Palomar Observatory Sky Survey with BJ , RF , and IN plates. We publicly release all generated data, accessible via a web search form, and the software pipeline to enable others to make galaxy mosaics by using other catalogs or surveys. Subject headings: techniques: image processing { astronomical databases: catalogs { astrometry 1.
    [Show full text]
  • The Linux Cookbook: Tips and Techniques for Everyday Use: the Linux Cookbook: Tips and Techniques for Everyday Use
    The Linux Cookbook: Tips and Techniques for Everyday Use: The Linux Cookbook: Tips and Techniques for Everyday Use: Table of Contents The Linux Cookbook: Tips and Techniques for Everyday Use.....................................................................1 Preface ................................................................................................................................................................3 1.0 Format of Recipes ............................................................................................................................4 1.1 Assumptions, Scope, and Exclusions ..............................................................................................5 1.2 Typographical Conventions .............................................................................................................6 1.3 Versions, Latest Edition, and Errata ................................................................................................7 1.4 Acknowledgments ............................................................................................................................8 PART ONE: Working with Linux .................................................................................................................10 2. Introduction .................................................................................................................................................11 2.1 Background and History ................................................................................................................11
    [Show full text]
  • GNU/Linux Command-Line Tools Summary
    GNU/Linux Command−Line Tools Summary Gareth Anderson <somecsstudent(at)gmail.com> Chris Karakas − Conversion from LyX to DocBook SGML, Index generation Revision History Revision 1.2 15th April 2006 Revised by: GA Corrected typing errors, generated new, much smaller index (more accurate in my opinion). Updated errors in document for TLDP. Revision 1.1 28th February 2006 Revised by: CK Corrected typos, generated new index (9000 index entries!). Revision 1.0 6th February 2006 Revised by: GA Major restructuring, now in a docbook book format. Removed large chunks of content and revised other parts (removed chapters and sectioned some areas more). This is likely the final release by the author, I hope that someone finds this guide useful as I do not intend to continue work on this guide. Revision 0.7.1 25th February 2005 Revised by: CK Set special characters in math mode, produced PDF and PS with Computer Modern fonts in OT1 encoding and created correct SGML for key combinations. Revision 0.7 5th December 2004 Revised by: GA Updated document with new grammatical review. Re−ordered the entire Text section. Removed a fair amount of content. Revision v0.6 20th April 2004 Revised by: GA Attempted to fix document according to TLDP criticisms. Added notes and tips more sectioning. Now complying to the open group standards for the UNIX system trademark. Document should be ready for TLDP site. Revision v0.5 6th October 2003 Revised by: GA Fixed a variety of errors as according to the review and made some consistency improvements to the document.
    [Show full text]
  • Sustaining the Montage Image Mosaic Engine Since 2002
    Sustaining the Montage Image Mosaic Engine Since 2002 G. Bruce Berriman*a, John C. Gooda a Caltech/IPAC-NExScI, 1201 East California Blvd., Pasadena, CA 91125, USA ABSTRACT This paper describes how we have sustained the Montage image mosaic engine (http://montage.ipac.caltech.edu) first released in 2002, to support the ever-growing scale and complexity of modern data sets. The key to its longevity has been its design as a toolkit written in ANSI-C, with each tool performing one distinct task, for easy integration into scripts, pipelines and workflows. The same code base now supports Windows, JavaScript and Python by taking advantage of recent advances in compilers. The design has led to applicability of Montage far beyond what was anticipated when Montage was first built, such as supporting observation planning for the JWST. Moreover, Montage is highly scalable and is in wide use within the IT community to develop advanced, fault-tolerant cyber-infrastructure, such as job schedulers for grids, workflow orchestration, and restructuring techniques for processing complex workflows and pipelines. Keywords: image processing, software toolkits, software engineering, software sustainability, image mosaics, astronomy imaging. 1. INTRODUCTION The Montage image mosaic engine is entering its sixteenth year of support to the astrophysics and IT communities. The mosaics created by Montage are intended for scientific analysis: they preserve the calibration and astrometric fidelity of the original images, and rectify the highly variable sky background to a common level across all the original images1. Montage has found applicability far beyond its original goal of enabling scientific analysis of images acquired in wide- area astronomy surveys.
    [Show full text]
  • Linux Fundamentals (Pdf)
    Linux Fundamentals A Training Manual Philip Carinhas, Ph.D. Fortuitous Technologies www.fortuitous.com [email protected] Version of August 26, 2001 Copyright 2000-2001 Fortuitous Technologies, Inc. Fortuitous Technologies Inc. 6909A Hardy Dr. Austin, Tx 78757 USA WWW: http://fortuitous.com E-mail: [email protected] This training manual is a free book; you may reproduce and/or modify it under the terms of version 2 of the GNU General Public License as published by the Free Software Foun- dation whose website is located at http://www.gnu.org/copyleft/gpl.html. This book is distributed in the hope it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. The author encourages wide distribution of this book for personal and commercial use, provided the above copyright notice remains intact and the method adheres to the provi- sions of the GNU General Public License located at http://www.gnu.org/copyleft/gpl.html In summary, you may copy and distribute this book free of charge or for a profit. No ex- plicit permission is required from the author for reproduction of this book in any medium, physical or electronic. Note, derivative works and translations of this document must be placed under the GNU General Public License, and the original copyright notice must remain intact. If you have contributed new material to this book, you must make the source code (e.g., LATEX source) available for your revisions. Contents 1 Introduction to Linux 5 1.1 Linux Features . 5 1.2 Multi-User Operation .
    [Show full text]
  • Software Design Specification for MONTAGE an Astronomical
    Software Design Specification for MONTAGE An Astronomical Image Mosaic Service for the National Virtual Observatory Version 2.1 (August 29, 2004): Detailed Design This Document is based on the template CxTemp_SoftwareDesignSpecification.doc (Draft X; August 9, 2002), released by Construx Software (www.construx.com) 1 MONTAGE DESIGN REVISION HISTORY Description of Revision Date Add MPI design, MPI flowcharts, revise existing flowcharts. August 29, 2004 Move API & return codes to web page; update delivery dates and version numbers; add design for fast algorithm and new co- addition algorithm. – Version 2.1 Detailed Design: revised to reflect architecture for Montage on January 13, 2004 the TeraGrid – Version 2.0 Detailed Design: add algorithm description, flow charts, November 18, 2002 interface specification, error handling methodology, and all success and error return codes; revised block diagrams of Montage design & process flow – Version 1.1 Initial Design – Version 1.0 August 9, 2002 2 Table of Contents 1. Introduction ................................................................................................................ 5 1.1 Purpose of this Document....................................................................................... 5 1.2 Schedule for Delivery of Software Design Specification for Montage .................. 5 1.3 Supporting Materials............................................................................................... 6 2. Design Considerations ..............................................................................................
    [Show full text]
  • Using Machine Learning for Resource Provisioning to Run Workflow Applications in Iaas Cloud
    DEGREE PROJECT IN COMPUTER SCIENCE AND ENGINEERING, SECOND CYCLE, 30 CREDITS STOCKHOLM, SWEDEN 2019 Using machine learning for resource provisioning to run workflow applications in IaaS Cloud WILLIAM VÅGE KTH ROYAL INSTITUTE OF TECHNOLOGY SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE Using machine learning for resource provisioning to run workflow applications in IaaS Cloud WILLIAM VÅGE Master in Computer Science Date: December 3, 2019 Supervisor: Hamid Reza Faragardi Examiner: Elena Troubitsyna School of Electrical Engineering and Computer Science Host company: KTH Swedish title: Att använda maskininlärning till resursförmedling för att köra arbetsflödesapplikationer i molnet iii Abstract The rapid advancements of cloud computing has made it possible to execute large computations such as scientific workflow applications faster than ever be- fore. Executing workflow applications in cloud computing consists of choos- ing instances (resource provisioning) and then scheduling (resource schedul- ing) the tasks to execute on the chosen instances. Due to the fact that finding the fastest execution time (makespan) of a scientific workflow within a spec- ified budget is a NP-hard problem, it is common to use heuristics or meta- heuristics to solve the problem. This thesis investigates the possibility of using machine learning as an al- ternative way of finding resource provisioning solutions for the problem of scientific workflow execution in the cloud. To investigate this, it is evaluated if a trained machine learning model can predict provisioning instances with solution quality close to that of a state-of-the-art algorithm (PACSA) but in a significantly shorter time. The machine learning models are trained for the scientific workflows Cybershake and Montage using workflow properties as features and solution instances given by the PACSA algorithm as labels.
    [Show full text]