DOCSLIB.ORG

Controlling Large Software Development . in a Distributed Environment

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Controlling Large Software Development . in a Distributed Environment Controlling Large Software Development . In a Distributed Environment Eric Emerson Schmidt Controlling Large Software Development in a Dist ri buted Envi ronment Eric Emerson Schmidt CSL-82-7 December 1982 [P82-00039] © Copyright Eric Emerson Schmidt 1982. All rights reserved. Xerox Corporation XEROX Palo Alto Research Centers 3333 Coyote Hill Road Palo Alto, California 94304 Controlling Large Software Development in a Distributed Environment Report number CSL-82-7. This report reproduces a dissertation submitted to the University of California. Berkeley in partial fulfillment of the degree requirements for the Doctor of Philosophy in Computer Science. The research in this dissertation was supported by Xerox Corporation. Abstract Breaking a program up into modules is an important technique for managing the complexity of large systems. As the number of modules increases, the modules themselves need to be managed. Changing even a single module can be difficult. Compilation and loading are complicated. Saving the state of a program for others to build on is quite error-prone. The development of a large program as part of a multi-person project is even worse. This thesis presents solutions to these problems. We use new languages to describe the modules that comprise a system and tools that automate software development. The first solution developed is a version control system of modest goals that has been used to maintain up to 450,000 lines of code over the past year. Users of this system list versions of files in description files (OF files) that are automatically maintained for the user. OF files may refer to other OF files when one software package depends on another. A working set of software that is saved in a safe location is called a release. The need for a release process was identified and an iterative algorithm that uses OF files to perform releases has been developed. Based on experience with the OF system and the desire to automate the entire compile-edit­ debug-release cycle, a second solution was developed in which the development cycle is controlled by the System M odeller. The modeller automatically manages the compilation, loading. and saving of new modules as they are produced. The user describes his software in a system model that lists the versions of files used. the information needed to compile the system. and the interconnections between the various modules. The modeller is connected to the editor and is notified when files are edited and new versions are created. To provide fast response. the modeller behaves like an incremental compiler: only those modules that change are analyzed and recompiled. CR categories: 0.1.1 [Programming Techniques]: Applicative (Functional) Programming; 0.2.2 [Software Engineering]: Tools and Techniques - Modules and Interfaces; 0.2.7 [Software Engineering]: Distribution and Maintenance - Version Control; 0.2.9 [Software Engineering]: Management - Software Configuration Management; 0.3.2 [Programming Languages]: Language Classifications - Applicative Languages. Key words and ph rases: Version control, release process, Cedar, Mesa, software management, automatic compilation, applicative language, modules, OF files, system models. TABLE OF CONTENTS Chapter 1 Page 1 Chapter 2 Page 15 Chapter 3 Page 49 Chapter 4 Page 75 Chapter 5 Page 103 References Page 107 Appendix A Page III Appendix B Page 115 Appendix C Page 125 Appendix D Page 129 Appendix E Page 135 Acknowledgments I would like to thank my committee, Robert S. Fabry and Richard J. Fateman of the Computer Science Department of V.C. Berkeley, James A. Reeds of the Statistics Department of Berkeley, and Butler W. Lampson of Xerox PARe's Computer Science Laboratory (esL) for their comments on this thesis. Fabry was my principal research adviser throughout my years at Berkeley; his encouragement and support were constant. Lampson is the intellectual father of the research described in this thesis. He coined the term "System Modelling" and designed the Cedar Kernel language, on which the SML language is based. Monthly meetings with him over the past three years gave me insight and encouragement in this research. Every idea in this thesis has been affected by his critical approach and incisive suggestions. Parts of section 3.4 are based on an internal memo by Lampson on the Cedar Kernel language. Roy Levin of eSL had a strong influence on this research. He proposed the use of OF files as part of the Cedar release process, served as Release Master for almost all Cedar releases, and participated in the design of the Release Tool. His thorough approach is responsible for the success of the release process. He has contributed several ideas to the design. presented here. of a release process based on system models. James H. Morris of eSL brought me into Xerox as a research intern five years ago. He remained my official sponsor at Xerox through another summer and later made me a member of the Cedar project. which he headed until this year. His strong personal influence through those years kept me directed toward a Ph.D.: he is both my friend and mentor. System Modelling has been strongly influenced by Edwin H. Satterthwaite of eSL, who has used the system modeller heavily and has played a major role in providing fast turnaround through module replacement. Satterthwaite has given me advice on the design of the SM language and its implementation, and has allowed me to reproduce the system model that he developed for the Cedar compiler as an appendix to the thesis. I would like to thank these fearless readers of earlier versions of my thesis: Mark Brown. Jim Donahue and Jim Horning of esL, Brian Lewis of Xerox SOD. David Elliott of SRI International, William N. Joy of Sun Microsystems, Inc. and Dan Halbert of V.C. Berkeley. Each made valuable structural and syntactic improvements to this thesis. Lewis took the OF software described in Chapter 2. converted it to run in SOD'S programming environment. and served as Release Master for their first release. Halbert carried many versions of this thesis between Palo Alto and Berkeley: his door-to-door service was invaluable. Every member of the Cedar project has used some of the software described here. I thank them all for their patience in helping to debug these programs, and for their many suggestions that helped turn the programs into a usable system. Thanks also go to Paul Rovner and Warren Teitelman of eSL for allowing me to reproduce some of their OF files in the appendices to the thesis. This research and the Cedar project would not have been possible without the support of CSL and particularly of its manager, Robert W. Taylor. The text of this thesis was prepared using the Bravo editor, and the figures were done using the SIL illustrator. Both of these programs were run on a Dorado. These three tools made preparation of this thesis much easier. I would like to thank my wife, Wendy, for editing this thesis as it was prepared and for supporting me through the good and bad times. This thesis is dedicated to the memory of my father, Dr. Wilson Schmidt, Professor of Economics, who died while this thesis was being prepared. His emphasis on the importance of graduate education and the personal value of a Ph.D. was the inspiration for my pursuit of a Ph.D. CHAPTER 1: INTRODUCTION 1 1. Introduction 1.1 Introduction This dissertation presents a solution for problems occurring when large numbers of components of a software system are developed and maintained by many different programmers. Tools and languages are described that automate production of new versions of the system and the integration of packages into a stable system. The solution is developed in the context of a project to build Cedar, a new programming environment at Xerox's Palo Alto Research Center. The Cedar project [Deutsch-Taft. 1980] is an attempt to take the Mesa language and build around it a programming environment based on ideas from Interlisp and Smalltalk, while retaining the strong type-checking properties of Mesa. Cedar makes it easy for programmers to share packages and collaborate on software development. Cedar programmers work in a distributed computing environment and have to be able to share each other's programs in various stages of development. In this setting, control of versions and file management is difficult because of the large number of files in Cedar and the requirement that versions of files must agree. The first solution developed manages versions of files based on description files (OF files) that have information about versions of files needed and their locations. OF files that describe packages of software are input to a release process. The release process checks the submitted OF files to see if the programs they describe are made from compatible versions of software, and. if so, copies the files to a safe location. The Release Tool performs these checks and copies the files; an interactive algorithm is used that can be repeated after errors in OF files are found and fixed. Use of the Release Tool allows the person making the release, who is called the Release Master. to release software with which he ·is not familiar. The OF system automates version control for the programmer. Based on experience with it. a second solution was developed that is a complete program management system. The user describes his software· in system models. which are complete descriptions of a software system. Similar to a blueprint or schematic, a model combines in one place 1) information about the versions of files needed and hints about their locations, 2) additional information needed to compile the system, and 3) information about interconnections between modules, such as which procedures are used and where they are defined.
Load more

Recommended publications
  • AVRDUDE a Program for Download/Uploading AVR Microcontroller flash and Eeprom
    AVRDUDE A program for download/uploading AVR microcontroller flash and eeprom. For AVRDUDE, Version 6.0rc1, 16 May 2013. by Brian S. Dean Send comments on AVRDUDE to [email protected]. Use http://savannah.nongnu.org/bugs/?group=avrdude to report bugs. Copyright c 2003,2005 Brian S. Dean Copyright c 2006 - 2008 J¨orgWunsch Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies. Permission is granted to copy and distribute modified versions of this manual under the con- ditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this manual into another lan- guage, under the above conditions for modified versions, except that this permission notice may be stated in a translation approved by the Free Software Foundation. i Table of Contents 1 Introduction............................... 1 1.1 History and Credits ......................................... 2 2 Command Line Options .................... 4 2.1 Option Descriptions ......................................... 4 2.2 Programmers accepting extended parameters ................. 15 2.3 Example Command Line Invocations ........................ 18 3 Terminal Mode Operation ................. 22 3.1 Terminal Mode Commands.................................. 22 3.2 Terminal Mode Examples ................................... 23 4 Configuration
    [Show full text]
  • AVRDUDE a Program for Download/Uploading AVR Microcontroller Flash and Eeprom
    AVRDUDE A program for download/uploading AVR microcontroller flash and eeprom. For AVRDUDE, Version 5.10, 19 January 2010. by Brian S. Dean Send comments on AVRDUDE to [email protected]. Use http://savannah.nongnu.org/bugs/?group=avrdude to report bugs. Copyright c 2003,2005 Brian S. Dean Copyright c 2006 - 2008 J¨orgWunsch Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies. Permission is granted to copy and distribute modified versions of this manual under the con- ditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this manual into another lan- guage, under the above conditions for modified versions, except that this permission notice may be stated in a translation approved by the Free Software Foundation. i Table of Contents 1 Introduction::::::::::::::::::::::::::::::::::::: 1 1.1 History and Credits :::::::::::::::::::::::::::::::::::::::::::: 2 2 Command Line Options :::::::::::::::::::::::: 3 2.1 Option Descriptions :::::::::::::::::::::::::::::::::::::::::::: 3 2.2 Programmers accepting extended parameters :::::::::::::::::: 13 2.3 Example Command Line Invocations :::::::::::::::::::::::::: 15 3 Terminal Mode Operation :::::::::::::::::::: 19 3.1 Terminal Mode Commands :::::::::::::::::::::::::::::::::::: 19 3.2 Terminal Mode Examples :::::::::::::::::::::::::::::::::::::
    [Show full text]
  • Remote Application Control of Equinox ISP Programmers Using the ISP-PRO Application Author: Date: Version Number: John Marriott 19Th June 2012 1.04
    Report No: AN109 Title: Remote Application Control of Equinox ISP Programmers using the ISP-PRO application Author: Date: Version Number: John Marriott 19th June 2012 1.04 All rights are reserved. Reproduction in whole or in part is prohibited without the prior written consent of the copyright owner. The information presented in this document does not form part of any quotation or contract, is believed to be accurate and reliable and may be changed without prior notice. No liability will be accepted by the publisher for any consequence of its use. Publication thereof does not convey nor imply any license under patent or other industrial or intellectual property rights Application Note – AN109 - ISP-PRO - Remote Application Control 1 Contents 1.0 Overview ............................................................................................................................ 3 1.1 Supported Programmers............................................................................................................ 3 1.2 Programmer Control Methodology ............................................................................................. 6 1.3 Why use an Interface Database ? .............................................................................................. 7 1.4 What is a Programming Script ? ................................................................................................ 7 2.0 Interfacing with ISP-PRO ................................................................................................... 9 2.1
    [Show full text]
  • Computing the Cost of Copyright */ Programmers Fight 4Look and Feel' Lawsuits
    l<L^r^- RICK FRIEDMAN-BLACK STAR Innovate don't litigate': Stallman and colleagues outside Lotus headquarters :••' •• ' .' TECHNOLOGY/' ;.:::'; : ' Computing the Cost of Copyright */ Programmers fight 4look and feel' lawsuits he Cambridge, Mass., protest was de-" Macs. Many software developers fear that cidedly different. These weren't the the Lotus win will make it harder to bring Tusual malcontents from Harvard or new products to market. So when protest­ Boston University; these were computer ers marched in front of Lotus's Cambridge programmers hewing picket signs. The en­ headquarters earlier this month, they emy: giant Lotus Development Corp., chanted with a sly reference to the hexa­ which is trying to protect the market lead decimal counting scheme that is a basic of its 1-2-3 software package by bringing tool of their trade: "look and feel" suits against look-alike 1-2-3A kick the lawsuits out the door competitors. Lotus had recently won its 5-6'7-8 innovate don't litigate first big victory against Paperback Soft­ 9-A-B-C interfaces should be free ware International, which has sold its VP D-E-F-0 look and feel has got to gol Planner for a fifth of 1-2-3's $495 list price. The group behind the protests is the. Copyright rules that worked fine for League for Programming Freedom, found­ books and paintings are now straining to ed by Richard Stallman, a software gum cover works from music videos to digital who recently got a $240,000 MacArthur audiotapes—and, increasingly, software. grant. Stallman says the suits stifle innova­ Paperback and other challengers believed tion, paving the way for Japan to take over that the interface that a program presents the industry.
    [Show full text]
  • Effective Distributed Pair Programming in Academic Environment Using Association Rule Based Approach International Journal of Re
    International Journal of Research ISSN NO:2236-6124 Effective Distributed Pair Programming in Academic Environment using Association Rule Based Approach 1 JMSV RaviKumar , 2 Dr.M. Babu Reddy 1 Research Scholar,PP.COMP.SCI.562, Department of Computer Science, Rayalaseema University, Kurnool . 2 HOD, Department of Computer Science, Krishna University, Machilipatnam, Krishna, Andhra Pradesh, India. ABSTRACT: A pair recommender system based on association rule mining approach is devised. In Data mining, Association Rule Mining (ARM) is a technique to discover frequent patterns, associations and correlations among item-sets in data repositories. Association among programmers is found using association rule mining which measures the pair compatibility between the programmers. ARM and Apriori algorithm to solve ARM problems was introduced by (Agarwal et al. 1993). Association rules are used in many areas like market basket analysis, social networks, stock market etc. Here we use association rules to discover compatibility between pairs. Pair compatibility is influenced by various parameters like skill level, technical competence, designation, experience, personality interests, time management, learning style and self esteem. Keywords: repositories, Apriori algorithm, correlations, associations, frequent patterns. 1 INTRODUCTION A database in which an association rule is to be found is viewed as a set of tuples. In market basket analysis a tuple could be {bread, butter, jam}, which is the list of items purchased by a customer. Association rules are discovered based on these tuples and it represents a set of items that are purchased together[12]. For example the association rule {bread} {butter,jam} means that whenever a customer purchases bread he/she will purchase both butter and jam in the same transaction.
    [Show full text]
  • Algorithmic and Programming Training Materials for Teachers
    Algorithmic ● Programming ● Didactics Algorithmic and Programming Training materials for Teachers MARIA CHRISTODOULOU ELŻBIETA SZCZYGIEŁ ŁUKASZ KŁAPA WOJCIECH KOLARZ This project has been funded with support from the European Commission. This publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. Algorithmic and Programming Training materials for Teachers MARIA CHRISTODOULOU ELŻBIETA SZCZYGIEŁ ŁUKASZ KŁAPA WOJCIECH KOLARZ Krosno, 2018 The Authors: Maria Christodoulou, Elżbieta Szczygieł, Łukasz Kłapa, Wojciech Kolarz Scientific reviewer: Marek Sobolewski PhD, Rzeszow University of Technology Publishing house: P.T.E.A. Wszechnica Sp. z o.o. ul. Rzeszowska 10, 38-404 Krosno Phone: +48 13 436 57 57 https://wszechnica.com/ Krosno, 2018 ISBN 978-83-951529-0-0 Creative Commons Attribution-ShareAlike 4.0 International Table of contents Instead of the introduction ................................................................................................... 5 1 Introduction to algorithmic .......................................................................................... 7 1.1 Computer Programs ................................................................................................... 8 1.2 Algorithms and their importance ............................................................................. 8 1.3 Algorithmic design ....................................................................................................
    [Show full text]
  • Downloads Distribution
    FLOSSSim: Understanding the Free/Libre Open Source Software (FLOSS) Development Process through Agent-Based Modeling by Nicholas Patrick Radtke A Dissertation Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy Approved October 2011 by the Graduate Supervisory Committee: James S. Collofello, Co-Chair Marco A. Janssen, Co-Chair Hessam S. Sarjoughian Hari Sundaram ARIZONA STATE UNIVERSITY December 2011 ABSTRACT Free/Libre Open Source Software (FLOSS) is the product of volunteers collaborating to build software in an open, public manner. The large number of FLOSS projects, combined with the data that is inherently archived with this online process, make studying this phe- nomenon attractive. Some FLOSS projects are very functional, well-known, and successful, such as Linux, the Apache Web Server, and Firefox. However, for every successful FLOSS project there are 100’s of projects that are unsuccessful. These projects fail to attract suf- ficient interest from developers and users and become inactive or abandoned before useful functionality is achieved. The goal of this research is to better understand the open source development process and gain insight into why some FLOSS projects succeed while others fail. This dissertation presents an agent-based model of the FLOSS development pro- cess. The model is built around the concept that projects must manage to attract contri- butions from a limited pool of participants in order to progress. In the model developer and user agents select from a landscape of competing FLOSS projects based on perceived utility. Via the selections that are made and subsequent contributions, some projects are propelled to success while others remain stagnant and inactive.
    [Show full text]
  • SIMD for C++ Developers Contents Introduction
    SIMD for C++ Developers Contents Introduction ............................................................................................................................... 2 Motivation ............................................................................................................................. 2 Scope ..................................................................................................................................... 2 History Tour ........................................................................................................................... 2 Introduction to SIMD ................................................................................................................. 3 Why Care? ............................................................................................................................. 5 Applications ........................................................................................................................... 5 Hardware Support ................................................................................................................. 6 Programming with SIMD ........................................................................................................... 6 Data Types ............................................................................................................................. 7 General Purpose Intrinsics .................................................................................................... 8 Casting Types ....................................................................................................................
    [Show full text]
  • Continuous Integration Empirical Results from 250+ Open Source and Proprietary Projects
    Characterizing The Influence of Continuous Integration Empirical Results from 250+ Open Source and Proprietary Projects Akond Rahman, Amritanshu Agrawal, Rahul Krishna, and Alexander Sobran* North Carolina State University, IBM Corporation* [aarahman,aagrawa8,rkrish11]@ncsu.edu,[email protected]* ABSTRACT 1 INTRODUCTION Continuous integration (CI) tools integrate code changes by auto- Continuous integration (CI) tools integrate code changes by auto- matically compiling, building, and executing test cases upon submis- matically compiling, building, and executing test cases upon sub- sion of code changes. Use of CI tools is getting increasingly popular, mission of code changes [9]. In recent years, usage of CI tools have yet how proprietary projects reap the benefits of CI remains un- become increasingly popular both for open source software (OSS) known. To investigate the influence of CI on software development, projects [3][12][25] as well as for proprietary projects [24]. we analyze 150 open source software (OSS) projects, and 123 propri- Our industrial partner adopted CI to improve their software de- etary projects. For OSS projects, we observe the expected benefits velopment process. Our industrial partner’s expectation was that after CI adoption, e.g., improvements in bug and issue resolution. similar to OSS projects [27][25], CI would positively influence However, for the proprietary projects, we cannot make similar ob- resolution of bugs and issues for projects owned by our industrial servations. Our findings indicate that only adoption of CI might partner. Our industrial partner also expected collaboration to in- not be enough to the improve software development process. CI crease upon adoption of CI.
    [Show full text]
  • Setting up AVR Development Environment
    Setting up AVR development environment Including support for Linux, MacOS and Windows Version 1.0 Annex Document A.0 Gerard Marull Paretas July 2007 Index 1. Introduction.............................................................................................................................3 2. AVR GNU Toolchain...............................................................................................................4 2.1. Introduction.......................................................................................................................4 2.1.1. GCC (GNU Compiler Collection).............................................................................4 2.1.2. Binutils.......................................................................................................................4 2.1.3. avr-libc.......................................................................................................................5 2.2. Installation of AVR GNU Toolchain.................................................................................5 2.2.1. Windows....................................................................................................................5 2.2.2. Linux / MacOS..........................................................................................................5 3. Programming tools..................................................................................................................8 3.1. Installing avrdude..............................................................................................................8
    [Show full text]
  • The Implementation of Time Management Solution
    Regis University ePublications at Regis University All Regis University Theses Spring 2006 The mpleI mentation of Time Management Solution Robert M. Katz Regis University Follow this and additional works at: https://epublications.regis.edu/theses Part of the Computer Sciences Commons Recommended Citation Katz, Robert M., "The mpI lementation of Time Management Solution" (2006). All Regis University Theses. 136. https://epublications.regis.edu/theses/136 This Thesis - Open Access is brought to you for free and open access by ePublications at Regis University. It has been accepted for inclusion in All Regis University Theses by an authorized administrator of ePublications at Regis University. For more information, please contact [email protected]. Regis University College for Professional Studies Graduate Programs Final Project/Thesis Disclaimer Use of the materials available in the Regis University Thesis Collection (“Collection”) is limited and restricted to those users who agree to comply with the following terms of use. Regis University reserves the right to deny access to the Collection to any person who violates these terms of use or who seeks to or does alter, avoid or supersede the functional conditions, restrictions and limitations of the Collection. The site may be used only for lawful purposes. The user is solely responsible for knowing and adhering to any and all applicable laws, rules, and regulations relating or pertaining to use of the Collection. All content in this Collection is owned by and subject to the exclusive control of Regis University and the authors of the materials. It is available only for research purposes and may not be used in violation of copyright laws or for unlawful purposes.
    [Show full text]
  • List of Programmers 1 List of Programmers
    List of programmers 1 List of programmers This list is incomplete. This is a list of programmers notable for their contributions to software, either as original author or architect, or for later additions. A • Michael Abrash - Popularized Mode X for DOS. This allows for faster video refresh and square pixels. • Scott Adams - one of earliest developers of CP/M and DOS games • Leonard Adleman - co-creator of RSA algorithm (the A in the name stands for Adleman), coined the term computer virus • Alfred Aho - co-creator of AWK (the A in the name stands for Aho), and main author of famous Dragon book • JJ Allaire - creator of ColdFusion Application Server, ColdFusion Markup Language • Paul Allen - Altair BASIC, Applesoft BASIC, co-founded Microsoft • Eric Allman - sendmail, syslog • Marc Andreessen - co-creator of Mosaic, co-founder of Netscape • Bill Atkinson - QuickDraw, HyperCard B • John Backus - FORTRAN, BNF • Richard Bartle - MUD, with Roy Trubshaw, creator of MUDs • Brian Behlendorf - Apache • Kent Beck - Created Extreme Programming and co-creator of JUnit • Donald Becker - Linux Ethernet drivers, Beowulf clustering • Doug Bell - Dungeon Master series of computer games • Fabrice Bellard - Creator of FFMPEG open codec library, QEMU virtualization tools • Tim Berners-Lee - inventor of World Wide Web • Daniel J. Bernstein - djbdns, qmail • Eric Bina - co-creator of Mosaic web browser • Marc Blank - co-creator of Zork • Joshua Bloch - core Java language designer, lead the Java collections framework project • Bert Bos - author of Argo web browser, co-author of Cascading Style Sheets • David Bradley - coder on the IBM PC project team who wrote the Control-Alt-Delete keyboard handler, embedded in all PC-compatible BIOSes • Andrew Braybrook - video games Paradroid and Uridium • Larry Breed - co-developer of APL\360 • Jack E.
    [Show full text]