EGEE Kick-Off Meeting – Cork – 18-22/04/2004 Software www.eu-egee.org Configuration Management and Integration Alberto Di Meglio EGEE JRA1 Integration Manager EGEE is a project funded by the European Union under contract IST-2003-508833 Contents • JRA1 Integration Mandate • Team Objectives • The SCM Plan: an Overview • SCM and Integration Implementation • Configuration and Version Control • Build Systems • Release Process • Other Configuration and Change Control Procedures EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 2 JRA1 Integration Mandate • EGEE JRA1 has a dedicated team for SCM and Integration activities • The vision of the team is that SCM and Integration must start before a single line of code is written and must provide an integrated development and collaboration environment • The mission is to provide a solid, reliable infrastructure to manage changes and release quality software based on industry-standard SCM procedures and tools throughout the entire duration of the project EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 3 Integration Team Objectives • The Integration System must be simple, flexible and solid • Excessive diversity in tools, languages, technologies must be avoided • Centralization of processes and procedures is favoured • Whenever possible industry-standard tools and methods with a large user base and reliable, official support must be used • Making it work well on one platform is preferable to making it available on all platforms (at least initially) • Extension to multiple platforms and usage scenarios will naturally follow for a well designed, developed and integrated software • EGEE Middleware is a complex, multi-disciplinary, highly distributed project. This is part of the novelty and challenge of it and will require a novel approach to old and new problems and some courageous decisions EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 4 SCM Plan Overview • A master Software Configuration Plan is being finalized now [1] • It contains basic principles and rules about the various areas of SCM and Integration (version control, release management, build systems, bug tracking, etc) • Compliant with internationally agreed standards (ISO 10007-2003 E [2], IEEE SCM Guidelines series [3]) • Most EGEE stakeholders have already been involved in the process to make sure everybody is aware of, contributes to and uses the plan • An EGEE JRA1 Developers’s Guide will follow shortly in collaboration with JRA2 (Quality Assurance) based on the SCM Plan • It is of paramount importance to deliver the plan and guide as early as possible in the project lifetime EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 5 SCM and Integration Implementation • As soon as the SCM Plan is finalized, we will start the implementation phase (some steps already taken) • All tools, servers, procedures and communication channels will be put in place • Developer’s and User’s Guides will be produced with clear instructions about how to use and interact with the project and manage changes in the correct way • The complete infrastructure will be in place at the end of Month 3 (30 June) (MJRA1.1) • Goals for the first 12 months are to release incremental code baselines to test with a very rapid lifecycle (weekly) and finally release v. 1.0.0 (main milestone is MJRA1.5 at Month 10: integrated EGEE RC 1 released to testing) EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 6 Version and Configuration Control • Based on CVS using CERN Central IT CVS service • Fixed directory structure for each module • Rules for tagging and branching (e.g. bug fix branches) • Common naming conventions for baseline and release tags • Configuration files to automate creation of workspaces Used to enforce build reproducibility Used to create private workspaces for developers • Authorization based on commitavail and tagavail according to SCM roles (developers, release managers, integrators, etc.) • Supported protocols: K4, SSH (pserver only for anonymous read-only). A CERN AFS account is required EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 7 CVS Structure JRA1 EGEE CVS project org.egee src test org.egee.subsys doc config org.egee.subsys.comp1 interface org.egee.subsys.compN examples EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 8 Workspaces • A Workspace is one of the basic SCM Patterns [4] • It is a private area where a build can be produced under configuration control • More independent workspaces can coexist on the same computer for parallel development • The workspace is created using a Configuration Specification File (CSF) • The CSFs contain all the information necessary to prepare the workspace extracting modules and other dependencies from CVS or the central software repository • There is a CSF file for the overall org.egee Middleware suite and one CSF for each subsystem • CSFs are stored in the corresponding modules in CVS and versioned so that any version of the MW suite and all its subsystems can be reproduced at any time EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 9 Build System • Common Build System A project-wide build system based on Ant to allow all developers, integrators and testers to manage private builds on their own computers with the same tools and settings (compilers, flags, configurations, etc) Based on the concept of Private Workspaces, more than one independent workspace can be created on the same computer Uses a set of common build and properties files to define targets and properties for all project modules External dependencies are stored and versioned either in CVS or a central repository maintained by CERN SPI. All modules must use the SAME versions of the external dependencies across the project • Central Build System Located at CERN and maintained by the JRA1 Integration Team Based on the Common Build System, but automated by means of CruiseControl and Maven to generate a continuous, automated integration process and the overall project web site (Apache style). EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 10 Build System (2) • The Ant build system provides the common build orchestration for all modules, not only Java • Modules can be built independently and use Ant or Make as specified by project requirements, independently from the module implementation language • A set of common targets has been defined for all modules (environment checks, style checks, compilation, unit tests, distribution, etc) • The entire MW suite, individual subsystems and individual modules can be built with the same set of targets • Quality Assurance checks are built into the build process and are mandatory. Failure to pass a quality check (coding styles, unit testing, minimum test coverage, documentation, etc) fails the build EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 11 Build System (3) • Five main types of builds: Continuous Incremental Builds: every few hours, mail messages are sent to developers who break the build Nightly build: every night from clean snapshot Weekly builds: every week from clean project configurations Maintenance builds: used to fix software defects on existing releases Private builds: whenever needed on the private development computers EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 12 Build Tools • Ant: used for the general build management and all Java modules. The advantages/disadvantages of using it also for C/C++ and other languages modules are under investigation • Make + GNU Autotools: for C/C++ and other languages as necessary. Although this is the standard, familiar approach, it has a lot of drawbacks, especially for large, distributed projects (difficult to maintain and centralize, lots of external dependencies, poor internal dependency tracking, file based instead of task based, etc) • CruiseControl: used to automate the integration builds • Maven: used to automate the project web site, reports and documentation builds EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 13 Packaging and Configuration • Installation packages for the EGEE Middleware will be produced by the integration team • Packages will be provided for the entire middleware and for all subsystems and deployment modules • Supported packaging formats are: Source tarballs with build scripts Native packages for supported platforms (e.g. RPMs for Red Hat Linux Enterprise 3.0, MSIs for Windows XP, etc) • Extracting and using modules directly from CVS is not recommended, since there is no guarantee the dependencies are satisfied outside the build system context • Configuration issues are being investigated with SA1 • The goal is to have a simple, common format for all configuration items, usable by standard installation/configuration tools • Some type of “Middleware Configuration Front-End” may also be provided in collaboration with SA1 EGEE Kick-Off Meeting – Cork – 18-22/04/2004 - 14 Release Process 1. Developers works on code functionality and documentation and produce baseline candidates 2. Baseline candidates are continuously integrated and unit and smoke tested 3. A build is successful if all the functionality contained in the module is properly unit tested and documented. If a build is not successful, the baseline candidate is Functionality finished and unit rejected and control returns to the developers tested No 4. Baselines are automatically tagged if the build process is successful 5. When all baseline candidates in the application build list (Application CSF file) are successfully built, an application baseline is produced and passed to testing 6. If all test, certification and validation procedures are successful, the individual