Generic Software Distribution and Deployment Master
Total Page:16
File Type:pdf, Size:1020Kb
View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by NORA - Norwegian Open Research Archives UNIVERSITY OF OSLO Department of Informatics Generic Software Distribution and Deployment Master thesis Arve Knudsen 31st January 2006 Acknowledgments My family and my supervisors Hans Petter Langtangen and Halvard Moe, thank you for keeping your faith in me.A special thanks to Professor Langtangen for providing constant encouragement and invaluable input in this difficult process. Contents Acknowledgments .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. iii Chapter 1. Introduction .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 1 1.1. A specific problem scenario .. .. .. .. .. .. .. .. .. .. .. .. .. 1 1.1.1. Manual deployment .. .. .. .. .. .. .. .. .. .. .. .. 2 1.1.2. Deployment solutions at our disposal .. .. .. .. .. .. .. 4 1.1.3. Applying PyPkg to the problem .. .. .. .. .. .. .. .. .. 5 1.2. Implementation notes .. .. .. .. .. .. .. .. .. .. .. .. .. .. 6 1.3. Thesis structure .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 6 Chapter 2. Vision .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 9 2.1. The current situation .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 9 2.1.1. Distribution .. .. .. .. .. .. .. .. .. .. .. .. .. .. 9 2.1.2. Deployment .. .. .. .. .. .. .. .. .. .. .. .. .. .. 10 2.1.3. Software directories .. .. .. .. .. .. .. .. .. .. .. .. 11 2.2. The PyPkg vision .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 11 2.2.1. The conduit concept .. .. .. .. .. .. .. .. .. .. .. .. 12 2.2.2. Precursors to the conduit concept .. .. .. .. .. .. .. .. 13 Chapter 3. Model .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 15 3.1. Considerations for an automated software transfer process .. .. .. .. 16 3.1.1. Involved problems .. .. .. .. .. .. .. .. .. .. .. .. .. 16 3.1.2. Achieving a generalized conduit .. .. .. .. .. .. .. .. .. 17 3.2. Distribution .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 18 3.2.1. Some words on the use of RDF .. .. .. .. .. .. .. .. .. 20 3.2.2. The PyPkg RDF model .. .. .. .. .. .. .. .. .. .. .. 22 3.2.3. The PyPkg project portal .. .. .. .. .. .. .. .. .. .. .. 26 3.3. Deployment .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 27 3.3.1. The agent concept .. .. .. .. .. .. .. .. .. .. .. .. .. 28 3.3.2. The design .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 29 3.4. The dependency scheme .. .. .. .. .. .. .. .. .. .. .. .. .. .. 34 3.4.1. Rationale .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 35 3.4.2. Semantics of the interface concept .. .. .. .. .. .. .. .. 36 Chapter 4. Usage .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 39 4.1. Distributing SciPy .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 39 4.1.1. Publishing the project .. .. .. .. .. .. .. .. .. .. .. .. 39 4.1.2. Adding a PyPkg package .. .. .. .. .. .. .. .. .. .. .. 40 4.1.3. Creating the installation script .. .. .. .. .. .. .. .. .. 44 4.2. Installing SciPy .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 46 4.2.1. Starting the installation .. .. .. .. .. .. .. .. .. .. .. 46 4.2.2. Calculating dependencies .. .. .. .. .. .. .. .. .. .. .. 46 4.2.3. Acquiring and installing packages .. .. .. .. .. .. .. .. 47 4.2.4. Finishing the installation .. .. .. .. .. .. .. .. .. .. .. 47 Chapter 5. Conclusion - Future Directions .. .. .. .. .. .. .. .. .. .. .. 49 5.1. Evaluation .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 49 5.1.1. Deployment .. .. .. .. .. .. .. .. .. .. .. .. .. .. 49 5.1.2. Distribution .. .. .. .. .. .. .. .. .. .. .. .. .. .. 50 5.1.3. Summing up .. .. .. .. .. .. .. .. .. .. .. .. .. .. 50 5.2. Future directions .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 51 5.2.1. Solidifying base functionality .. .. .. .. .. .. .. .. .. 51 5.2.2. Extending the PyPkg system .. .. .. .. .. .. .. .. .. .. 52 Appendix A. PyPkg RDF vocabulary .. .. .. .. .. .. .. .. .. .. .. .. .. 55 A.1. sw .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 55 A.2. pypkg .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 57 A.3. swidx .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 57 Appendix B. PyPkg Framework Design Document .. .. .. .. .. .. .. .. .. 59 B.1. Introduction .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 59 B.2. Installation .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 62 B.3. Installation.DependencyHandling .. .. .. .. .. .. .. .. .. .. .. 64 B.4. Installation.PackageFormat .. .. .. .. .. .. .. .. .. .. .. .. .. 64 B.5. Installation.ParallelDownload .. .. .. .. .. .. .. .. .. .. .. .. 65 B.6. Installation.Profile .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 65 B.7. Installation.Environment .. .. .. .. .. .. .. .. .. .. .. .. .. 66 B.8. Uninstallation .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 67 B.9. Uninstallation.DependencyHandling .. .. .. .. .. .. .. .. .. .. 68 B.10. Uninstallation.Environment .. .. .. .. .. .. .. .. .. .. .. .. 68 References .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 71 vi Glossary .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 75 vii List of Figures 2.1. The software conduit .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 13 3.1. From vision to model .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 15 3.2. The PyPkg distribution model .. .. .. .. .. .. .. .. .. .. .. .. .. .. 18 3.3. Graph representation of example RDF triple .. .. .. .. .. .. .. .. .. .. 21 3.4. Example RDF triple in XML syntax .. .. .. .. .. .. .. .. .. .. .. .. 22 3.5. DOAP class hierarchy (simplified) .. .. .. .. .. .. .. .. .. .. .. .. .. 23 3.6. PyPkg RDF model (DOAP extension) .. .. .. .. .. .. .. .. .. .. .. .. 26 3.7. PyPkg portal RDF model .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 27 3.8. The PyPkg deployment model .. .. .. .. .. .. .. .. .. .. .. .. .. .. 27 3.9. PyPkg installation process .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 31 3.10. PyPkg uninstallation process .. .. .. .. .. .. .. .. .. .. .. .. .. .. 33 3.11. The PyPkg dependency scheme .. .. .. .. .. .. .. .. .. .. .. .. .. 34 4.1. SciPy DOAP/PyPkg description .. .. .. .. .. .. .. .. .. .. .. .. .. .. 41 4.2. Description of SciPy version 0.3.2 .. .. .. .. .. .. .. .. .. .. .. .. .. 42 4.3. Portal index .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 43 4.4. SciPy build recipe .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 44 4.5. SciPy PyPkg package description .. .. .. .. .. .. .. .. .. .. .. .. .. 44 4.6. Creating SciPy package with PyPkg.PkgMaker .. .. .. .. .. .. .. .. .. 45 4.7. Product data for SciPy installation script .. .. .. .. .. .. .. .. .. .. .. 46 4.8. scipy-installer.py presents itself .. .. .. .. .. .. .. .. .. .. .. 47 4.9. scipy-installer.py after calculating dependencies .. .. .. .. .. .. .. .. .. 48 4.10. Installing SciPy and dependencies .. .. .. .. .. .. .. .. .. .. .. .. .. 48 5.1. CodeZoo entry for the SciPy project .. .. .. .. .. .. .. .. .. .. .. .. 54 List of Tables 1.1. The Simula Suite .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 3 Chapter 1. Introduction Distribution and deployment of software is a recurring theme in dealing with computers, whether the emphasis is on the former or the latter depends on the individual’s level of involve- ment. As a user one will typically deal with deployment tasks on a fairly regular basis, in order to add to one’s system or as part of maintaining existing installations. If one is to install some- thing or apply an update,acquiring the software generally involves some contact with distribution mechanisms. As a provider (developer) of software on the other hand, a deeper level of involve- ment with the distribution process is required, since it governs how one’s products are offered to users. PyPkg is intended as an integrated, generic, system for handling tasks related to distribution and deployment of software, in a way that serves both the user and provider perspective.The two are integrated since they are ultimately facets to the same higher purpose:to deliver software into the hands of users so it can be applied. PyPkg approaches the problem by presenting facilities for distribution and deployment, aimed at providers and users respectively. These facilities together form a system, through which users can obtain and deploy software from providers in one seamless operation. The distribution facilitiesserve to establish a layer of software providers,that describe them- selves and their products in a generic language (expressed through XML).The deployment facili- ties,which collaborate with the distribution layer,manifest themselvesas Python [36]-based tools that follow a common, platform-agnostic, scheme for deploying software on target platforms.As of the current version of PyPkg, deployment tasks are served by a single tool, a fully automated installation client. Also, PyPkg tools are at this stage invariably bound to the Linux platform. The focus on Linux during development was a conscious decision,a way to keep complexity down. This platform is somewhat of a moving target, given the differences between individual Linux distributions, so it presents a significant challenge in itself. As such, this thesis is largely coloured by a Linux-centric perspective. We will now go on to present a concrete scenario involving distribution and deployment of a software suite for simulating tsunamis. This example should give a suitable impression of the complexity involved in providing Linux users with a less than trivial software package, and, ultimately, how PyPkg may be of help in dealing with this complexity. 1.1. A specific problem scenario