Handbook of Opensource Tools.Pdf

Total Page:16

File Type:pdf, Size:1020Kb

Handbook of Opensource Tools.Pdf Handbook of Open Source Tools Sandeep Koranne Handbook of Open Source Tools Sandeep Koranne 2906 Bellevue Ct West Linn, Oregon 97068 USA [email protected] ISBN 978-1-4419-7718-2 e-ISBN 978-1-4419-7719-9 DOI 10.1007/978-1-4419-7719-9 Springer New York Dordrecht Heidelberg London Library of Congress Control Number: 2010938855 © Springer Science+Business Media, LLC 2011 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com) Preface The constant and speedy progress made by humankind in the industrial revolution, and more recently in the information technology era can be directly attributed to sharing of knowledge between various disciplines, reuse of the knowledge as sci- ence and technology advanced, and inclusion of this knowledge in the curriculum. The phrases “do not reinvent the wheel” and “to stand upon the shoulders of giants” come to our mind as representative of this thought process of using existing solu- tions and building upon existing knowledge, but at the same time contributing to the society as a whole. It was with this intention of documenting existing (circa 2010) Open Source Tools for Scientists and Engineers, that I set about to write this book. Computer technology has progressed at such a fast pace that it is difficult (nary impossible) to catalog all of the existing software systems which are available to us. To simplify our task I have chosen a representative software to solve a class of problem. Where space and time permitted I have provided alternatives as well. A key benefit of using open-source applications is that the code can be compiled on a system which is non-standard. Or, it can be compiled with CPU specific opti- mizations which a general purpose binary released from an ISV cannot assume. As CPU technology advances rapidly, and software has a longer lifespan, the ability to recompile the source code becomes more and more important. The same can be said with open-source implementations of data-standard in image processing, and docu- mentation retrieval. In this media focused era, more and more content is being stored as digital data. Unlike, paper, whose archival properties have been refined over cen- turies, digital media has not gone through the same process of archival management. In situations where data archival is necessary, a key component is the persistence of the key software components which read and write the digital data files. Since no one can predict the computers of the next century, open-source software is essential to long term archival of information. Rather than duplicate the fine documentation for each package, this book is orga- nized in sections related to solve a particular problem. This book should be treated as an “existential quantifier” 9, rather than 8, on the information provided for each task. Once the existence of a solution or software tool to address the problem is v vi Preface known, more details about the solution can be researched. Each software system or tool is presented in a simple to read manner describing the main problem the system addresses and the tasks performed. Each chapter is presented in a similar manner to ease referencing. Although many of the software mentioned in this book are routinely used in science and engineering tasks, more and more I have found that students and gen- eral practitioners from other fields, such as liberal arts, music, statistics, are using these in their work and study. This book contains references to artificial intelligence programs and tools which are being widely used in cognitive sciences. Many of the software tools use libraries and development tools which are themselves open- source; this synergy is representative of the open-source concept, and a key driver to its proliferation. As such, any large open-source software is a good learning ex- ample to study the use of its components. For example the GRASS GIS software (presented in Section 14.9.1) is itself developed using a number of libraries such as (i) PROJ4, (ii) HDF5, (iii) MySQL, (iv) FFTW, and many others. To learn how to use these libraries in a real world example, one only has to study the GRASS GIS source code. This is a key advantage of open-source tools. Another argument (made mostly in the context of mathematical proofs) stems from the scientific validity and acceptance of computer generated, or computer as- sisted proofs. For such proofs to be included as standard material, the software sys- tem used to arrive at the result must also be available to researchers, as well as its own correctness be verified. These goals are readily achieved by open-source math- ematical software as presented in Chapter 16. I present a short summary of the book contents: This book is divided in six parts. The first part describes the open-source operat- ing systems and user interaction as well as introspection tools. Chapter 1 includes a discussion on Bash shell, POSIX compliant libraries and open-source programming languages (including Erlang, Lua and Smalltalk). External utilities (such as tar, find and rsync) as well as OpenSSH are discussed to ease the users interaction with a modern GNU/Linux type operating system. Chapter 2 presents several text processing and document creation and management tools. These include OpenOf- fice and various LATEXprocessing tools. Software for Wiki management as well as graphical page layout are also described. Part II of this book focuses on the process of open-source software creation. For the reader who wants to know more about the methods and systems used by the authors to create the open-source tools, this part provides information on the GNU build system, version control, compilers, APIs and much more. In Chapter 3 I present the GNU Compiler Collection. Commonly used command-line options, pragmas, pre-processor defines as well as GCC intrinsics are explained. Examples of using GCC to compile Java and Ada are presented, as well as recent features of GCC including OpenMP support and C++ advice features. Source code version control with CVS and SVN is presented with the help of examples; GUI front-ends for version control (TkCVS) is presented and used in many examples. The GNU Build system is discussed with the help of examples in Section 3.3. GNU Make as Preface vii well as SConstruct are described in Section 3.4. Both GNU make and SConstruct are also used in many examples in the sequel of this book. Chapter 3 also contains a description of Bugzilla for defect tracking (Section 3.5) and a section on various editors and IDEs available on GNU/Linux for editing source code. Static code checking and analysis of source code is presented in Section 3.8, while the use of GNU debugger GDB is shown with examples in Section 3.9.1. Graphical front-ends to GDB (including GDB Insight) are shown in Section 3.9.2. Code optimization using profiling and cache measurement with GNU profiler and Valgrind is discussed with examples in Section 3.12. The C standard library and the C++ standard library (including STL) are discussed in Chapter 4. In Chapter 5 I describe the Apache Portable Runtime (APR) library. In particular APR memory pools, APR process library, APR thread and thread pool library, APR file information and memory mapping library are explained with the help of short examples. Advanced APR concepts, dealing with the use of Memcache library are also demonstrated. All examples have been compiled and run on GNU/Linux system running Fedora Core 12, and thus are known to work. Using the examples presented in this part of the book, alongwith the documentation for the library, simplifies the learning process of the API. Chapter 6 contains a description of the most useful parts of the Boost C++ API. For lack of space, I had to choose only a small portion of Boost for demonstra- tion. I depict the design and usage of Boost library with the help of examples in- cluding Boost smart pointer and memory pool, the Boost asynchronous IO (asio) framework, Boost data-structures. Boost Graph Library (BGL) is an almost com- plete graph representation library, which also includes an implementation of many graph algorithms. BGL is presented with the help of examples in Section 6.4. Boost multi-threading (like APR) is a portable and integrated (with C++) threading sys- tem. I have presented examples which the reader can contrast with APR threading and POSIX pthread examples presented in this book. Python language integration with C++ can be achieved using SWIG as well as using Boost Python integration framework, an example of which is presented in Section 6.7. Boost generic im- age processing library is presented in Section 6.8, while Boost parsing framework (SPIRIT) is presented in Section 13.4.1 of Chapter 13 on Compiler systems. Performance optimization of programs using Google perftools memory alloca- tion and profiler is shown in Chapter 7.
Recommended publications
  • Apache Shindig V
    ...................................................................................................................................... Apache Shindig v. 1.0 User Guide ...................................................................................................................................... The Apache Software Foundation 2012-03-11 T a b l e o f C o n t e n t s i Table of Contents ....................................................................................................................................... 1. Table of Contents . i 2. Introduction . 1 3. Download . 3 4. Overview . 6 5. Getting Started . 16 6. Documentation Centre . 22 7. Java . 23 8. Building Java . 24 9. Samples . 28 10. PHP . 29 11. Building PHP . 30 12. Features . 32 13. Community Overview . 35 14. Getting Help . 37 15. Code Conventions . 38 16. Jira Conventions . 39 17. SVN Conventions . 40 18. Shindig Release Process . 42 19. FAQ . 46 20. Powered By . 48 21. Resources . 49 © 2 0 1 2 , T h e A p a c h e S o f t w a r e F o u n d a t i o n • A L L R I G H T S R E S E R V E D . T a b l e o f C o n t e n t s ii © 2 0 1 2 , T h e A p a c h e S o f t w a r e F o u n d a t i o n • A L L R I G H T S R E S E R V E D . 1 I n t r o d u c t i o n 1 1 Introduction ....................................................................................................................................... 1.1 Welcome To Apache Shindig ! Apache Shindig is an OpenSocial container and helps you to start hosting OpenSocial apps quickly by providing the code to render gadgets, proxy requests, and handle REST and RPC requests.
    [Show full text]
  • S O F T W a R E D E V E L O P E R ' S Q U a R T E R
    SOFTWARE DEVELOPER’S QUARTERLY Issue 15• Oct 2010 Editor’s Note ........................................................................... 1 ITKV4 DEVELOPMENT Over the last quarter, the ITK development team released Recent Releases ..................................................................... 1 the first two Alpha versions of ITKv4. ITKv4 is a major refac- toring of the ITK toolkit that introduces improved wrapping Distributed Version Control: The Future of History ............ 2 for other languages, a modular architecture and revisions to Insight Toolkit Plug-ins: VolView and V3D .......................... 6 many of ITKs components. These two releases were intended to perform a general code clean up, dropping the tricks to VTK Wrapper Ovehaul 2010 .................................................. 9 support now-defunct compilers used in the past while paving the way for major refactoring activities to commence. The The CDash "@Home" Cloud ................................................ 12 next decade of ITK has begun. Multi-Resolution Streaming in VTK and ParaView ........... 13 Details One of the most significant operational changes is that the Community Spotlight .......................................................... 15 source code of ITK was moved to a Git repository and a new development workflow has been put in place in order to Kitware News ...................................................................... 20 integrate the teams that are collaborating in this new version of the toolkit. The major changes introduced in these two releases are described below. ITKv4-Alpha-01 The Kitware Source contains articles related to the develop- The following compilers were deprecated: Borland 5.5, Visual ment of Kitware projects in addition to a myriad of software Studio 6.0 and 7.0, SGI CC, Sun CC 5.6, Metrowerks. Source updates, news and other content relevant to the open source code that was intended solely to support these compilers community.
    [Show full text]
  • S O F T W a R E D E V E L O P E R ' S Q U a R T E R
    SOFTWARE DEVELOPER’S QUARTERLY Issue 15• Oct 2010 Editor’s Note ........................................................................... 1 ITKV4 DEVELOPMENT Over the last quarter, the ITK development team released Recent Releases ..................................................................... 1 the first two Alpha versions of ITKv4. ITKv4 is a major refac- toring of the ITK toolkit that introduces improved wrapping Distributed Version Control: The Future of History ............ 2 for other languages, a modular architecture and revisions to Insight Toolkit Plug-ins: VolView and V3D .......................... 6 many of ITKs components. These two releases were intended to perform a general code clean up, dropping the tricks to VTK Wrapper Ovehaul 2010 .................................................. 9 support now-defunct compilers used in the past while paving the way for major refactoring activities to commence. The The CDash "@Home" Cloud ................................................ 12 next decade of ITK has begun. Multi-Resolution Streaming in VTK and ParaView ........... 13 Details One of the most significant operational changes is that the Community Spotlight .......................................................... 15 source code of ITK was moved to a Git repository and a new development workflow has been put in place in order to Kitware News ...................................................................... 20 integrate the teams that are collaborating in this new version of the toolkit. The major changes introduced in these two releases are described below. ITKv4-Alpha-01 The Kitware Source contains articles related to the develop- The following compilers were deprecated: Borland 5.5, Visual ment of Kitware projects in addition to a myriad of software Studio 6.0 and 7.0, SGI CC, Sun CC 5.6, Metrowerks. Source updates, news and other content relevant to the open source code that was intended solely to support these compilers community.
    [Show full text]
  • Handbook of Open Source Tools
    Handbook of Open Source Tools Sandeep Koranne Handbook of Open Source Tools Sandeep Koranne SW Boeckman Road 8005 97070 Wilsonville Oregon USA [email protected] ISBN 978-1-4419-7718-2 e-ISBN 978-1-4419-7719-9 DOI 10.1007/978-1-4419-7719-9 Springer New York Dordrecht Heidelberg London Library of Congress Control Number: 2010938855 © Springer Science+Business Media, LLC 2011 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com) Preface The constant and speedy progress made by humankind in the industrial revolution, and more recently in the information technology era can be directly attributed to sharing of knowledge between various disciplines, reuse of the knowledge as sci- ence and technology advanced, and inclusion of this knowledge in the curriculum. The phrases “do not reinvent the wheel” and “to stand upon the shoulders of giants” come to our mind as representative of this thought process of using existing solu- tions and building upon existing knowledge, but at the same time contributing to the society as a whole.
    [Show full text]
  • Gem E Objetos De Aprendizagem Especializados Para a Educação Musical
    Revista Brasileira de Informática na Educação, Volume 23, Número 2, 2015 Plataforma Mignone: Ambiente Virtual de Aprendiza- gem e Objetos de Aprendizagem Especializados para a Educação Musical Mignone Platform: Virtual Learning Environment and Learning Objects Specialized for Music Education Fernando Pinhati Sean W. M. Siqueira Universidade Federal do Estado do Rio de Janeiro Universidade Federal do Estado do Rio de Janeiro (UNIRIO) (UNIRIO) [email protected] [email protected] Resumo Atualmente, há uma demanda por recursos multimídia e sociais de forma integrada em ambien- tes de aprendizagem, o que é uma necessidade ainda maior na Educação Musical. Neste artigo descrevemos as particularidades da Educação Musical e apresentamos a Plataforma Mignone. Esta plataforma é constituída por uma arquitetura formal para o desenvolvimento de ambientes virtuais e de um modelo para criação facilitada de objetos (e atividades) de aprendizagem, am- bos especializados para a área da música. Aqui detalhamos as diretrizes que guiam a constru- ção do ambiente, bem como apresentamos mais detalhes do estudo de caso que foi realizado com 27 alunos do ensino médio de uma escola pública no Rio de Janeiro. A partir de análises quantitativas e qualitativas, foi possível concluir que o oferecimento de recursos multimídia e sociais, estimulados pelo uso da Plataforma Mignone como guia tanto na construção do ambi- ente quanto na dos objetos de aprendizagem, influenciam indiretamente na aceitação de ambi- entes virtuais de aprendizagem para Educação Musical pelos alunos. Palavras-Chave: Educação Musical, Modelo C(L)A(S)P, UTAUT, Aceitação de Tecnologias, Ambientes de Aprendizagem, Redes Sociais Abstract Nowadays, there is a great demand for multimedia and social resources in an integrated way in learning environments, which is a still bigger need for Music Education.
    [Show full text]
  • A Domain-Specific Language for Internal Site Search
    A Domain-Specific Language for Internal Site Search Master’s Thesis Elmer van Chastelet A Domain-Specific Language for Internal Site Search THESIS submitted in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE in COMPUTER SCIENCE by Elmer van Chastelet born in Dordrecht, the Netherlands Software Engineering Research Group Department of Software Technology Faculty EEMCS, Delft University of Technology Delft, the Netherlands www.ewi.tudelft.nl c 2013 Elmer van Chastelet A Domain-Specific Language for Internal Site Search Author: Elmer van Chastelet Student id: 1213539 Email: [email protected] Abstract The importance of search facilities on a website grows with the size of the con- tent being served. User expectations for internal site search are greatly influenced by global web search engines, requiring developers of web applications to go beyond basic search functionality. In this thesis, a domain-specific language (DSL) for in- ternal site search is designed and integrated as a sublanguage of WebDSL (the base language). WebDSL is an existing DSL for web development. Through an explo- ration of the problem and solution space, the facets related to internal site search are explained. Furthermore, an iterative approach applied at the development of the DSL is presented. This approach is based on the use of existing base language constructs as core language. The core languages provide access to implemented search features. Linguistic abstractions are added on top of the core languages, constituting the even- tual interface of the language. Evaluation by means of enriching two web applications with search features show that the DSL has substantial coverage of the internal site search domain.
    [Show full text]