A Build System for Software Development in Robotic Academic Collaborative Environments
Total Page：16
File Type：pdf, Size：1020Kb
A Build System for Software Development in Robotic Academic Collaborative Environments Daniele E. Domenichelli∗, Silvio Traversaro∗, Luca Muratorey, Alessio Rocchiy, Francesco Nori∗, Lorenzo Natale∗ ∗iCub Facility, Istituto Italiano di Tecnologia (IIT), Via Morego 30, 16163 Genoa, Italy yDepartment of Advanced Robotics, Istituto Italiano di Tecnologia (IIT), Via Morego 30, 16163 Genoa, Italy fdaniele.domenichelli, silvio.traversaro, alessio.rocchi, luca.muratore, francesco.nori, [email protected] Abstract—The software development cycle in the robotic re- and in particular the iCub software project1 [1]. This article search environment is hectic and heavily driven by project or is organized as follows: paper deadlines. Developers have only little time available for packaging the C/C++ code they write, develop and maintain the • We analyze the academic software development cycle and build system and continuous integration tools. will point the issues that arise in this environment (Sec- Research projects are joint efforts of different groups working tion II), emphasizing the critical role of the build system remotely and asynchronously. The typical solution is to rely (Section III), and describing the concept of “superbuild” on binary distributions and/or large repositories that compile explaining their importance (Section IV). all software and dependencies. This approach hinders code sharing and reuse and often leads to repositories whose inter- • We will list some of the tools commonly used in software dependencies are difficult to manage. development to handle dependencies (Section V), and we Following many years of experience leading software integra- will state our objectives and explain the reasons why tion is research projects we developed YCM, a tool that supports we decided not to introduce a new tool in our workflow our best practices addressing these issues. YCM is a set of CMake (Section VI). scripts that provides (1) build system support: to develop and package software libraries and components, and (2) superbuild • We will introduce YCM the set of CMake modules that deployment: to prepare and distribute sets of packages in source we developed, and we will show how YCM superbuilds form as a single meta build. In this paper we describe YCM are currently being used in four different European- and report on our experience adopting it as a tool for managing funded research projects by different teams, and success- software repositories in large research projects. fully promote reuse of code and collaboration between Index Terms—Build system; software development cycle; soft- ware reuse; superbuild; researchers from different groups (Section VII). II. ACADEMIC SOFTWARE DEVELOPMENT CYCLE I. INTRODUCTION It is an indisputable fact that lots of software created for In robotic research environments the software development research is wasted. Academic research products are scientific cycle is quite hectic and often heavily driven by project or papers. Data produced using the software is more important paper deadlines. The “publish or perish” model that drives the than software itself. Researchers may know how to write research has a sensible impact on the re-usability of the code code, but they are not software engineers [2] and they lack developed for research purpose on two different levels: on a experience in coding using established best practices. They micro scale, the research products are small tools that are not work individually most of the time and have to be at the sufficiently engineered to be reused in large projects; on a same time developers, designers, testers, release managers and larger scale, the products are large conglomerates of software integration managers. Moreover they have academic duties with strong coupling (i.e. lots of references between files in such as teaching, tutoring and taking classes [3]. different modules) and weak cohesion. In both cases reusing The continuous renewal of Ph.D. and Grad Students without the code is not easy. experience in software development weighs heavily on the This situation is worsened by the fact that most robotics software development process. A review process, besides software is either developed in C/C++ or has extensive depen- improving the overall quality of the code [4], could contribute dencies written in this languages, for which no standard cross- to improve the developers’ software skills. Unfortunately in platform tools for packaging and distribution is available, as most of the cases, this kind of process does not exist for opposed to higher level languages. academic software. The goal of our research is to foster re-usability and the All these factors introduce several different effects in the overall quality and maintainability of software developed for software products developed for research purposes. academic research purposes and its build system. It is moti- vated by our experience in managing large C/C++ software 1The iCub (http://www.icub.org/) is an open source humanoid robot adopted in more than 30 research groups world-wide, whose software is the result projects in collaborative robotic research environments (con- of the work of a large community (1.8M LOC, 170 contributors and 103 sisting of several research groups based on different countries), developers, source: OpenHub, https://www.openhub.net/p/robotology) On a micro scale, the research driven software development The pitfall in this approach is that when the funded project is cycle heavily impacts the self contained tools, algorithms, and over, the parts of the code produced that should be reusable are libraries released. The goal of the development phase is often supposed to be re-engineered and moved out of the repository. to produce a working prototype as fast as possible, with the Unfortunately this rarely happens, due to lack of resources intention to clean up the code later. Unfortunately experience and to the need to move on to new projects. If some code shows that the Agile software development principle known as is required for a later project, even if just a small part, “re- LeBlanc’s Law “Later equals never” [5], [6] is almost always usability” is achieved by depending on the whole software true for academic software, unless some commercial interests collection, or by duplicating the code in the new repository. intervene. The opposite approach is to let each team use their favorite While the code related to the subject of the study itself might tools and to store their code using their favorite SCM and host. be good quality, the code surrounding it is often low quality, This approach apparently has the advantage that every group written in a hurry, and good practices are often neglected. can continue its work as they did before without spending Researchers have only little time available for packaging the time to learn how the other groups work. Unfortunately when code they write and the build system maintenance is something it is time to integrate the efforts of different groups, the effect that goes often beyond their skills and interests. As a result, the is exactly the opposite. Setup of all the software required product might perform “well enough” for writing an article, becomes complicated and time consuming, due to the variety but when this is published – due to the tight research schedule of tools required, to the lack of testing and documentation [8], driven by projects, Ph.D. programs or paper deadlines – the and to the fact that build instructions are not always complete author is likely to abandon the code and start working on or up to date. Similarly, updates of the whole stack are painful, something new. The code is never properly re-engineered, and sometimes dependencies are added, and the other developers it is hardly re-usable as a component of a larger system. are not notified, so they have to figure it out by themselves. On a macro scale, the research driven software development The learning curve for newcomers is steep and it could take cycle affects the whole software ecosystem around funded days before they can be operational. research projects where different researchers, teams, and insti- Another defect of this kind of approach is that it makes tutions work remotely and asynchronously on different parts harder to give visibility to the software developed for a funded of a big project. project, since code and documentation are scattered around Trying to achieve integration and interoperability between different repositories and hosts. software developed by different groups is definitely a push In this paper we suggest a hybrid approach that uses a meta- towards better practices, but that’s not enough. repository whose only job is to fetch external repositories, When a funded project starts, one of the first decision to where the actual source code is stored, and build it eventually take is where to store the source code, which tools will be with the assistance of an external tool to handle dependencies used for development, and which operating systems, IDEs, (we called this approach “superbuild”). and compilers will be supported. Users have very different sets of skills, they work in III. BUILD SYSTEMS IN ACADEMIC SOFTWARE different fields and have different roles in the organizations: The build system of a software project converts the source students, researchers, developers, team leaders. Each of them code into libraries and executables. It has an important role in will use the software with different goals: to produce some any software project [9] and it needs to evolve together with data for a thesis or for an article, to implement some new the source code. A build system that grows uncontrolled or algorithm, to show some demo to the projects reviewers. that becomes too hard to understand and to maintain can slow Research groups are usually interested in enforcing their down the development process [10] or even stop the developers group identity, while coordinators need to show an integrated from making any progress [6].