DOCSLIB.ORG

Efficient Automated Build and Deployment Framework with Parallel Process

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Efficient Automated Build and Deployment Framework with Parallel Process ISSN (Online) : 2278-1021 ISSN (Print) : 2319-5940 International Journal of Advanced Research in Computer and Communication Engineering Vol. 3, Issue 6, June 2014 Efficient Automated Build and Deployment Framework with Parallel Process Prachee Kamboj1, Lincy Mathews2 Information Science and engineering Department, M. S. Ramaiah Institute of Technology, Bangalore, India 1, 2 Abstract: Continuous integration has been in theory for a long time but successful practicing is still a dream. To resolve the integration problems and increase the customer satisfaction, it is important to follow the practice of continuous integration in software development. Doing manual build and deployment regularly results in lot of time consumption and thus becomes an inefficient process. This paper proposes architecture for build and deployment by following best practices. It aims to fully automate the build process by using some available tools for source management, binary management and compilation. Build architecture gives a method to integrate all tools in build server. Successful completion of build is followed by deployment. This paper also proposes a deployment architecture which is common for all development environments. Deployment architecture states three steps, required to do the deployment and also proposes optional features to enhance the deployment process. Keywords: Automation, Build, Continuous Integration, Deployment, Parallel I. INTRODUCTION Software development is done in different environments. Generally industry follows four environments which are Development, Quality Assurance, Stage, and Production [1] . Before delivering the software to the customer it is essential to test the software and ensure the quality of Figure 1: Build and Deployment automation for different development software. The environments should not intersect as the Environments activities between each group in software development This paper proposes a new build and deployment should not interfere with one another. The development framework suitable for enterprise. Framework has focused environment is used for developing the application. Users on automation and introduction of parallelism to increase in development environment can expect the application to efficiency and reduce response. Build and deployment change often and be broken for extended periods of process is same for all environments to avoid confusion time. The Quality Assurance environment is used for for the users. testing. The staging environment is the dummy of production environment and is used to test the application II. BUILD for production environment without having users accessing the application. This environment is used to A. Build Steps make sure security settings are correct and that other Conventionally building software is referred to as services on the user computer or server do not interfere converting the source code into executable file [2]. In this with the application. The production environment is the era of software development getting mature, it needs to do environment where the users actually run the live some more tasks which makes the build stable and more application. Changes to this environment happen in meaningful. It is an end-to-end process that comprises of periodic updates to the software after the application many steps as shown in figure 2 below. changes have passed through the previous three environments. The other environments help ensure that the application is well tested and stable when it is released. Build and deployment is an integral part of any software development process. Software build refers to the process Figure 2: Block diagram of Build Steps of compiling the source code into a form which can be Creating a Clean Build Location prepares an environment deployed in a computer or a server. In an environment like for the build process to execute. Same source code file can development and quality assurance, build and deployment be modified by many users simultaneously. To maintain can be done many times in a day also. It makes sense to consistency, user’s files are tagged. Original copies of all make whole build and deployment process automated. source files are labelled in baseline separately. Labelling Automation is being adopted by many organization for its the source code approves the source file which is unmatchable advantages compared to manual process. Fig consistent among all files. Running Pre-Build Tasks 1 shows the block diagram of build and deployment includes capturing metadata about the code like setting the automation on different development environments [1]. version number for the output file, etc. Compiling the Code & Linking Libraries includes Unit and integration Copyright to IJARCCE www.ijarcce.com 7009 ISSN (Online) : 2278-1021 ISSN (Print) : 2319-5940 International Journal of Advanced Research in Computer and Communication Engineering Vol. 3, Issue 6, June 2014 testing is also done before compilation to ensure manually to initiate the build so that steps 3 to 7 are reliability. Other types of testing can be done depending automated. on the application requirement. At the end, it outputs the The key component of this architecture is to set a build executable code which is ready for deployment. server. Build server is the next step to have a stable and robust build. [9] This server is the heart of the whole build B. Build Automation process. Build server performs all major tasks like pulling Agile software development is a proven method adopted down the latest labelled source code, acts as a continuous by many enterprises. Among the basic principle of agile integration server by installing build tool which triggers software development methodology is contiguous build automatically, does the compilation of the source integration [4] [5]. Continuous integration is a practice based code etc. Build server will always have the clean code of on principle of detecting integration error in early stage. It the source code label. Build server will also publish the is practiced by integrating developers codebase many binary to binary repository management system. It even times in a day. Usually integration is done at least on daily resolves the encoding dependency problem for binaries. basis. Build is a repeated process, done in every Software development process produces mainly source environment before going live for the users of application. code and binary artifacts (or executable code) which serve Even if one is not practicing continuous integration, build different purpose. Source configuration management automation is required. Build automation helps in system is commonly required where there are multiple improving the software quality, reduces compile and developers working on the same project concurrently on linking the library time, eliminates redundant tasks [3], same set of source file. It helps in source control, change minimizes broken builds occurrence, reduces key management, and in version control. It keeps the track of personnel dependencies, improves traceability through the changes happening in the file, assigns a temporary logging the build steps etc. number to the users file and keeps the original file preserved. When a developer is satisfied with the source file then source file is checked in to source configuration C. Build Architecture management server to create a new version of the file. If many users check in the file at same time then the source Figure 3 explains the build process. In build architecture configuration management server combines the changes steps 2 to 7 can be automated if one is practicing and is merged to create a new version of file. continuous Integration, else user can trigger the build Figure 3: Build Architecture There are many ways to trigger build but as this paper In build server, build tool creates a clean build location for talks about continuous integration practice, this step the build and deletes old temporary build space that is not should be automated. Ideally as soon as developer required for the current build. Build tool is configured to commits the source code in source code repository, build get the latest source code in the build server from source should be triggered. Build can be scheduled for nightly configuration management server. This source code is basis and there should be scope of manually triggering the stored in build server and it should not be modified further build. After this all steps should be automated and in the build server. Compiling the code is the major step executed sequentially. and the most time consuming step. Code is compiled base Copyright to IJARCCE www.ijarcce.com 7010 ISSN (Online) : 2278-1021 ISSN (Print) : 2319-5940 International Journal of Advanced Research in Computer and Communication Engineering Vol. 3, Issue 6, June 2014 on the compiler required by the application programming language. Unit testing can also be performed as part of this step. After successful completion of the testing, it should be archived with a meaningful label on it. It should have some numbering or a time stamp value to distinguish the file from the other builds. Once the build is completed, archived file should be send to the Binary Repository Management server at a common location for all the builds of this application. Doing so helps other team members to download the file when required. If a build fails concerned person should be informed about the same. Build logs are also generated to find the reason of the build failure. III. DEPLOYMENT A. Deployment
Load more

Recommended publications
  • Version Control Systems, Build Automation and Continuous Integration Integration and Verification Techniques
    Version Control Systems, Build Automation and Continuous Integration Integration and Verification Techniques Systematic methods and automation can highly enhance the teamwork on software projects. During this laboratory we will learn the basics of the following techniques: • Version control systems to support efficient teamwork. • Build automation to ensure stable and consistent builds. • Continuous Integration to provide automatic and systematic build, test execution and also deployments. 1 Version Control Systems Version control systems allow to keep all the historical versions of your software for easy tracking. Using version control also benefits team collaboration and improves the efficiency of the development team. In addition, it can be used as a central repository for the data, making build automation and continuous integration possible. There are historically two different approaches for managing the repositories: • Centralized model (CVS and Subversion/SVN): there is one server that contains “the repository” and everyone else checks code into and out of that repository. An important feature of these systems is that only the repository has the full history of all changes made. A checkout from this central repository will place a “working copy” on the user’s machine. This is a snapshot from a certain version of the project on his/her disk. • Distributed model (Git): In a distributed version control system, instead of a checkout, a user will clone a repository from a remote server. In return, he/she receives a full-fledged repository, not just a working copy. The user then has his/her own repository on the local machine – including all of the project’s history.
    [Show full text]
  • Devops Point of View an Enterprise Architecture Perspective
    DevOps Point of View An Enterprise Architecture perspective Amsterdam, 2020 Management summary “It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change.”1 Setting the scene Goal of this Point of View In the current world of IT and the development of This point of view aims to create awareness around the IT-related products or services, companies from transformation towards the DevOps way of working, to enterprise level to smaller sizes are starting to help gain understanding what DevOps is, why you need it use the DevOps processes and methods as a part and what is needed to implement DevOps. of their day-to-day organization process. The goal is to reduce the time involved in all the An Enterprise Architecture perspective software development phases, to achieve greater Even though it is DevOps from an Enterprise Architecture application stability and faster development service line perspective, this material has been gathered cycles. from our experiences with customers, combined with However not only on the technical side of the knowledge from subject matter experts and theory from organization is DevOps changing the playing within and outside Deloitte. field, also an organizational change that involves merging development and operations teams is Targeted audience required with an hint of cultural changes. And last but not least the skillset of all people It is specifically for the people within Deloitte that want to involved is changing. use this as an accelerator for conversations and proposals & to get in contact with the people who have performed these type of projects.
    [Show full text]
  • Coverity Static Analysis
    Coverity Static Analysis Quickly find and fix Overview critical security and Coverity® gives you the speed, ease of use, accuracy, industry standards compliance, and quality issues as you scalability that you need to develop high-quality, secure applications. Coverity identifies code critical software quality defects and security vulnerabilities in code as it’s written, early in the development process when it’s least costly and easiest to fix. Precise actionable remediation advice and context-specific eLearning help your developers understand how to fix their prioritized issues quickly, without having to become security experts. Coverity Benefits seamlessly integrates automated security testing into your CI/CD pipelines and supports your existing development tools and workflows. Choose where and how to do your • Get improved visibility into development: on-premises or in the cloud with the Polaris Software Integrity Platform™ security risk. Cross-product (SaaS), a highly scalable, cloud-based application security platform. Coverity supports 22 reporting provides a holistic, more languages and over 70 frameworks and templates. complete view of a project’s risk using best-in-class AppSec tools. Coverity includes Rapid Scan, a fast, lightweight static analysis engine optimized • Deployment flexibility. You for cloud-native applications and Infrastructure-as-Code (IaC). Rapid Scan runs decide which set of projects to do automatically, without additional configuration, with every Coverity scan and can also AppSec testing for: on-premises be run as part of full CI builds with conventional scan completion times. Rapid Scan can or in the cloud. also be deployed as a standalone scan engine in Code Sight™ or via the command line • Shift security testing left.
    [Show full text]
  • Security Automation Best Practices
    SECURITY AUTOMATION BEST PRACTICES A Guide to Making Your Security Team Successful with Automation TABLE OF CONTENTS Introduction 3 What Is Security Automation? 3 Security Automation: A Tough Nut to Crack 4 Prepare Your Security Organization for Success 6 Make a Choice: Build or buy? 8 Add Automation When the Time Is Right 10 Know Which Tasks Are Ideal for Automation 12 Testing Automation’s Capabilities 14 Implementing Security Automation 15 About Rapid7 16 Appendix 17 | Rapid7.com Security Automation Best Practices - 2 INTRODUCTION The best security postures are those that are built on efficiency and time-to-response. While processes make it possible to get a job done faster, creating ones that solve practical problems and result in measurable efficiency gains can be a time-consuming task, and without the expertise required to create and build them, they simply don’t get done. This is where security automation comes in. WHAT IS SECURITY AUTOMATION? Security automation streamlines a series of repetitive, manual tasks into cohesive and automated workflows. By plugging a set of tasks into an automated system (such as those involved in phishing investigations), security processes become: • More efficient • Less prone to human error With increased efficiency, better and faster decisions can be made, which in turn can improve your organization’s entire security posture. Even better, with repetitive and manual tasks taken care of by automation, security personnel can instead focus on more strategic work, which boosts their job satisfaction and ensures you’re retaining good talent. | Rapid7.com Security Automation Best Practices - 3 SECURITY AUTOMATION: A TOUGH NUT TO CRACK Historically, security automation has been difficult to implement, which is why many companies have yet to take advantage of it.
    [Show full text]
  • How to Get the Most out of Your Ci/Cd Workflow Using Automated Testing
    WHITE PAPER HOW TO GET THE MOST OUT OF YOUR CI/CD WORKFLOW USING AUTOMATED TESTING This paper is aimed at Test and QA Executives as well as Project Managers who are considering adopting automated testing, but are unsure of how to get started. It highlights the benefits of automated testing, the recommended technical approach to take, and suggests tools that enable teams to successfully adopt automated testing as part of a healthy continuous integration and delivery process. It also examines which tests to automate and which to continue to do manually. TABLE OF CONTENTS 3 Executive Summary 10 Mobile Testing 3 Automated Testing as Part of the Broader 11 Which Tests to Continue Manually CI/CD Pipeline 11 Usability Tests 4 The Kubernetes Effect 12 One-off Tests 5 GitOps = Fully-automated CI/CD 12 Selecting the Right Test Automation Solution 5 Reality Check - The Majority of Testing is 12 The Crucial Decision - In-house, Open Still Manual Source, or Commercial 6 What is Test Automation? 13 Selenium - The Leading Test Automation Tool 7 Manual Testing vs Automated Testing - for Web Apps Weighing the Benefits 13 Appium - The Leading Test Automation Tool 7 Obstacles to Adopting Automated Testing for Mobile Apps 8 The Right Approach to Test Automation 14 Open Source Tools Require Expertise to Run In-House 9 Unit and Component Testing 14 The Ideal Solution Should Combine the Best of 9 Headless Testing Both Worlds - Selenium & Appium 9 API or Web Services Testing 15 Conclusion 9 UI Testing 15 About Sauce Labs 10 Regression Testing 15 Appendix 10 Functional Testing EXECUTIVE SUMMARY In today’s hyper-competitive cloud economy, it’s important to be first to market to gain a competitive edge.
    [Show full text]
  • Continuous Integration (CI) Needs and Wishes for Developers of Proprietary Code
    Continuous Integration (CI) Needs and Wishes for Developers of Proprietary Code Michael Hilton, Nicholas Nelson, Danny Dig Timothy Tunnell, Darko Marinov School of EECS, Oregon State University CS Department, University of Illinois {hiltonm,nelsonni,digd}@oregonstate.edu {tunnell2,marinov}@illinois.edu Abstract—Continuous integration (CI) systems automate the Open-source projects are often considered fundamentally dif- compilation, building, and testing of software. Despite CI being ferent from proprietary-code projects [6]. Differences between one of the most widely used processes in software engineering, open-source and proprietary-code projects include that open- we do not know what motivates developers to use CI, and what barriers and unmet needs they face. Without such knowledge source projects foster more creativity [7], have fewer defects [7], developers make easily avoidable errors, managers reduce the and assign work differently [8]. However, similarities have also productivity of developers by making misinformed decisions, tool been found, including how GitHub is used [9], overall project builders invest in the wrong direction, and researchers miss many complexity [7], and amount of modularity [7]. opportunities for improving the software engineering practice. There are still many gaps in our knowledge about why Given the large fraction of proprietary code development, un- derstanding how proprietary developers are using CI is vital to developers use CI, and what barriers they face, especially for improving it. projects of proprietary code. What motivates developers to use We present the first study of how CI is used in the proprietary CI? What are the barriers that developers face when using development of software. We conduct 16 semi-structured inter- CI? What needs do developers have that are unmet by their views with developers from different industries and development current CI system(s)? Do proprietary developers have the same scale.
    [Show full text]
  • From Build Automation to Continuous Integration
    Beginners guide to continuous integration Gilles QUERRET Riverside Software About the speaker • Working with Progress and Java since 10 years • Started Riverside Software 7 years ago • Based in Lyon, France • Focused on technical expertise and continuous integration in those environments • Code analysis for OpenEdge http://geekandpoke.typepad.com define:continuous integration Continuous Integration is a software development practice of performing software integration frequently…several times a day, in fact. Ideally, your software application or system should be built automatically after each commit into a shared version control repository. Upon each successful build, the system integrity should be verified using automated tests that cover if not all, then at least most of the functionality. If some tests fail, the developer responsible is notified instantly and the problem can be identified and solved quickly. Using this approach, you can deliver working and reliable code to the customer much faster, while also mitigating the risk of releasing unstable, buggy software to your users. define:continuous integration 5 steps to continuous integration 1 •Source code repository 2 •Build automation 3 •CI server setup 4 •Automated deployment 5 •Automated tests Use the right tools Step 1 : Source code repository • Keeps track of every change in your codebase • Using homegrown solution can be challenging in a continuous integration environment • Lots of tools on the market : CVS, Subversion, Mercurial, Perforce, BitKeeper, Roundtable… • Side note
    [Show full text]
  • Automatic Building of Java Projects in Software Repositories: a Study on Feasibility and Challenges
    Automatic Building of Java Projects in Software Repositories: A Study on Feasibility and Challenges Foyzul Hassan1, Shaikh Mostafa1, Edmund S. L. Lam2 and Xiaoyin Wang1 1Department of Computer Science, University of Texas San Antonio 2Department of Computer Science, University of Colorado Boulder Email: [email protected], {shaikh.mostfa, xiaoyin.wang}@utsa.edu, [email protected] Abstract—Despite the advancement in software build tools can be applied to just the source code of the repositories, such as Maven and Gradle, human involvement is still often many useful analysis techniques (e.g., points-to analysis [28], required in software building. To enable large-scale advanced call-graph generation [30], string analysis [29]) depend on program analysis and data mining of software artifacts, software engineering researchers need to have a large corpus of built either the resolution of dependencies, types, and bindings to software, so automatic software building becomes essential to extract dependable information, or the build artifacts (byte- improve research productivity. In this paper, we present a code, program traces) as input of the analysis. Furthermore, feasibility study on automatic software building. Particularly, we many well-maintained frameworks (e.g., Soot [25], Wala [6], first put state-of-the-art build automation tools (Ant, Maven and and LLVM [27]) require built software or a successful build Gradle) to the test by automatically executing their respective default build commands on top 200 Java projects from GitHub. process, and a large number of program analysis techniques Next, we focus on the 86 projects that failed this initial automated are based on these frameworks.
    [Show full text]
  • Static Code Analysis: a Buyer’S Guide
    WHITE PAPER STATIC CODE ANALYSIS: A BUYER’S GUIDE KEY DECISION CRITERIA FOR SELECTING TOOLS TO DEVELOP EMBEDDED SOFTWARE THAT IS MORE RELIABLE, SAFE, AND SECURE. www.programmingresearch.com TABLE OF CONTENTS OVERVIEW ..................................................................................................................... 3 WHAT IS STATIC CODE ANALYSIS? ............................................................................. 4 BENEFITS OF STATIC CODE ANALYSIS ...................................................................... 4 Reduces Risks Associated with Large, Complex Code Bases .............................. 4 Improves Security and Reduces Vulnerabilities..................................................... 4 Enhances Reliability and Safety ............................................................................ 5 Streamlines Processes .......................................................................................... 5 Saves Time and Reduce Costs through Early Detection and Remediation ........... 5 CHALLENGES OF SOURCE CODE ANALYSIS ............................................................ 6 Static Analysis is Not Dynamic .............................................................................. 6 Too Much Noise and Too Many False Results ...................................................... 6 Too Many Errors To Fix ......................................................................................... 7 Diffculties with Process Integration ......................................................................
    [Show full text]
  • Understanding and Improving Software Build Teams
    Understanding and Improving Software Build Teams Shaun Phillips Thomas Zimmermann Christian Bird University of Calgary Microsoft Research Microsoft Research Calgary, Alberta, Canada Redmond, WA, USA Redmond, WA, USA [email protected] [email protected] [email protected] ABSTRACT In real-world scenarios, a build process will likely consist of Build, creating software from source code, is a fundamental activity many more complex tasks. For instance, there may be scheduling in software development. Build teams manage this process and en- of build machines, build metric collection (such as time or power sure builds are produced reliably and efficiently. This paper presents consumption), security checks, so-called “smoke” tests to ensure an exploration into the nature of build teams—how they form, work, some baseline level of quality, language localization, reporting of and relate to other teams—through three multi-method studies con- build results, and triaging of build problems. All together, the ducted at Microsoft. We also consider build team effectiveness and collection of scripts, programs, and services that comprise and find that many challenges are social, not technical: role ambiguity, automate a build process is called a build system. knowledge sharing, communication, trust, and conflict. Our findings The build process is cyclical. Development teams write code validate theories from group dynamics and organization science, and which at some point is processed by the build system. The output, using a cross-discipline approach, we apply learnings from these the build, is deployed back to the development teams to verify the fields to inform the design of engineering tools and practices to changes [19] and the cycle begins again.
    [Show full text]
  • Mgr Inż. Marcin Kawalerowicz Classification Of
    MGR INŻ. MARCIN KAWALEROWICZ CLASSIFICATION OF AUTOMATIC SOFTWARE BUILD METHODS Summary: The process of creating working software from source code and other components (like libraries, database files, etc.) is called “software build”. Apart from linking and compiling, it can include other steps like automated testing, static code analysis, documentation generation, deployment and other. All that steps can be automated using a build description of some sort (e.g. script). This article classifies the automatic software build processes beginning at build script and reaching the various types of continuous integration. Keywords: software build automation, continuous integration, commanded integration, scheduled software build, triggered software build Chapter 1. Classification of automatic software build methods Software build process at a basic level consists of the translating the source code into working software. The translated code is often unit tested. More mature software build processes include more steps like complex testing, code metrics, code analysis, deployment [1] and other. So the build process contains a set of repetitive manual tasks that integrate the various components (source code, libraries, data files, etc.) into software. The tasks need to be executed in certain chronological order. Output of every task can vary from execution to execution, depending on the state the source code is in. So the next task to execute can vary depending on the output of the previous task. As a series of repetitive tasks the software build process done by humans is error prone [2]. To eliminate the uncertainty of the “human factor” the build process is often recommended to be automated [3]. Figure 1 shows the build automation diagram containing the classification of the automatic software build processes.
    [Show full text]
  • Plan, Code, Collaborate, and Ship Your Applications Faster with Azure Devops
    Guide Azure DevOps Services Plan smarter, collaborate better, and ship faster with a set of modern dev services. 0 Contents Azure DevOps user guide Overview What is Azure DevOps? Overview of services Where's VSTS? Quickstarts for users Code with Git Set up continuous integration & delivery Plan & track work Add & run manual tests View permissions Quickstarts for admins Sign up for Azure DevOps Create organizations Add users to a project or team Configure team resources Get started as an admin Install a Marketplace extension Tutorials for users Set favorites Follow work & pull requests Get started as a Stakeholder Tutorials for admins Change individual permissions Grant or restrict permissions to select tasks Concepts Key concepts Define organizations and projects Source control Backlogs, boards, & Agile tools Clients & tools Software development roles What's the difference? Azure DevOps Services vs. TFS Glossary How-to guides Sign-in to the web or a client Troubleshooting Troubleshoot connection TF31002: Unable to connect Troubleshoot network connections and whitelist addresses Get support, provide feedback Reference Permissions & access (Security) About access levels Keyboard shortcuts Navigate in Team Explorer FAQs Resources Web portal navigation Manage projects Security & identity Billing Visual Studio IDE Visual Studio Code Visual Studio for Mac What is Azure DevOps Services? 9/10/2018 • 2 minutes to read • Edit Online Azure DevOps Services Azure DevOps Services is a cloud service for collaborating on code development. It provides
    [Show full text]