DOCSLIB.ORG

Seisio: a Fast, Efficient Geophysical Data Architecture for the Julia

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Seisio: a Fast, Efficient Geophysical Data Architecture for the Julia 1 SeisIO: a fast, efficient geophysical data architecture for 2 the Julia language 1∗ 2 2 2 3 Joshua P. Jones , Kurama Okubo , Tim Clements , and Marine A. Denolle 1 4 4509 NE Sumner St., Portland, OR, USA 2 5 Department of Earth and Planetary Sciences, Harvard University, MA, USA ∗ 6 Corresponding author: Joshua P. Jones ([email protected]) 1 7 Abstract 8 SeisIO for the Julia language is a new geophysical data framework that combines the intuitive 9 syntax of a high-level language with performance comparable to FORTRAN or C. Benchmark 10 comparisons with recent versions of popular programs for seismic data download and analysis 11 demonstrate significant improvements in file read speed and orders-of-magnitude improvements 12 in memory overhead. Because the Julia language natively supports parallel computing with an 13 intuitive syntax, we benchmark test parallel download and processing of multi-week segments of 14 contiguous data from two sets of 10 broadband seismic stations, and find that SeisIO outperforms 15 two popular Python-based tools for data downloads. The current capabilities of SeisIO include file 16 read support for several geophysical data formats, online data access using FDSN web services, 17 IRIS web services, and SeisComP SeedLink, with optimized versions of several common data 18 processing operations. Tutorial notebooks and extensive documentation are available to improve 19 the user experience (UX). As an accessible example of performant scientific computing for the 20 next generation of researchers, SeisIO offers ease of use and rapid learning without sacrificing 21 computational performance. 2 22 1 Introduction 23 The dramatic growth in the volume of collected geophysical data has the potential to lead to 24 tremendous advances in the science (https://ds.iris.edu/data/distribution/). Leveraging the data rev- 25 olution to gain knowledge that is useful for earthquake science, hydrology, industry, and climate 26 science requires new tools to help Earth scientists extract meaningful information from arbitrarily 27 large data sets. High-performance computing is necessary to manage the scale of these prob- 28 lems; however, this requires specialized training at the undergraduate and graduate levels, which is 29 rarely taught in undergraduate-level science curricula. On the other hand, open-source computing 30 languages (Python) and codes (e.g., ObsPy; Beyreuther et al (2010)) have standardized seismic 31 data processing and improved access to seismic data analysis for a new generation of seismolo- 32 gists. However, these tools suffer from slow computation time and inefficient memory allocation 33 at scale. Therefore, the geophysics community is in need of a computational framework that is 34 simultaneously easy to learn and efficient. 35 The Julia language combines the syntactic ease of high-level languages like MATLAB and Python 36 with the performance of FORTRAN and C. Developed for fast, efficient numerical computing, 37 Julia version 1.0.0 was released August 2018, while the first beta version appeared February 38 2012 (Bezanson et al., 2017, 2018). The language is known for impressive speed and compu- 39 tational efficiency: while still in beta testing, Julia became the fourth programming language 40 to achieve a petaflop, after FORTRAN, C, and C++ (Reiger et al., 2018; Perkel, 2019). De- 41 spite its relative youth, Julia supports a growing collection of open-source modules for numer- 42 ical and scientific computing. Julia wrappers to C, FORTRAN, R, and Python allow seamless 43 execution of external code, and third-party packages (https://github.com/JuliaInterop) extend in- 44 teroperability to C++, Java, Mathematica, and MATLAB, including the ability to read .mat files 45 (https://github.com/JuliaIO/MAT.jl). 3 46 2 SeisIO 47 The SeisIO package was created in May 2016 with the goal of rapid, efficient analysis of univariate 48 geophysical data in the Julia language, using comprehensible, uniform syntax, and simple but 49 powerful commands. Its design allows users to read univariate data from arbitrary instruments 50 (e.g., seismic, geodetic, gas flux) into a single structure, including gapped and irregularly-sampled 51 data. In the subsections below, we describe the capabilities of SeisIO, conduct benchmark tests, 52 and introduce tutorials. 53 2.1 Capabilities 54 SeisIO includes well-tested read support for many geophysical time-series formats (Table 1). Read- 55 ers for all formats but ASDF strictly use the Julia language; ASDF uses wrappers to libhdf5, written 56 in C. Current data processing operations include filling time gaps, removing the mean and linear 57 trend, band-pass filtering, instrument response translation and removal (i.e., flattening to DC), 58 resampling, cosine tapering, merging, seismogram differentiation/integration, and time synchro- 59 nization. Tools for online acquisition support FDSN services (station, event, and dataselect), IRIS 60 time-series requests, FDSN SeedLink, and the IRIS TauP interface (Crofwell et al., 1999). 61 SeisIO has been officially listed in the Julia package ecosystem since early 2019. Automated 62 testing with Travis-CI (https://travis-ci.org/) and AppVeyor (https://www.appveyor.com/) supports 63 Linux, Mac OS, and Windows installations. Code coverage estimates of 97-98% on Codecov 64 (https://codecov.io/) and Coveralls (https://coveralls.io/) exceed the 95% coverage threshold typical 65 of enterprise-level commercial software releases, yet both Julia and SeisIO are free. 4 66 2.2 Installation 67 Typical installation of the Julia language, SeisIO, and all dependencies requires three total steps: 68 1. Download and install the Julia language from https://julialang.org/downloads/ 69 • The Julia install directory will be denoted (juliaroot) hereafter. 70 • (juliaroot) is typically a pattern like /home/username/julia-v.v.v/ in 71 Linux, e.g., /home/josh/julia-1.1.0/. 72 2. Start the Julia command-line interface (CLI) with (juliaroot)/bin/julia 73 3. Type or copy: using Pkg; Pkg.add("SeisIO"); using SeisIO 74 Julia installs package dependencies automatically when Pkg.add is invoked. There is no need 75 for dedicated environments or session-specific user settings; however, FFT performance can some- 76 times be improved by starting Julia in parallel-ready mode with (juliaroot)/bin/julia 77 --procs auto. Total disk space required is typically under 4 GB: 300-400 MB for Julia; 4.2 78 MB for SeisIO v0.4.1; 300 MB for optional test and benchmark data; and 1-3 GB for a typical 79 set of scientific computing packages. The last space requirement is much lower for non-Windows 80 users who manually link existing libraries and software (e.g., BLAS, Conda, FFTW) to Julia, but 81 this is only recommended for experienced Linux users. 82 2.3 SeisIO Data Structure 83 SeisIO is designed around easy, fluid, and fast data access. For example, a complete sequence 84 of commands to download and process channel data can be executed in one function call with 85 keywords: 5 86 87 julia> S = get_data("FDSN", "UW.LON..BH?", src="IRIS", s="2019-01-01", t=3600, detrend=true, rr= 88 true, w=true) 89 90 SeisData with 3 channels (2 shown) 91 ID: UW.LON..BHE UW.LON..BHN ... 92 NAME: Longmire CREST broad-band Longmire CREST broad-band ... 93 LOC: 46.7506 N, -121.81 E, 853.0 m 46.7506 N, -121.81 E, 853.0 m ... 94 FS: 40.0 40.0 ... 95 GAIN: 7.51485e8 7.51485e8 ... 96 RESP: a0 1.0, f0 1.0, 1z, 1p a0 1.0, f0 1.0, 1z, 1p ... 97 UNITS: m/s m/s ... 98 SRC: http://service.iris.edu/fdsnws/da http://service.iris.edu/fdsnws/da ... 99 MISC: 4 entries 4 entries ... 100 NOTES: 2 entries 2 entries ... 101 T: 2019-01-01T00:00:00.010 (0 gaps) 2019-01-01T00:00:00.010 (0 gaps) ... 102 X: -1.511e+03 +4.669e+03 ... 103 -1.512e+03 +4.699e+03 ... 104 ... ... ... 105 +1.540e+03 +7.483e+02 ... 106 (nx = 144000) (nx = 144000) ... 107 C: 0 open, 0 total 108 110109 111 This example downloads 3600 seconds of data beginning 2019-01-01 00:00:00 (UTC) using FDSN 112 dataselect with the IRIS DMC server. The keyword ”detrend” removes the linear trend after down- 113 load; ”rr” removes (flattens to DC) the instrument response and replaces the .resp field of each 114 channel with an all-pass filter. The keyword ”w” writes the download directly to disk before pro- 115 cessing. Access to data properties is straightforward and intentionally simple: for example, in all 116 timeseries-data structures, the field .x holds univariate data. 117 2.4 Tutorials 118 A SeisIO tutorial is available from the project GitHub site, with three short, interactive Jupyter 119 notebooks designed to take 5-10 minutes each. A few additional commands in the Julia CLI are 120 required to run interactive notebooks: 6 using Pkg Pkg.add(["Dates", "IJulia"]) 121 using IJulia cd(dirname(pathof(SeisIO))*"/../tutorial/") jupyterlab(dir=pwd()) 122 The three tutorials are: 123 Part_1-Basic.pynb: introduction to SeisIO 124 Part_2-Data_Acquisition.pynb: downloading data & reading files 125 Part_3-Processing.pynb: data processing 126 Researchers familiar with MATLAB/Octave or Python will find Julia syntax intuitive and may 127 need only the language’s official documentation to begin coding. However, many Julia-language 128 tutorials can be downloaded from https://julialang.org/learning/ . 129 3 Benchmarking 130 We conduct a series of benchmark tests on a 64-bit personal computer equipped with an Intel 131 DH67CL motherboard, i7-2600 (3.4 GHz) CPU, and 16 GB Kingston DDR3 RAM, running Julia 132 v1.1.0 on 64-bit Ubuntu Linux 18.04.3 (kernel 5.0.0-29). File read tests (Table 2) use SeisIO v0.4.1 133 and BenchmarkTools.jl with 100 samples per benchmark and one evaluation per sample.
Load more

Recommended publications
  • Seismic Wave Analysis and Real-Time Monitor: User Manual and Reference Guide
    SWARM Seismic Wave Analysis and Real-time Monitor: User Manual and Reference Guide Version 3.1.0 February 2020 Table of Contents 1 Introduction .......................................................................................................................................... 6 1.1 About ............................................................................................................................................. 6 2 Getting Started ...................................................................................................................................... 6 2.1 System Requirements ................................................................................................................... 6 2.2 Installing SWARM .......................................................................................................................... 6 2.3 Running SWARM ........................................................................................................................... 7 3 Data Sources and Channels ................................................................................................................... 8 3.1 Introduction .................................................................................................................................. 8 3.2 General Usage ............................................................................................................................... 8 3.3 Data Source Types ........................................................................................................................
    [Show full text]
  • Open-Source ANSS Quake Monitoring System Software J
    Focus Section: Regional Seismic Networks in North America Open-Source ANSS Quake Monitoring System Software J. Renate Hartog*1, Paul A. Friberg2, Victor C. Kress1, Paul Bodin1, and Rayomand Bhadha3 Abstract ANSS stands for the Advanced National Seismic System of the U.S.A., and ANSS Quake Monitoring System (AQMS) is the earthquake management system (EMS) that most of its member regional seismic networks (RSNs) use. AQMS is based on Earthworm, but instead of storing files on disk, it uses a relational database with replication capability to store pick, amplitude, waveform, and event parameters. The replicated database and other features of AQMS make it a fully redundant system. A graphical user interface written in Java, Jiggle, is used to review automatically generated picks and event solutions, relo- cate events, and recalculate magnitudes. Add-on mechanisms to produce various post- earthquake products such as ShakeMaps and focal mechanisms are available as well. It provides a configurable automatic alarming and notification system. The Pacific Northwest Seismic Network, one of the Tier 1 ANSS RSNs, has modified AQMS to be com- patible with a freely available, capable, open-source database system, PostgreSQL, and is running this version successfully in production. The AQMS Software Working Group has moved the software from a subversion repository server hosted at the California Institute Cite this article as Renate Hartog, J., of Technology to a public repository at gitlab.com. The drawback of AQMS as a whole is P. A. Friberg, V. C. Kress, P. Bodin, and that it is complex to fully configure and comprehend. Nevertheless, the fact that it is very R.
    [Show full text]
  • Seisan = Earthquake Analysis Software
    SEISAN EARTHQUAKE ANALYSIS SOFTWARE FOR WINDOWS, SOLARIS, LINUX and MACOSX Version 12.0 Lars Ottem¨oller(1) Peter H. Voss(2) Jens Havskov(1) (1) Department of Earth Science (2) Geological Survey of Denmark and Greenland University of Bergen Øster Voldgade 10 Allgaten 41 1350 Copenhagen K 5007 Bergen Denmark Norway http://seisan.info Bergen June 08, 2021 2 Publisher University of Bergen Department of Earth Science Allegatan 41 NO-5007 Bergen NORWAY ISBN 978-82-8088-501-2 (PDF) Cover The photo on the cover show the sensor at the BSD station, in Denmark, spring 2021. Photograph: P. Voss. Citiation If you need to cite this manual : Ottem¨oller,L., Voss, P.H. and Havskov J. (2021). SEISAN Earthquake Analysis Software for Windows, Solaris, Linux and Macosx, Version 12.0. 607 pp. University of Bergen. ISBN 978-82-8088-501-2, URL http://seisan.info. Alternatively use: Havskov et al. [2020] Havskov, J., Voss, P.H. and Ottem¨oller,L. (2020). Seismological Observatory Software: 30 Yr of SEISAN. Seismological Research Letters, 91 (3): 1846-1852. DOI: https://doi.org/10.1785/0220190313 Contact See SEISAN mailing lists on page 6. Alternatively use: Lars Ottem¨[email protected] Peter H. Voss [email protected] Jens Havskov [email protected] Copyright ©2021 Ottem¨oller,Voss and Havskov. Contents 1 Introduction 1 1.1 Changes in the SEISAN version 12.0 (released 2021-06-08) . .2 1.1.1 A new Nordic Format, Nordic2 . .3 1.2 Information about SEISAN online . .5 2 Structure of SEISAN 7 2.1 Directories .
    [Show full text]
  • Empirical Study of Restarted and Flaky Builds on Travis CI
    Empirical Study of Restarted and Flaky Builds on Travis CI Thomas Durieux Claire Le Goues INESC-ID and IST, University of Lisbon, Portugal Carnegie Mellon University [email protected] [email protected] Michael Hilton Rui Abreu Carnegie Mellon University INESC-ID and IST, University of Lisbon, Portugal [email protected] [email protected] ABSTRACT Continuous integration is an automatic process that typically Continuous Integration (CI) is a development practice where devel- executes the test suite, may further analyze code quality using static opers frequently integrate code into a common codebase. After the analyzers, and can automatically deploy new software versions. code is integrated, the CI server runs a test suite and other tools to It is generally triggered for each new software change, or at a produce a set of reports (e.g., the output of linters and tests). If the regular interval, such as daily. When a CI build fails, the developer is result of a CI test run is unexpected, developers have the option notified, and they typically debug the failing build. Once the reason to manually restart the build, re-running the same test suite on for failure is identified, the developer can address the problem, the same code; this can reveal build flakiness, if the restarted build generally by modifying or revoking the code changes that triggered outcome differs from the original build. the CI process in the first place. In this study, we analyze restarted builds, flaky builds, and their However, there are situations in which the CI outcome is unex- impact on the development workflow.
    [Show full text]
  • Usage, Costs, and Benefits of Continuous Integration in Open
    Usage, Costs, and Benefits of Continuous Integration in Open-Source Projects Michael Hilton Timothy Tunnell Kai Huang Oregon State University University of Illinois at University of Illinois at [email protected] Urbana-Champaign Urbana-Champaign [email protected] [email protected] Darko Marinov Danny Dig University of Illinois at Oregon State University Urbana-Champaign [email protected] [email protected] ABSTRACT CI using several quality metrics. However, the study does Continuous integration (CI) systems automate the compi- not present any detailed information about the use of CI. lation, building, and testing of software. Despite CI rising In fact, despite some folkloric evidence about the use of CI, as a big success story in automated software engineering, it there is no systematic study about CI systems. has received almost no attention from the research commu- Not only do we lack basic knowledge about the extent nity. For example, how widely is CI used in practice, and to which open-source projects are adopting CI, but also what are some costs and benefits associated with CI? With- we have no answers to many other important questions re- out answering such questions, developers, tool builders, and lated to CI. What are the costs of CI? Does CI deliver researchers make decisions based on folklore instead of data. on the promised benefits, such as releasing more often, or In this paper, we use three complementary methods to helping make changes (e.g., to merge pull requests) faster? study in-depth the usage of CI in open-source projects. To Are developers maximizing the usage of CI? Despite the understand what CI systems developers use, we analyzed widespread popularity of CI, we have very little quantita- 34,544 open-source projects from GitHub.
    [Show full text]
  • Spice Program Open Source 2015 Learnings
    Spice Program Open Source 2015 learnings Please see this Futurice Spice Program blog post to learn the context for this list. ​ ​ Learned even more about practical problems in measuring time. Direct uploads from browser to AWS S3. Basics of Polymer. React + Redux (lots of it). Basic info about Google Cloud Services. Practical usage of Google Cloud Storage (the Google's S3 equivalent). Heroku Pipelines. Improved & more modular Angular application structure formation. Webpack, JSON Web Token implementation on the backend, and ES6. Actually learning how Promises should be used in JavaScript. Some knowledge about Dokku, the host­it­yourself­Heroku. Got practical experience and more understanding about Node.js, npm, and bacon.js. How the Azure Web App continuous deployment works. A bit on how Travis CI operates under the hood. How to make Joomla templates. Learned about Let's Encrypt on Apache with vhosts. Learned more about Web MIDI API and RxJS. Also more about ES6 and setting up a compact development environment for that. Learned basics of RxJS. Learned a lot about creating a reusable library with JS/browserify. Learned some Cycle.js. Learned a little about Node.js testing. Learned the EMV chip card specification on very detailed level by implementing large part of the specification. I'm using the EMV knowledge on daily basis in my current client project. Improved my Clojure language skills. Learned to work with the Go programming language. Wrote an OSS tool that I later then used in a customer project. I learned better ways to use Dagger 2 in Android and trial and evaluate other app architectural patterns based around this.
    [Show full text]
  • Open Source Best Practices: from Continuous Integration to Static Linters
    THE MOLECULAR SCIENCES SOFTWARE INSTITUTE - LOGO CONCEPTS ROUND 2 Open Source Best Practices: From Continuous Integration to Static Linters Daniel G. A. Smith and Ben Pritchard The Molecular Sciences Software Institute !1 @dga_smith / @molssi_nsf 1. 3 The Molecular Sciences Software Institute (MolSSI) •Designed to serve and enhance the software development efforts of the broad field of computational molecular science (CMS). •Funded at the institute level under the Software Infrastructure for Sustained Innovation (SI2) initiative by the National Science Foundation. •Launched August 1st, 2016 •Collaborative effort by: •Virginia Tech •U.C. Berkeley •U. Southern •Rice U. •Stanford U. California •Stony Brook U. •Rutgers U. •Iowa State U. !2 The Molecular Sciences Software Institute (MolSSI) •Software Infrastructure •Build open-source frameworks and packages for the entire CMS community •Education and training •Online materials and training guides •Host a number of targeted workshops, summer schools, and other training programs •Community Engagement and Leadership •Interoperability standards and best practices !3 •Data and API standardization targets Talk Motivation •Many terms thrown around: •Best Practice •Manage Code •DevOps •Build and Package •Open Source •“Just get the physics •Software Sustainability working” •“Incorrect Software” “Best” is subjective and depends on your own community. Lets discuss practical applications and how to make our lives easier! !4 Open Source Open Source software is software that can be freely accessed, used,
    [Show full text]
  • Datadeps.Jl: Repeatable Data Setup Journal of for Reproducible Data Science
    White, L, et al. 2019 DataDeps.jl: Repeatable Data Setup Journal of for Reproducible Data Science. Journal of Open Research open research software Software, 7: 33. DOI: https://doi.org/10.5334/jors.244 SOFTWARE METAPAPER DataDeps.jl: Repeatable Data Setup for Reproducible Data Science Lyndon White, Roberto Togneri, Wei Liu and Mohammed Bennamoun The University of Western Australia, Crawley, Western Australia, AU Corresponding author: Lyndon White ([email protected]) We present DataDeps.jl: a julia package for the reproducible handling of static datasets to enhance the repeatability of scripts used in the data and computational sciences. It is used to automate the data setup part of running software which accompanies a paper to replicate a result. This step is commonly done manually, which expends time and allows for confusion. This functionality is also useful for other packages which require data to function (e.g. a trained machine learning based model). DataDeps.jl simplifies extending research software by automatically managing the dependencies and makes it easier to run another author’s code, thus enhancing the reproducibility of data science research. Keywords: data management; reproducible science; continuous integration; software practices; dependency management; open source software; JuliaLang 1 Introduction In the movement for reproducible sciences there have These tasks are automatable and therefore should be been two key requests upon authors: 1. Make your automated, as per the practice “Let the computer do the research code public, 2. Make your data public [3]. In work” [7]. practice this alone is not enough to ensure that results can DataDeps.jl handles the data dependencies, while Pkg1 be replicated.
    [Show full text]
  • Code Coverage & Continuous Integration
    Code Coverage & Continuous Integration ATPESC 2019 Jared O’Neal Mathematics and Computer Science Division Argonne National Laboratory Q Center, St. Charles, IL (USA) July 28 – August 9, 2019 exascaleproject.org License, citation, and acknowledgments License and Citation • This work is licensed under A CreAtive Commons Attribution 4.0 InternAtionAl License (CC BY 4.0). • Requested citAtion: Jared O’NeAl, Code CoverAge & Continuous IntegrAtion, in Better Scientific SoftwAre TutoriAl, Argonne TrAining ProgrAm on Extreme-Scale Computing (ATPESC), St. ChArles, IL, 2019. DOI: 10.6084/m9.figshAre.9272813. Acknowledgements • This work wAs supported by the U.S. DepArtment of Energy Office of Science, Office of Advanced Scientific Computing ReseArch (ASCR), And by the Exascale Computing Project (17-SC-20-SC), A collAborAtive effort of the U.S. DepArtment of Energy Office of Science And the NAtionAl NucleAr Security AdministrAtion. • This work wAs performed in pArt At the Argonne NAtionAl LAborAtory, which is mAnAged by UChicago Argonne, LLC for the U.S. DepArtment of Energy under ContrAct No. DE-AC02-06CH11357 • AliciA Klinvex 2 ATPESC 2019, July 28 – August 9, 2019 CODE COVERAGE How do we determine what other tests are needed? Code coverage tools • Expose parts of the code that aren’t being tested • gcov o standard utility with the GNU compiler collection suite o Compile/link with –coverage & turn off optimization o counts the number of times each statement is executed • lcov o a graphical front-end for gcov o available at http://ltp.sourceforge.net/coverage/lcov.php
    [Show full text]
  • Continuous Testing & Integration
    Continuous Testing & Integration Philippe Collet (avec beaucoup de M1 PROJET - 2019-2020 (images: pixabay) 1 slides de Guilhem. Molines) Delivering Software P. C o l l e t 2 Delivering software… • In 1995, MS released Windows 95 – Major breakthrough for consumers – Real multi-tasking with background apps • How many service packs were released? 3 Delivering software… • In 1995, MS released Windows 95 – Major breakthrough for consumers – Real multi-tasking with background apps • How many service packs were released? à 2 (yes, two) (6 months and 1 year after release) 4 Delivering software… • In 2015, MS released Windows 10 – 4 editions – Continuous, forced updates • How many service packs were released? 5 Delivering software… • In 2015, MS released Windows 10 – 4 editions – Continuous, forced updates • How many service packs were released? à couldn’t count… (1 GB on first day) 6 What is the most important thing you need to deliver software continuously? Trust 7 How do you get trust in your software delivery? • Good architecture • Development guidelines • Project management • Controlled processes • Traceability of requirements • Automation • Repeatable pipeline • Testing • Testing • And… Testing 8 Software Delivery Life Cycle (SDLC) In a nutshell P. C o l l e t 9 Single student, simple project • Ant • Maven? • SCM? • Code and click 10 Group of students, simple project • Ant / Maven • Github • Code and click • Build on local machine • Ticket management ? 11 Industrial complex project • 20 – 200 contributors • One release / year • One patch / month • > 20M LOCs • Deployment complexity • à What do you need? 12 Industrial complex project • Componentization • Independence of builds • Each component tested • Clear quality indicators • Requirement traceability • Fast builds • Each contributor only builds what they code • Deployment automation 13 Software quality • What is it? • No Defects? • Or….
    [Show full text]
  • I'm Leaving You, Travis: a Continuous Integration Breakup Story
    I’m Leaving You, Travis: A Continuous Integration Breakup Story David Gray Widder, Michael Hilton, Christian Kästner, Bogdan Vasilescu Carnegie Mellon University, USA ABSTRACT it’s natural for different projects to have different needs, as wellas Continuous Integration (CI) services, which can automatically build, for a project’s CI needs to change over time, as developers navigate test, and deploy software projects, are an invaluable asset in dis- these tradeoffs [2, 27]. Simply stated, one size does not fitall. tributed teams, increasing productivity and helping to maintain As a realization of the diversity of CI needs, the marketplace of code quality. Prior work has shown that CI pipelines can be sophisti- available CI tools in open-source software (OSS) is booming. While cated, and choosing and configuring a CI system involves tradeoffs. not long ago Jenkins and Travis CI were the only broadly used As CI technology matures, new CI tool offerings arise to meet the client-side and cloud-based CI services, respectively, there are now 1 distinct wants and needs of software teams, as they negotiate a 17 CI services that integrate with GitHub directly, and online 2 path through these tradeoffs, depending on their context. In this discussions are explicit about the need for tailored CI solutions. paper, we begin to uncover these nuances, and tell the story of To better support developers looking for bespoke CI solutions as open-source projects falling out of love with Travis, the earliest well as designers and builders of CI tools, in this paper we begin to and most popular cloud-based CI system.
    [Show full text]
  • Download Download
    p-ISSN : 2443-2210 Jurnal Teknik Informatika dan Sistem Informasi e-ISSN : 2443-2229 Volume 7 Nomor 1 April 2021 Continuous Integration and Continuous Delivery Platform Development of Software Engineering and Software Project Management in Higher Education http://dx.doi.org/10.28932/jutisi.v7i1.3254 Riwayat Artikel Received: 5 Januari 2021 | Final Revision: 15 Maret 2021 | Accepted: 24 Maret 2021 Sendy Ferdian#1, Tjatur Kandaga#2, Andreas Widjaja#3, Hapnes Toba#4, Ronaldo Joshua#5, Julio Narabel#6 Faculty of Information Technology, Universitas Kristen Maranatha, Bandung, Indonesia [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] Abstract — We present a report of the development phase of a likely to be very diverse and composed of various tools, platform that aims to enhance the efficiency of software commercially or open-source technologies, therefore project management in higher education. The platform managing application delivery lifecycle is difficult because accommodates a strategy known as Continuous Integration and Continuous Delivery (CI/CD). The phase consists of of those complexities. Dealing with those daunting and several stages, followed by testing of the system and its complex tasks, one solution is using the DevOps [3] [4] deployment. For starters, the CI/CD platform will be deployed approach. for software projects of students in the Faculty of Information The relatively new emerging DevOps is a culture where Technology, Universitas Kristen Maranatha. The goal of this development, testing, and operations teams join together paper is to show a design of an effective platform for collaborating to deliver outcome results in a continuous and continuous integration and continuous delivery pipeline to accommodate source code compilation, code analysis, code effective way [2] [5].
    [Show full text]