Qualifying Software Engineers Undergraduates in DevOps - Challenges of Introducing Technical and Non-technical Concepts in a Project-oriented Course Isaque Alves Carla Rocha University of Bras´ılia (UnB) University of Bras´ılia (UnB) Bras´ılia, Brasil Bras´ılia, Brasil [email protected] [email protected] Abstract—The constant changes in the software industry, prac- that impact frequent and reliable release process, the student tices, and methodologies impose challenges to teaching and learn- should understand concepts such as continuous improvement, ing current software engineering concepts and skills. DevOps is product management over project management, small features, particularly challenging because it covers technical concepts, such as pipeline automation, and non-technical ones, such as team measurement, change, risk cost, short feedback cycle, commu- roles and project management. The present study investigates a nication, a culture of collaboration, and cross-functional/full- course setup to introduce these concepts to software engineering stack team, among others. Additionally, mastering technical undergraduates. We designed the course by employing coding concepts are essential to enable a continuous deployment to associate DevOps concepts to Agile, Lean, and Open source pipeline. Deployment pipeline, continuous deployment, ver- practices and tools. We present the main aspects of this project- oriented DevOps course, with 240 students enrolled it since its sioning, containerization, build automation, static code anal- first offering in 2016. We conducted an empirical study, with both ysis, continuous integration, testing automation, microservice a quantitative and qualitative analysis, to evaluate this project- architecture, automation tools, cloud services, and continuous oriented course setup. We collected the data from the projects runtime monitoring guarantee supports a stable and repeatable repository and students’ perceptions from a questionnaire. We deployment process. However, it is far from trivial to bring to mined 148 repositories (corresponding to 72 projects) and ob- tained 86 valid responses to the questionnaire. We also mapped a classroom context the complexity and the interdependence the concepts which are more challenging to students learn between these concepts and their practical implications. How from experience. The results evidence that first-hand experience to qualify engineers for DevOps practice is still an open facilitates the comprehension of DevOps concepts and enriches research question [11][3]. classes discussions. we present a set of lessons learned, which A commonly adopted strategy in software engineering may help professors better design and conduct project-oriented courses to cover DevOps concepts. courses to present complex concepts and develop technical Index Terms—DevOps, education, Open source, OSS, FOSS, students’ skills is to foster them to participate in Open Source Empirical software engineering, Agile software development, Software (OSS) projects as part of the course [14][15][16]. Emerging domains of software, Tools and environments. By working with OSS projects, students learn skills, practices, toolsets, and technologies aligned with the current software I. INTRODUCTION development landscape [17]. Such a scenario also enhances The development rate of the software industry has evolved students’ resumes and parallels software engineering concepts from delivering the software product in the waterfall model with real-world scenarios. arXiv:2102.06662v1 [cs.SE] 12 Feb 2021 to continuous delivering in agile and DevOps [1]. This con- This paper presents a project-oriented course setup introduc- stant evolution of software industry standards, practices, and ing both technical and non-technical DevOps concepts to un- methodologies imposes challenges to teaching and learning dergraduates. We employ coding to associate DevOps concepts software engineering [2][3]. Education programs must con- to Agile, Lean, and OSS practices and tools. Throughout the nect abstract concepts taught in the classroom to skills needed course, teams contribute to a capstone OSS project, adopting for software engineering practitioners [4][5][6][7][8]. How- these proposed practices in the project development cycle. ever, it is common to see undergraduate software engineering Many OSS communities have embraced DevOps tools and courses covering contents and concepts disconnected from practices. We adopt many of these automation tools in the current industry practices. projects. In the classroom, the professor introduces DevOps In this context, teaching DevOps is exceptionally challeng- theoretical concepts, correlates them with these practices, and ing. DevOps is a natural evolution of the agile movement [9] presents how they impact continuous delivery. [10], and it proposes a complementary set of practices and The insights obtained from this empirical research provide automation to allow the iterative delivery of software in short lessons learned for professors, students, and OSS communi- cycles effectively [11][12][13]. To understand the factors ties that want to take advantage of this process. The main contributions of this papers are the following: To create the course syllabus, we adopted the DevOps • We present a project-oriented course setup that associates conceptual map from the survey [11], organized into four DevOps concepts with practices and tools from Open categories: process, people, delivery, and runtime. Each cat- Source Community, Agile and Lean. Previous work on egory presents a set of concepts and how they relate to each the topic focused mostly on tools and automation of other in the DevOps perspective. Delivery and Runtime cate- the deployment pipeline, and DevOps still imposes chal- gories cover mostly technical aspects of DevOps, including: lenges due to its complexity [11]. We could not find Continuous delivery/deployment, deployment pipeline stages, a research paper that covers the teaching challenges of automation, open source, versioning, build management, static presenting the complexity of DevOps [3], to give an un- analysis, testing automation, continuous integration, build derstanding that non-technical aspects, such as communi- once, configuration management, microservices, rollback, con- cation and management, together with technical aspects, tinuous runtime monitoring, infrastructure as code, container- such as pipeline and provision automation, impacts the ization, cloud services, environments are similar, performance, continuous software delivery; availability, scalability, resilience, reliability, less human inter- • From the course data analysis, we present a set of vention. recommendations/lessons learned to guide professors, in- People and Process categories cover concepts for project structors, and practitioners. Based on the course data management and collaboration, including: frequent and reli- analysis, we present a set of lessons learned, which may able release process, agile/lean, short feedback cycle, prod- help professors better design and conduct project-oriented uct quality, small features, innovation, trust relationships, courses to cover DevOps concepts. continuous improvements, product management over project The rest of the paper is organized as follows. In Section management, risk and cost, a culture of collaboration, DevOps II we present our research question and the research method- role, aligning incentives, personal responsibility, failure as an ology. Section III presents the course setup. In SectionIV, opportunity for improvement, cross-functional, ”you build it, we present a quantitative analysis with data mining from you run it”, deployment triggered by dev, knowledge, tools, the course repository. Finally, in SectionVI, we present our processes, and practices. lessons learned from four years of course experience, and the related works in Section VII. The conclusions are in Section We excluded the concepts related to software maintenance IX. and evolution, like feature toggle, run time experiments, A/B testing, Canary release, chaos engineering, log management, II. METHOD and continuous post-production maintenance. They require In this section, we state the research question and the advanced projects, and it is out of the scope of a capstone process followed in conducting data analysis. We defined one project lifecycle. From this first filter, we obtained the list of research question that guided our study: DevOps concepts related to the software project development RQ. Can a project-oriented course help to teach both tech- cycle, as depicted in TableI. nical and non-technical DevOps concepts to undergraduates? Then, we listed agile, OSS, lean, and DevOps practices; This research question guides us to plan, execute, and we also listed the open source tools adopted by DevOps evaluate a project-oriented course focused on DevOps. We practitioners community to encourage students to use tools conducted empirical research to answer this research question. adopted in the industry. We employed coding to assign each First, we assigned practices and tools from agile, lean, and practice to at least one DevOps concept [20]. From codes to OSS to DevOps concepts. It guides us to plan the presented categories [21]: based on the initial list of practices (the codes), project-oriented course. we analyzed to identify the category of these practices. In the To evaluate the course syllabus and the relevance of these following, we present some examples of concepts