Utilizing Engineering Education Support System ALECSS at an Actual Experiment: A Case Study

Mika Ohtsuki and Tetsuro Kakeshita Department of Information Science, Saga University, Saga, Japan

Keywords: Education, DevOps Tool, Software Verification, Coding Style Checking, Execution Checking, Static Checking, Peer Review.

Abstract: We have proposed a software engineering education support system named ALECSS in our previous paper. ALECSS utilizes various DevOps tools such as Jenkins, Git, JUnit, Checkstyle and FindBugs to automatically check student’s programs from various viewpoints and to quickly provide feedbacks to the students. At the same time, ALECSS collects student’s log so that a teacher can easily observe the status of each student and/or each project team to improve software engineering education. In this paper, we utilize ALECSS at an actual software development experiment for self and peer review of the . Students are grouped into project teams and each student can view summary pages for the student or the team containing the messages generated by the DevOps tools integrated into ALECSS. We also collected feedback from the students and received many positive comments.

1 INTRODUCTION Evaluation Cycle Support System). ALECSS automatically checks source codes submitted by a It is quite important to utilize contemporary software students and/or a team from various viewpoints and development tools to realize practical software returns feedbacks to them. Various DevOps tools, engineering education (Nandigam, 2008). It is also such as JUnit, Git, Ant, Checkstyle, FidBugs and important to teach management of quality, cost and Jenkins, are utilized for the checking and for the delivery (QCD) of software development. integration of the checking tools. We also added Contemporary software engineering education original scripts to ALECSS for the checking of test requires collaboration of students as a team since the code and Git working status. A student or a team can software size is typically too large to work on a check their codes quickly and can improve them student. promptly by utilizing ALECSS. At the same time, the We are teaching collaborative software teacher can easily observe progress of the students development at the third academic year of our and the team. department majored in . In our We utilized ALECSS to an actual software class, a student team faces with many problems. For development experiment in the 2018 spring semester. example, it is hard to see and control progress of the In this paper, we report the results of the experimental software development project without appropriate application. Although the students use ALECSS for sharing of the known problems and progress of the the first time, their evaluation of ALECSS is quite software development tasks at each source code good. among team members. It is also necessary to ensure In Section 2, we shall explain the major functions QCD of the software. In order to cope with such of ALECSS. We next explain the software problems, realistic software developers are shifting to development experiment in Section 3 and how to utilize various DevOps tools (Allspaw, 2009; Bass, utilize ALECSS in the experiment in Section 4. We 2015). present and discuss the result of our evaluation in In our previous paper (Ohtsuki, 2016), we Sections 5 and 6. In section 7, we show the related proposed a software engineering education support works and compare them with our contribution. system named ALECSS (Automated Learning and

367

Ohtsuki, M. and Kakeshita, T. Utilizing Software Engineering Education Support System ALECSS at an Actual Software Development Experiment: A Case Study. DOI: 10.5220/0007723203670375 In Proceedings of the 11th International Conference on Computer Supported Education (CSEDU 2019), pages 367-375 ISBN: 978-989-758-367-4 Copyright c 2019 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved

CSEDU 2019 - 11th International Conference on Computer Supported Education

Figure 1: Entire structure of ALECSS.

2 SOFTWARE ENGINEERING student number or project name at several exercises. Therefore, it is necessary to have scripts to generate EDUCATION SUPPORT appropriate names from student number and/or SYSTEM ALECSS project name. Thus we can generate build.xml utilizing the scripts for each exercise of the ALECSS is developed using various DevOps tools experiment. such as Git, Jenkins, Ant, Checkstyle, FindBugs and JUnit as well as additional scripts for further checking 2.2 Coding Standard Checking of the student’s code. Figure 1 illustrates the entire structure and the behaviour of ALECSS. Coding standard checking ensures that the submitted A student can edit and submit a source program to Java program keeps one of the default coding style ALECSS by uploading the file(s) to the Git repository definitions (Sun Code Conventions and Google Java from the integrated development environment style). The checking is performed by executing Eclipse. Then Git notifies the file uploading to coding style checker Checkstyle. Execution of Jenkins. Jenkins automatically starts the build tool Checkstyle is defined as an Ant task by using the Ant by utilizing the submission as a trigger. Ant is taskdef clause in build.xml. controlled by the configuration file build.xml which contains setup and execution commands of the 2.3 Compile Checking various DevOps tools. The checking results are collected by Jenkins at the activity log. Then the Compile checking is a prerequisite of all other student and student team can browse the checking checking. ALECSS executes a standard compilation result and the status report to improve their source by invoking the javac command as an Ant task and code. The teacher can also browse the same report to can show the compilation result of the submitted Java understand status of each project. program. If the compilation fails, students can Various types of evaluation criteria can be browse the error messages on Jenkins. automatically checked using ALECSS. The criteria and the implementation of the checking mechanism 2.4 Output Result Checking are explained at the rest of this section. The output result checking is performed to ensure that 2.1 File Structure Checking the output of the submitted program matches to the output defined by the specification. ALECSS We can perform the existence checking of the executes a compiled Java program and record the required files by using condition tag and available tag output to the log which can be observed on Jenkins. defined as Ant tasks. File Structure Checking is Furthermore, we prepare a script to compare the required for all exercises and the file names are output log with a predefined file. different depending on the exercise. Furthermore, file Such script can be implemented by using the diff and folder names are assigned depending on the command if the result is fixed. For the case that the

368

Utilizing Software Engineering Education Support System ALECSS at an Actual Software Development Experiment: A Case Study output depends on student number, student name or 2.9 Static Code Checking project name, we develop a special script using the same technique as explained for File Structure Static code analysis tool FindBugs is utilized for the Checking. checking to detect pitfalls which can be observed within a Java source code. Execution of FindBugs is 2.5 Git Work Execution Checking defined as an Ant task as in the case of Checkstyle.

Git maintains a commitment log storing four types of Git actions: file addition (add and commit), file 3 COOPERATIVE SOFTWARE deletion (remove and commit), file update (edit and commit), revert (return to a former commitment DEVELOPMENT state). The Git work execution checking examines EXPERIMENT the Git log to confirm that a student correctly performs the required Git operations. The Our cooperative software development experiment is commitment log can be accessed by the Git log provided for the undergraduate student at the third command. We are developing a script for the Git academic year. The experiment is a compulsory work execution checking. subject for graduation and usually about 60 students are enrolled to the experiment each year. Our depart- 2.6 JUnit Execution Checking ment is accredited as a computer science program and the students have learned software engineering and JUnit is utilized to check whether the subroutines in basic Java programming before the experiment. The the submitted program return correct values. experiment consists of fifteen weeks of 3 hour Teachers need to provide a set of test codes executed exercises. Table 1 represents the experiment plan. by JUnit for the checking. JUnit can be executed as an Ant task with a junit tag. We can obtain the log Table 1: Experimental plan. using the task and can observe the number of Week Description successes/failures of the unit test on Jenkins. 1 Setting up software development environment (Git and Eclipse) 2.7 Test Case Null Implementation 2 Git Exercise Checking 3 Java Exercise (including Checkstyle and Javadoc) Our experiment also contains exercises to develop 4-5 JUnit Exercise 6 Introduction to Group Exercise (Group test code. If a student develops an empty test code, formation, Ice Breaking and Explanation of any test will succeed in the JUnit framework. The test Requirements) case null implementation checking detects such 7-9 Implementation (First Iteration) empty test cases. This can be implemented by Introduction to ALECSS (at Week 9) counting the number of lines in each test case method. 10 Peer Review (First Iteration) We have developed a Java language parser utilizing 11-13 Explanation of Additional Requirements JavaCC to extract a test case method and a script Implementation (Second Iteration) counting the number of lines of the test case method Student Survey (at Week 12) (Koga, 2018). The parser and the script are defined 14 Peer Review (Second Iteration) as Ant tasks for automatic execution. 15 Bug Fix and Second Student Survey Students work on individual exercises at the first 2.8 Test Code Validation 5 weeks. We introduce Git, Eclipse, Checkstyle, Javadoc and JUnit during these weeks. The test code validation is executed to detect The group exercises starts at week 6. Group incorrect test code which succeeds for any input. In exercises are carried out using the baseline project order to implement such validation, we prepare initially distributed to the 8 student teams (Cherry, project code which all tests fail. The project codes Dandelion, Lily, Peach, Plum, Rose, Sunflower, and are copied to the working area of the student in order Violet). Each team consists of 7-8 students. The group to confirm that the test code developed by the student exercise consists of two iterations. Each of the correctly fails for the project code. iterations contains three weeks for implementation

369

CSEDU 2019 - 11th International Conference on Computer Supported Education

exercise and one week for peer review by other students. We applied ALECSS to the experiment in 2018. At first, we introduce ALECSS at week 9 so that students start using ALECSS from the week. At week 9, student teams can only check their own project using ALECSS. However they can also utilize ALECSS to check projects developed by other teams at week 10 by allowing access to other project. Then at the end of the experiment, student teams utilize ALECSS for the second peer review (at week 14) in the second iteration. We performed the second student survey and collected the final project data after week 15 where detected bugs are expected to be fixed. In the peer review, a group reviews another team’s Figure 2: Result of static code checking. project assigned randomly by the teacher and reports review result containing detected issues. The types of When a student selects a category in Figure 2, the issues required to be reported are coding standard warnings belonging to the selected category are violations, portion of algorithms which are not bugs shown (Figure 3). Each warning is represented by the but contain some problems, software bugs or class name and the line number at the File column. unimplemented requirements. The reported issues Priority, author and commit ID are also represented. are checked by the teachers whether they are reasonable. A team gets some score when the team reports reasonable issues and the team loses the same score when the issue is reported. The score for each issue is defined as follows depending on the importance of the issues. . 1/3 : Coding Style Violation . 1-3: Problems in Algorithm . 2-5: Software Bugs and Unimplemented Specification Figure 3: FindBugs warnings belonging to the The number of issues which can be reported by a CORECTNESS category. team is at most twenty so that the score to be earned or lost is bounded. When a student selects a warning message in Fig. 3, the corresponding code will be shown (Figure 4). Jenkins also shows the detailed explanation for the 4 UTILIZING ALECSS AT THE selected warning message. EXPERIMENT

ALECSS can be executed either automatically or manually. We have already explained the automatic execution in Section 2. A student or a team can also execute ALECSS manually after selecting a project. The manual checking function is used at the peer reviews. Figure 4: Code corresponding to a FindBugs warning. Figure 2 represents a result of static code checking. The warning messages generated by Figure 5 represents messages of the coding FindBugs are classified by categories. The red bars standard checking. This time the warning messages represent the number of warnings with high priority, are classified based on the detected file. It is also while the yellow bars represent the number of possible to browse the warning messages classified warnings with normal priority. Each category has a by the categories as in the case of Fig. 2. priority represented by the color of the corresponding bar.

370

Utilizing Software Engineering Education Support System ALECSS at an Actual Software Development Experiment: A Case Study

5 EFECT OF INTRODUCING ALECSS TO THE EXPERIMENT

Table 2 illustrates the size of each project code at the end of the first iteration (week 10). The number of files in each project is the same since file organization is determined by the teacher. Although implement- tation is different among the teams, the total code size is approximately 2000 lines. The readers can observe that we are using a reasonably large scale code for the experiment. The number of test cases of the project is 27 as Figure 5: Result of coding style checking. instructed from the teacher in the first iteration.

Figure 6 represents a result of JUnit execution Table 2: Project size at week 10. checking. The readers can find that one test failed Project Total # Average # of while the remaining three tests succeeded. It is also # of Files possible to browse the corresponding source code by Team of lines lines per file selecting the name of a test method. Cherry 19 2002 105.4 Dandelion 19 1997 105.1 Lily 19 1945 102.4 Peach 19 1962 103.3 Plum 19 1909 100.5 Rose 19 1703 89.6 Sunflower 19 1981 104.3 Violet 19 2019 106.3

Table 3: Project size at the end of the experiment. Figure 6: Result of JUnit execution checking. Average # Project # of Total # of # of Test of lines lines Cases Team Files per file Cherry 26 3130 120.4 44 Dandelion 25 2906 116.2 44 Lily 25 2712 108.5 37 Peach 26 3097 119.1 45 Plum 26 3159 121.5 52 Rose 25 3359 134.4 53 Sunflower 26 3227 124.1 43 Violet 25 3071 122.8 42

Table 3 represents the project size at the end of the Figure 7: Example result of test case null implementation experiment (week 15). Although file organization is checking. determined by the teacher at the first iteration, several groups have extra classes. Implementation of the Figure 7 represents the result of test case null code is more different among the teams than the first implementation checking. The result contains the test iteration. The project size is significantly larger than class name, test method name and the number of Table 2. The number of test cases is distributed statements in the test method. In this case, for between 37 and 53 since the students had to add more example, the method “testParse02” contains only one than four originally test cases for each class added statement so that we can guess that the method is still during the second iteration. at the initial state and is not implemented yet.

371

CSEDU 2019 - 11th International Conference on Computer Supported Education

Table 4 represents the number of warnings Table 5: The number of warnings and failed test cases of generated by Checkstyle, FindBugs and the number the second iteration. of failed JUnit test cases at the time before review Checkstyle FindBugs Failed Test Cases (week 9) and at the review (week 10). The number of Project Warnings Warnings test cases of all projects is 27 as instructed from the Team Week Week Week Final Final Final teacher in the first iteration. “N/A” means that the 14 14 14 project failed to be compiled and the project code was Cherry 169 178 33 33 11 3 not checked by the corresponding tool. Dandelion 134 130 19 13 15 6 Lily 120 156 16 29 2 3 Table 4: The number of warnings and failed test cases of Peach 186 188 25 20 7 8 the first iteration. Plum 233 194 24 24 11 10 Rose 101 104 45 38 9 5 Checkstyle FindBugs Failed Test Cases Sunflower 141 137 43 42 4 3 Warnings Warnings Project Violet 126 126 33 29 7 3 Team Week Week Week Week Week Week 9 10 9 10 9 10 Tables 6 and 7 represent the review score and the Cherry 141 69 1 1 7 0 number of detected issues categorized by the type of Dandelion N/A 66 N/A 1 N/A 0 issues at the peer review in the first and the second Lily 95 74 5 7 2 4 iterations respectively. The review score is calculated according to the rules which we have explained at the Peach 100 81 1 1 4 3 end of Section 3. The difference of the review score Plum 97 76 3 7 8 1 among the reviewer team is caused mainly by the Rose 78 64 2 1 7 0 utilization of ALECSS by the teams. Sunflower 73 63 7 2 0 0 Violet 105 N/A 2 N/A 10 N/A Table 6: Review score and the detected issues at the first iteration. The readers can observe that the number of # of # of Algo- # of Reviewer Review Reviewed warnings and the number of failed test cases are Coding rithm Bugs Team Score Project decreasing from week 9 to week 10 in most cases in viola-tions issues etc. the first iteration. This is the effect of peer review. Sunflower 2 7 0 0 Cherry Each team utilize ALECSS to detect and fix software Cherry 1 5 0 0 Dandelion bugs and potential problems by utilizing ALECSS. Rose 9 10 3 1 Lily The number of Checkstyle warnings is still large at Dandelion 14 4 5 2 Peach week 10. We consider that Checkstyle detects Peach 7 6 0 3 Plum possible coding style errors regardless of their Violet N/A N/A N/A N/A Rose priorities and student teams do not have enough time Lily 3 8 1 0 Sunflower to fix all the warnings with low priority. Plum 8 5 2 1 Violet However the number of warnings and failed test cases are increasing in some cases. One reason of this Table 7: Review score and the detected issues at the second is that some project team did not finish the iteration. implementation at week 9 and continuously # of # of Algo- # of Reviewer Review Reviewed Coding rithm Bugs implement the code just before the peer review. Team Score Project Another reason is that some project team viola-tions issues etc. misunderstand our instruction and failed to correctly Sunflower 14 4 1 4 Cherry Cherry 11 2 2 3 Peach implement the test code at week 9. Rose 21 3 2 7 Lily Table 5 shows the number of warnings generated Dandelion 15 1 2 5 Rose by Checkstyle, FindBugs and the number of failed Peach 5 4 0 2 Dandelion JUnit test cases at the review (week 14) in the second Violet 16 0 5 4 Plum iteration and at the end of the lecture (final). The Lily 10 3 4 4 Violet readers can observe that the number of warnings and Plum 6 2 3 1 Sunflower the number of failed test cases are decreasing from week 14 to final in all cases. This is the effect of peer The results of Pearson correlation analysis among review as same as the first iteration. the numbers of warnings, the numbers of failed test cases and review score are illustrated in Table 8. Here, C10 and C14 are the number of Checkstyle warnings

372

Utilizing Software Engineering Education Support System ALECSS at an Actual Software Development Experiment: A Case Study at week 10 and 14. F10 and F14 are the number of 6.1 Utilization of ALECSS for Their FindBugs warnings at week 10 and 14. And T10 and Own Project F14 are the number of failed test cases at week 10 and 14. Review1 and Review2 mean the review scores of 47 students (82.4%) at week 10 answered that they iteration 1 and 2. In iteration 1 the numbers of used ALECSS to check their own project. The ratio Checkstyle warnings and the failed test cases are is quite high considering that they use ALECSS for correlated with review results. In the second iteration, the first time at week 9. The reasons of not using however, these are not correlated. It implies that the ALECSS at this week are as follows. Since some of students could not detect the faults in the target codes the teams are still working on the implementation of in the second iteration because the code became more the code, they could not have enough time to complicated, and the test cases were not enough understand and utilize ALECSS for the checking of because they were designed by the students although their own code. the test case specifications were supplied by the The students also replied the following questions teacher in the first iteration. in five levels.

Table 8: Pearson correlation analysis among results Q1. Was ALECSS useful to check your own project? Q2. Did you quickly obtain the result from ALECSS? Result correlation t-value Df p-value Pair Q3. Did you get the detailed result from ALECSS? C10, 0.903 4.204 4 0.014 Review1 F10, 0.282 0.587 4 0.589 Review1 T10, 0.849 3.207 4 0.033 Review1 C14, -0.455 -1.251 6 0.257 Review2 F14, 0.0635 0.156 6 0.881 Review2 T14, -0.395 -1.053 6 0.333 Review2 Figure 8: Evaluation of ALECSS at week 9.

As shown Figure 8, we obtained positive answers 6 STUDENT SURVEY (5 or 4) from 89% of the students for usefulness, 65% for quick feedback and detailed checking. The After the peer review at weeks 10 and 15, we number of negative answers (2 or 1) is quite few. conducted a student survey to evaluate ALECSS. The At week 15 after the second iteration, 55 students survey contains the following questions. (89%) answered that they used ALECSS to check their own project. Figure 9 shows the results for the . Did you utilize ALECSS to check your evaluation of ALECCS. project code at week 9? If no, why? . Did you utilize ALECSS to find issues of another project at the peer review (week 10)? If no, why? . How useful is ALECSS? . Did you quickly obtain the feedback from ALECSS? . Did you get a detailed result from ALECSS?

. Please provide comments to improve ALECSS. Figure 9: Evaluation of ALECSS at week 15. 57 of the 62 enrolled students at week 10 and all We obtained positive answers (5 or 4) from 83.6% of them at week 15 answered the survey. We shall of the students for usefulness, 76.3% for quick report the results of the survey in the succeeding feedback and 81.8% for detailed checking. The utiliza- subsections. tion of ALECSS increased during the second iteration.

373

CSEDU 2019 - 11th International Conference on Computer Supported Education

6.2 Utilization of ALECSS to Review . Very useful since feedbacks are quickly Other Project provided. . Hard to understand how to use ALECSS. 52 students (91.2%) replied that they used ALECSS to find issues in another project at the peer review . I found some unexpected behaviour of (week 10). Thus the percentage of the students is ALECSS. Need to be fixed. increased compared to the result at week 9. The reasons of not utilizing ALECSS at this week are as follows. 7 RELATED WORKS . Still did not understand ALECSS messages. Utilization of software tools for software engineering . Did not need ALECSS since Checkstyle education has a long history (Douce, 2005; Ihantola, reports possible coding style violations when 2010; Ala-Mutka, 2007; Caiza, 2013). At that time, I use Eclipse. prevention of mindless resubmission of the student project was an important issue. Furthermore only few . Already knew that the reviewing code does tools were available for free. On the other hand, not contain problems from the explanation ALECSS utilizes various open source software tools provided by the teacher. so that we are preparing to distribute ALECSS as At week 15 after the second iteration of the open-source software for free. The concept of experiment, 54 students (87%) replied that they used ALECSS is very similar to utilization of DevOps ALECSS to find issues in another project at the peer tools presented in Eddy’s paper (Eddy 2017). review. Teachers can easily check progress and update history They replied also if the system is useful in five of the project by utilizing version management tool levels at week 10 and week 15. Figure 10 shows the Git and continuous integration tool Jenkins. There are results. several education support systems for DevOps (Rong, 2017; Krusche, 2014). Unlike their work, we aim to support more general and flexible software engineering education using DevOps tools. ALECSS is designed to integrate various checking tools for the programming exercise so that we added original checking functions as we explained in Sections 2.1, 2.4, 2.5, 2.7 and 2.8. Pape proposed a software tool STAGE for automatic grading of testing exercises (Pape, 2016). The tool utilizes CodeCover to measure several code Figure 10: Evaluation of ALECSS at week 10 and week 15. coverage metrics in the context of white-box testing. It utilizes Moodle as a frontend to the students. For week 10, we have Positive answers (5 or 4) However extension of the checking function is not are from 91% of the students and negative answers (2 addressed. or 1) from 9% of the students for usefulness of Nandigam et al. reported that student’s ALECSS. Then for week 15, we have Positive understanding of various software engineering answers (5 or 4) are from 85% of the students and principles by utilizing various software tools negative answers (2 or 1) from 4% of the students for (Nandigam, 2008). They also proposed to utilize usefulness of ALECSS. software tools for iterative and incremental We also collected the following comments about development, documenting software requirements, ALECSS. Although we obtained some negative version control and source code management, coding comments, we are going to improve the system for standards compliance, design visualization, software further utilization. testing, software metrics, etc. for undergraduate software engineering courses (Nandigam, 2014). . Excellent checking system Although the motivation of the research is similar to . Very useful since ALECSS provides detailed ours, they did not develop a system for automatic information about who did which task. checking of the project code such as ALECSS. Yu et al. also utilize free/open-source data and tools for upper-level software engineering class for

374

Utilizing Software Engineering Education Support System ALECSS at an Actual Software Development Experiment: A Case Study

two semesters, where instructor’s experiences are Bass, L., Weber, I., Zhu, L., 2015. DevOps: A Software assembled and analysed (Yu, 2014). They also Architect’s Perspective, SEI Series in Software reported that utilization of free/open-source data and Engineering, Addison-Wesley. tools facilitate understanding of the students and that Caiza, J. C., Del Alamo, J. M., 2013. Programming assignments automatic grading: Review of tools and their course can be kept up-to-date according to the implementations, In Proc. 7-th Technology, Education advancement of the tools and development methods and Development (INTEND 2013), pp. 5691-5700. in industry. Along with the same experience as theirs, Douce, C., Livingstone, D., Orwell, J., 2005. Automatic we developed ALECSS which can be utilized at other test-based assessment of programming: A review, In software engineering courses. ACM Journal of Educational Resources in Computing, Vol. 5, No. 3, Article 4, 113 pages. Eddy, B. P. et al., 2017. CDEP: Continuous Delivery Educational Pipeline, In Proc. South East Conference, 8 CONCLUDING REMARKS pp. 55-62, ACM. Ihantola, P. et al., 2010. Review of recent systems for Before developing ALECSS, the codes submitted by automatic assessment of programming assignments, In the students have been manually checked by the Proc. 10-th Koli Calling International Conference on teachers. These checks are large burden for teachers Computing Education Research, pp. 86-93. and take time to return feedback to students. We Koga, A., Feb. 2018. Test case checking functions for often observe the similar situation at many exercise software engineer education support system ALECSS, As Graduation Thesis, Saga University. (in Japanese) for programming education. ALECCS can Krusche, S., Alperowitz, L., 2014. Introduction of automatically check the submitted codes so that continuous delivery in multi-customer project courses, students (as individual or team member) can check Proc. ICSE2014, pp. 335-343, ACM. the results and correct their projects quickly. On the Nandigam, J., Gudivada, V.N., Hamou-Lhadj, A., 2008. other hand, the teachers can monitor students’ Learning software engineering principles using open progress through Jenkins so that the burden of daily source software, In Proc. Frontiers in Education checking can be significantly decreased. Then the Conference, FIE 4720643, pp. S3H18-S3H23. opportunity to guide students will be increased for the Nandigam, J., Gudivada, V.N., 2014. Learning software teachers. industry practices with open source and free software tools, In Open Source Technology: Concepts, Methodo- For the future work, we are planning to integrate logies, Tools, and Applications 2-4, pp. 997-1012. tool such as Redmine to Ohtsuki, M., Ohta, K., Kakeshita, T., 2016. Software maintain master schedule and integrate to ALECSS. engineer education support system ALECSS utilizing Then the communication within each student team DevOps tools, In Proc. 18-th International Conference can be improved and the time management of the on Information Integration and Web-based project will become easier. We also have a plan to Applications & Services (iiWAS2016), pp. 209-213. distribute ALECSS by utilizing incremental Pape, S., Flake, J., Beckmann, A., Jürjens, J. 2016. STAGE: deployment tool such as Chef or Docker. Then other A software tool for automatic grading of testing educational institution can utilize ALECSS for their exercises: Case study paper, In Proc. International Conference on Software Engineering, pp. 491-500. software engineering education. Rong, G. et al., 2017. DevOpsEnvy: An Education Support System for DevOps, In Proc. CSEET2017, pp. 37-46, IEEE. ACKNOWLEDGEMENTS Yu, L., Surma, D. R., Hakimzadeh, H., 2014. Incorporating free/open-source data and tools in software engineering education, In Overcoming Challenges in Software This research is supported by JSPS Kakenhi Grant Engineering Education: Delivering Non-Technical No. 17K01036. We also appreciate the students Knowledge and Skills, pp. 431-441. working on the development of ALECSS.

REFERENCES

Ala-Mutka, K. M., 2005. A survey of automated assessment approaches for programming assignments, Computer Science Education Journal, Vol. 15, Issue 2, pp. 83-102, DOI: 10.1080/08993400500150747 Allspaw, J., Hammond, P., 2009. 10+ Deploys Per Day: Dev and Ops Cooperation at Flickr.

375