Bachelor Degree Project Interactive Execution of Non-interactive Test Framework Features - In Collaboration with Ericsson AB Author: Niklas Eriksson Supervisor: F rancis Palma Semester. VT 2020 Subject: Computer Science Abstract Some software libraries and frameworks get built to assist engineers in other domains than software engineering. However, these tools can be too complicated for these engineers to utilize due to their inexperience in software engineering. The engineers must then request assistance from more experienced software engineers. And when experienced software engineers must assist, they can spend less time on their original work. This problem got identified at Ericsson AB for one of their test automation frameworks for radio development, where test cases had to be written in pure Java code by test engineers without the test engineers always having the necessary skills to do so. In collaboration with Ericsson, we address this problem by applying design science to develop a novel technique that enables interactive execution of parts of this framework, called MJE Actions. We then present a prototype that embodies this novel technique. It enables test engineers at Ericsson to interactively execute MJE Actions via command-line without having to write a single line of Java code. Keywords: command-line interface, fluent interface, software design, software architecture, test automation framework, method chaining. Preface The basis for this research initially stemmed from my passion for software design and development. Due to my inexperience in developing command-line interfaces, I saw collaborating with Ericsson AB to design and develop one as an excellent opportunity to learn and gain more experience. In truth, I could not have achieved my current level of success without all the support from supervisors and family. First of all, I thank my parents, who have provided me with love and understanding. Secondly, I thank my university supervisor and my supervisor at Ericsson that has guided me through the process of succeeding with my research. Thank you all for your unswerving support. Contents 1 Introduction 5 1.1 Background 5 1.2 Problem formulation 6 1.3 Motivation 7 1.4 Related work 7 1.5 Objectives 8 1.6 Target group 9 1.7 Scope/Limitation 9 1.8 Outline 9 2 Method 10 2.1 Design Science 10 2.2 Instantiation of Design Science 11 2.3 Reliability and Validity 11 2.4 Ethical Considerations 11 3 Objectives for a solution 12 4 Implementation 13 4.1 Abstract Design 13 4.2 Concrete Design 15 4.3 Development 17 5 Results 23 6 Analysis 29 6.1 Fulfillment of requirements 29 6.2 Evaluation of the prototype 30 6.3 Concluding discussion 30 7 Conclusion & Future work 32 References 34 Appendix A - The requirements of the prototype 36 Appendix B - The test cases for the prototype 37 1 Introduction This report presents the execution of a bachelor's degree project at Linnaeus University of 15 credits. The project is a collaboration with the telecom company Ericsson AB targeting a problem about their test engineers having a hard time using a Java Test Framework practically and efficiently to fulfill their needs and goals. 1.1 Background Most software libraries and frameworks are designed and implemented for trained software engineers and software developers. Some software packages are built for domain experts in other domains than computer science or software engineering. These domain experts might not have enough programming skills to efficiently use various software tools [1], resulting in heavily requested assistance by more experienced software developers. In Development Services and Infrastructure (DSI), Ericsson develops Continuous Integration framework, test tools, test automation framework, test orchestration, and test environment. The mission for DSI is to enable cost-efficient radio development worldwide with their development environment as a competitive advantage, both for present and future business needs. In the test department of DSI, they develop the test automation framework, called MJE (MSRAN 1 JCAT [Java Common Automatic Test] Extension). It is developed in Java and uses, except for functionality developed in the organization, open-source software like TestNg [2], and Spring beans [3]. MJE provides functionality that provides nodes (e.g., base stations), user equipment (e.g., mobiles, simulators), logging features, and much of other common functionality to the test programs written in Java, i.e., test cases. In other words, an MJE test program is a Java program implemented on top of the MJE framework. For using MJE as described above, test engineers and developers need to write Java programs that implement the test cases. Finally, they can execute these programs in the test environment. In case of failure, the test engineers can study the logs to determine the root cause. MJE has three abstraction levels where the top-level, i.e., Action, could be 1 MSRAN is an organization name in Ericsson AB. used to write short test programs by using the built-in functionality provided by the framework. The test engineers would benefit considerably from improved test support that enables interactive execution of MJE actions via a command-line interface. 1.2 Problem formulation Having to request more experienced software developers for assistance when utilizing a given software library demands more time and other resources. This approach is not practical since the domain experts must spend much time on things that are not in their area of expertise, and the software developers assisting the domain experts must spend much time doing tasks that are very much routine for them. This degree project investigates how to enable interactive MJE Action execution via a command-line-interface(CLI). Currently, MJE Actions can only execute via invocations declared in the source code of an MJE-based test program. The command-line based solution makes it possible to execute MJE Actions as if they were commands, interactively, and with faster turnaround time; from writing a test case to the execution of the test case. As a proof of concept, we would like to find out if there is a way to enable interactive execution of MJE Actions via the command-line and develop a prototype as a proof of concept. The core problem addressed is How can we create a novel technique for a CLI that supports the interactive execution of MJE Actions? A new technique will assist test engineers to explore and find new ways to test the System Under Test [10] that was never thought of before. The prototype will have generated commands based on the existing MJE Actions, that the user can run. After the completion of this degree project, Ericsson will further investigate the prototype by letting test engineers try it, to find out if the test engineers would gain any value of the prototype and if further development should take place or not. 1.3 Motivation By nature, an MJE-based test program is non-interactive. The life cycle of writing a test case and executing it takes in the best case some hours, up to a few days. Also, for writing the test case, Java competence is needed, which is not the case for all test engineers. An interactive and command-line based abstraction on top of MJE is supposed to help the above obstacles. The interactivity of the system increases the possibility of doing Exploratory Testing [11] and finding the weak parts of the System Under Test. The test engineers who lack Java experience would also be able to use the system quickly to realize their test ideas. The time for performing spontaneous and quick testing gets shortened. Ericsson also has a similar system, MoShell, which operates on a single node (e.g., base station). The simplicity and popularity of an interactive and command-based test system are already proven. Ericsson is interested in evaluating the same idea (a command-based interactive system) with MJE, which, unlike MoShell, operates on multiple nodes, tools, and protocols. Ericsson, with its mobile networks, among other product offerings, has a significant impact on society. Therefore, we believe that the findings of this study will positively affect society by prototyping easier test writing for Ericsson mobile networks. 1.4 Related work There exists a considerable body of work regarding CLI based development. Wesley Bland et al. developed a command-line interface (CLI) to interact with the software tool called OSCAR (open-source cluster application resources) [4]. Users could only interact with OSCAR via a graphical user interface (GUI). The CLI is an alternative user interface to the GUI and is a close mirror of the GUI, providing the same functionality [4]. In our case, there is no user interface to the MJE framework already, as test programs are written manually in Java. Similar to OSCAR CLI, our CLI does not change any functionality of the underlying system. Accelerating Test Automation Platform (ATAP) was developed to make test automation accessible to non-programmers by Anurag Dwarakanath et al. It works by automatically converting English-like test scripts to machine-executable code using Selenium WebDriver [1]. Comparing the ATAP project with ours, we find the similarity in that both want to enable non-programmers to utilize an underlying test framework. Martin Fowler created a framework for thinking about domain-specific languages (DSLs) [5]. It divides DSLs into internal DSLs, external DSLs, and language workbenches. An external DSL is an entirely separate language having a full written parser. In contrast, an Internal DSL is an idiomatic way of using a general-purpose language. A language workbench is a tool that, other than parsing and code generation, provides a better editing experience for DSL users. In all these three cases, it is important to have an explicit semantic model so that they form a "thin skin" over an underlying library [5].