Software Testing: a Changing Career
Total Page:16
File Type:pdf, Size:1020Kb
Software Testing: A Changing Career Sean Cunningham1, Jemil Gambo1, Aidan Lawless1, Declan Moore1, Murat Yilmaz2, Paul M. Clarke 1,3 [0000-0002-4487-627X], Rory V. O’Connor1,3 [0000-0001-9253-0313] 1 School of Computing, Dublin City University, Ireland {sean.cunningham29,jemil.gambo2,aidan.lawless8, declan.moore39}@mail.dcu.ie 2 Department of Computer Engineering, Cankaya University, Turkey [email protected] 3 Lero, the Irish Software Engineering Research Center {rory.oconnor,paul.m.clarke}@dcu.ie The software tester is an imperative component to quality software development. Their role has transformed over the last half a century and volumes of work have documented various approaches, methods, and skillsets to be used in that time. Software projects have gone from using monolithic architectures and heavy- weight methodologies, to service-oriented and lightweight. Testing has trans- formed from a sequential step performed by dedicated testers to a continuous activity carried out by various development professionals. Technological ad- vancements have pushed automation into routine test tasks permitting a change of focus for the tester. Management styles and methodologies have pushed de- velopment to be agile and lean, towards continuous integration and frequent re- lease. Regardless of these many important changes, the software tester’s role re- mains the verification and validation of software code. Keywords: software quality improvement, test drive development, continuous development, software development lifecycle 1 Introduction It is well documented that software development is a complex socio-technical activity [42], the process for which needs to be vigilantly maintained and evolved [43]. We furthermore find that roles within software development can have varying names [44], and that the very terminology adopted is perhaps the source of some fusion [45] [46]. We have also seen that the process and the details of work carried out when producing software are also variable given any particular development setting [47] [48]. Irrespec- provided by DCU Online Research Access Service View metadata, citation and similar papers at core.ac.uk CORE tive of these many important concerns, as a fundamental activity,brought successful to you by commer- cial software development must incorporate a testing function. The precise role of the tester may vary in different organisations and change over time, and in this paper, we examine the nature of this change. The role of a software tester has not fundamentally changed in the sense that soft- ware testers verify and validate that software complies with requirements. However, 2 what has changed are the tools and practices used to achieve this, and this affects both traditional and contemporary development methodologies. Recent changes in the prac- tices and tools used by the software tester are a result of the increased utility of faster, better, and cheaper computing; which has permitted automation as a core component of modern test suites. Despite the evolution of the methods and tools of a software tester, the role at its foundation remains unchanged. The tester has adjusted the testing to suit the modern environment of software development in order to achieve the same outcome. In this paper, we examine the impact on the role of the software tester arising from changes in software development lifecycle models, carefully examining the specific impact of agile methodologies and DevOps based software development. We also ex- amine the changing nature of software projects themselves over time and identify how automation has had a lasting and significant impact on the role of a software tester. 2 Related Literature 2.1 Methodology The method used for finding appropriate papers involved a set of search strings on academic search engines which include ScienceDirect, ACM Digital Library, IEEE Xplore, and Wiley Online Library. Search strings used included and combined: “soft- ware testing”, “role software tester”, “agile software testing”, “heavyweight software methodologies”, “software methodology changes”, “future software testing”, “function as a service testing”, “software test automation”, and “manual automated software test- ing”. Software quality focused publications required more specific strings, whereas general software journals produced relevant papers on general software testing strings. Many search strings were generated through iterations of a snowball approach to the literature found. Recurring themes and elements of relevant literature would produce new search strings which could be used to generate further relevant sources. This snow- ball approach is also applicable to additional paper discovery. Relevant literature would cite sources for its content which in turn provided additional sources for our process. While this paper seeks to primarily source any statements from academic, peer-re- viewed sources, there is a clear value to utilizing multivocal sources. Particularly in the discipline of computer science, where there is a diverse and expansive online commu- nity discussing the role of the software tester. As such we have included analyses from certain non-peer reviewed sources. Academic textbooks on the topic of software testing and software development have too proved useful in literature discovery. These texts reference academically reliable sources which can be also gathered to strengthen statements of this work. 2.2 Inclusion/Exclusion Criteria Any of the sources this paper uses must be available in full-text; either electronically or in print. All papers should also be in English. While we are approaching this research 3 paper from a multivocal point-of-view, the merit of a peer-reviewed paper will super- sede a non-peer review document. Discovered papers, beyond the title relevance should present further potential as a relevant source to the topic. Each paper with relevant title was briefly assessed for ad- ditional significance and compiled into a list of relevant papers along with a brief sum- mary for later. Papers found through the methodology outlined above are given additional merit and considered more likely for inclusion if they are from the following journals; or where the authors are otherwise reputable contributors to these publications: Transactions on Software Engineering; Information & Software Technology; Journal of Systems & Software; Software Testing, Verification & Reliability; Transactions on Software En- gineering & Methodology; Software Quality Journal; IEEE Software; Journal of Soft- ware: Evolution & Process. Exclusion criteria on papers discovered included documented specific, non-industry standard, approaches or applications of specific tools. These papers, while loosely rel- evant to the topic don’t typically provide much insight into general topics but more theoretical assessments of their subject’s validity. 2.3 Papers Discovered Table 1. Papers discovered using methodology outlined above # of papers TSE IST JSS STVR ToSEM SQJ IEEES JSEP Identified 29 29 25 11 6 22 19 6 Included in 3 3 1 1 0 4 2 0 analysis 3 Analysis 3.1 Definition Definitively, software testing can be traced back to 1957 when the differentiation be- tween code debugging and program testing was established by C L Baker [1].The role developed from then in various directions. However, few were formalised until Myers book in 1979 [2] which discussed how despite 50% of software project time being spent on testing, that less is known about it than any other component of SDLC. The traditional role of the software tester can be defined as one responsible for cre- ating test plans and testing the developed programs [3]. From a philosophical approach, the software tester is responsible for verification and validation of developed software. 3.2 Software Development Lifecycle Models Traditional Methodologies 4 While many would assume that the practices of Incremental and Iterative Development (IID) are recent concepts evolving from the surge in agile methodologies, they can be seen throughout the history of software development [4]. Practices like those employed in Extreme Programming are evident as far back as NASA’s Project Mercury (1958), which used test-first development, which included time-boxed, half-day iterations. Despite early evidence of IID, many software developers in the 1960s used the ad- hoc ‘code-and-fix’ method. This method has been described as “one year of slamming code, one year of debugging” [5]. This flawed method was the worst-of-both worlds for the software tester. It provided little in defining test parameters, didn’t improve pro- cess quality, nor did it assist in optimising the economics of software testing. The 1970s heralded the ‘Waterfall’ method, which was introduced by Winston Royce [6]. This methodology advocates a logical sequence of phases to be followed. The completion of the previous phase must be complete before continuing to the next. However, common misconception of Royce’s method views the entire process as a sin- gle-pass through the phases, whereas Royce suggests the process be carried out across two iterations [4]. For the software tester, their role is fixed to the later phases of the life-cycle. As such, testing tended to be viewed as a dynamic testing activity that com- menced once workable software was available from the programming team. The V-Model approach to software development views development and testing as corresponding activities of equal importance. This model attempts to validate and ver- ify the development based