Recent Advances in Computer Science Nonlinear and Quantitative Software Engineering Method Based on Complexity Science Jay Xiong Lin Li NSEsoftware, LLC., USA www.nsesoftware.com [email protected] [email protected] Abstract—This paper introduces a nonlinear and quantitative software engineering method (NSE-Method), which is the core part of the Nonlinear Software Engineering paradigm (NSE) based on complexity science. NSE-Method offers a nonlinear, holistic, global, and quantitative approach for software development. The objective of NSE-Method is to make it possible to help software development organizations double their productivity, halve their cost, and improve the quality of their software products by several orders of magnitude simultaneously. NSE-Method has been successfully implemented and supported by the NSE platform, Panorama++, integrated from many automated software engineering tools. Keywords: Software engineering method; complexity science; nonlinear software engineering; software engineering paradigm (a) Incomplete – the process is incomplete: “The Unified I. INTRODUCTION Process suffers from several weaknesses. First, it is only a Today software has become the driving force for the development process… it misses the concept of maintenance development of all kinds of sciences, engineering, and and support…. The relative software investment that most businesses. As pointed out by David Rice, “Like cement, organizations make is allocating roughly 20% of the software software is everywhere in modern civilization. Software is in budget for new development, and 80% to maintenance and your mobile phone, in your home computer, in cars, airplanes, support efforts.” （Scott Ambler, [2]). In fact not only RUP, hospitals, businesses, public utilities, financial systems, and but almost all existing software development methods do not national defense systems. " [1] really support software maintenance without various bi- But unfortunately, software itself is not well engineered, directional traceabilities. many critical issues (low quality and productivity, and high (b) Inconsistent - the design documents and the source cost and risk, etc.) have existed with software development for code of a software product developed using existing methods more than 50 years. are inconsistent with each other after code modification again The root cause is that software is a nonlinear system where and again. a small change may bring big impacts to the entire system – the (c) Unreliable - “Major software projects have been “Butterfly-Effect” — but the old-established software troubling business activities for more than 50 years. Of any engineering paradigm with all existing software development known business activity, software projects have the highest methods is an outcome of linear thinking, reductionism, and the probability of being cancelled or delayed. Once delivered, these superposition principle that the whole of a nonlinear system is projects display excessive error quantities and low levels of the sum of its parts, so that with it almost all software reliability.” (Capers Jones, [3]). This is why software disasters engineering activities are performed linearly, partially, locally, happen often. and qualitatively. (d) Un-maintainable - “Over three decades ago, software For efficiently resolving those critical issues, a new maintenance was characterized as an ‘iceberg’. We hope that software engineering method called NSE-Method has been what is immediately visible is all there is to it, but we know innovated which complies with the essential principles of that an enormous mass of potential problems and cost lies complexity science. Theoretical comparison with initial under the surface. In the early 1970s, the maintenance iceberg applications shows that compared with the old software was big enough to sink an aircraft carrier. Today, it could engineering methods it is possible for NSE-Method and the easily sink the entire navy!” (Roger S. Pressman, [4]). “The corresponding tools to help software development fundamental problem with program maintenance is that fixing organizations double their productivity, halve their cost, and a defect has a substantial (20–50%) chance of introducing improve the quality of their software products by several orders another”(Brooks, [5]). This is why so many cloud computing of magnitude simultaneously. system failures were reported in 2011, including * Sony’s Playstation Network (4/21/2011) II. CRITICAL ISSUES ADDRESSED * Amazon Web Services (4/21/2011) The critical issues addressed by NSE-Method have existed * Twitter Service (2/25/2011) with software development for more than 50 years, including * Netflix Streaming Service (3/22/2011) ISBN: 978-960-474-311-7 276 Recent Advances in Computer Science * Intuit Service and Quickbooks (3/28/2011), and more [6]. (e) Invisible – the existing visualization methods, techniques, and tools do not offer the capability to make the entire software development process and the work products visible; in most cases, the generated charts and diagrams are used for modeling only, and are not holistic and not traceable. (f) More – there are more critical issues existing with software development, including low quality and productivity, high cost and risk, etc. Fig. 2 Linear software engineering methods III. WHY HAVE THOSE CRITICAL ISSUES EXISTED FOR MORE (Source: “An Analysis of Model Driven Architecture (MDA) and THAN 50 YEARS WITHOUT BEING RESOLVED? Executable UML (xUML)”, http://www.powershow.com/view/30871- As we know well, many software engineering methods and MjU5N/An_Analysis_of_Model_Driven_Architecture_MDA_and_Exec utable_UML_xUML_flash_ppt_presentation ) technologies have been innovated and applied by software scientists, experts, and engineers such as the Object-Oriented software development approach, the platform-independent Java programming language, the model-driven software Fig. 3 Software development methods based on the development methods, and the Agile software development Constructive Holism Principle approaches, but why those critical issues have existed for more than 50 years without being resolved? (b) The existing software modeling approaches - they are Before answering this question, let us consider what makes outcomes of reductionism and the superposition those critical issues exist: principle, which use different sources for human (a) The process models – they are linear ones, no matter if understanding and computer understanding of a it is a waterfall-like model, an incremental development software system separately, with a big gap between “ ” model which is a series of Waterfalls , or an iterative them as shown in Fig. 4. The resulting models are not development model in which each time of the iteration is a traceable for static defect removal, not executable for waterfall, or a new process model recommended by debugging, and not testable for dynamic defect Alistair Cockburn to combine both Incremental and removal, not consistent with the source code, and not Iterative development together [7] shown in Fig. 1, with qualified as the road map for software development - which there is no upstream movement at all, the work nobody knows whether they are complete or not, flow is always going forward from the upper phases to the correct or not, and consistent with each other or not. lower phases. The result is that defects introduced in the We hope the next generation of UML and the upper phases easily propagate to the lower phases to make MDA/MDD/MDE can solve those problems. the defect removal cost increase as much as tenfold. Fig. 1 “Putting Iterative and Incremental Development Together” (b) The software development methods – as shown in Fig. 2, Fig. 4 The existing software modeling approaches they are one-way software development methods (Top- Down or Bottom-Up) based on linear thinking, (c) The software testing paradigm – most software reductionism, and Constructive Holism principle that the defects are introduced into a software product in the software components are developed first, then the system requirement development phase and the product design of a software product is built through the integration of phase as shown in Fig. 5, but the existing software the components developed (see Fig. 3) - “Assemble the testing paradigm can only be dynamically used after product from the product components, ensure the production as shown in Fig. 6 and Fig. 7, so that NIST product, as integrated, functions properly and deliver the (National Institute of Standards and Technology) product.” [8] , so that system testing is performed after concluded that “Briefly, experience in testing software all components have been developed. With these and systems has shown that testing to high degrees of methods, the quality of a software product is carried out security and reliability is from a practical perspective through inspection and software testing after production – not possible. Thus, one needs to build security, it is too late. reliability, and other aspects into the system design itself and perform a security fault analysis on the ISBN: 978-960-474-311-7 277 Recent Advances in Computer Science implementation of the design.” (("Requiring Software or more projects being developed, it is hard to balance Independence in VVSG 2007: STS the development. Recommendations for the TGDC," November 2006, (j) The entire software engineering paradigm - it is an http://vote.nist.gov/DraftWhitePaperOnSIinVVSG2007 outcome of linear thinking, reductionism, and