SOFTWARESTANDARDS TECHNOLOGIES

Formality, Agility, Security, and Evolution in Software Development

Jonathan P. Bowen, Birmingham City University Mike Hinchey, Lero—the Irish Research Centre, University of Limerick Helge Janicke and Martin Ward, De Montfort University , Applied Science University

Combining formal and agile techniques in software development has the potential to minimize change-related problems.

omplex systems have problems if used carefully, follow- and security are central aspects of always been problematic ing standard patterns of use. Formal the system to be designed. While with respect to software methods have been advocated for im- a formal approach can be used to development. Simplic- proving the correctness of software derive a proof, potentially at great C 1 ity is desirable, but the reality of systems, and agile software develop- cost, a also pro- dealing with a customer means that ment has been promoted by the Agile vides a framework for undertaking requirements are likely to change, Manifesto (http://agilemanifesto.org) rigorous testing, potentially with resulting in a corresponding loss of and others for enabling adaptive de- cost savings since the specification elegance in the solution. velopment in the face of changing can be used to direct the tests that A good software engineer will requirements, typically introducing are needed.3 Without a formal speci- design with the knowledge that additional complexity in the process. fication, software testing is a much the system is likely to evolve over more haphazard affair. time, even if the exact nature of the FORMALITY Misunderstanding the term changes is unknown. Such exper- Although using to “formal methods” can also be an tise only comes with experience develop a software system can be issue. The relevance of the formal ap- and an innate aptitude, especially slow and cumbersome, these meth- proach must be understood by every in the understanding and use of ods deliver lower error rates because team member, even if formality won’t abstraction. Software engineering ap- they use a mathematical approach actually be used by everyone. In fact, proaches such as object orientation from the requirements specifica- teaching software engineers to read and modularization—for example, tion onward.2 Formal methods are formal specifications is much easier the schema construct— an important software engineer- than teaching engineers how to write can help minimize change-related ing technique in cases when safety them. Reading and understanding

86 COMPUTER Published by the IEEE Computer Society 0018-9162/14/$31.00 © 2014 IEEE the specification is necessary for and XP continue their advance into process is similar to the widely most members of a software de- mainstream software development, known agile technique planning velopment team—for example, with sometimes impressive results, poker, which is used to establish programmers and testers—whereas there are still reservations in areas effort estimates for user stories. specification writing requires the in- where security is a paramount The recent adaptation of Micro- volvement of a much smaller number concern to stakeholders. Until rela- soft’s Secure Development Lifecycle of more highly trained people. tively recently there was little focus (SDL) to integrate with agile prod- on how to integrate security best ucts shows that security is a major AGILITY Agile software development pro- vides an iterative approach where It’s necessary to integrate security activities in the the requirements and the associated agile development process and its practices. solution evolve through the collabo- ration of team members, and rapid response to change is encouraged. This contrasts with the traditional practices into agile development, concern for companies employing view of formal methods, but modern although these shortcomings have agile methodologies. The key prob- tools can enable a much more agile existed for at least a decade. lem addressed by Bryan Sullivan approach even within formal devel- To address this problem, it’s is how activities from the SDL can opment. For example, the RODIN tool important to make security re- be integrated effectively into the (http://rodin.cs.ncl.ac.uk) minimizes quirements explicit and to include short-release cycles that charac- the amount of re-proof that is needed security concerns in the product terize agile development projects.6 if the system is changed. Tools like backlog with adequate priority, so This approach divides SDL activi- the Alloy Analyzer (http://alloy.mit. that the concerns continue to be ad- ties into three categories: every edu) are relatively easy for a capable dressed by the development team sprint, such as running automated software engineer to learn and can during the iterations. Agile methods security-analysis tools and updating be quickly used in an agile manner.4 strive to satisfy the customer, which the threat model; bucket require- The concept of agile formal meth- frequently prioritizes the develop- ments consisting of verification ods first appeared in the literature ment of functionality that produces tasks, design review activities, and in the mid-2000s, with events such the primary business value. Secu- response planning; and one-time as the Formal Methods and Agile rity, on the other hand—because it’s requirements such as the develop- Methods (FM+AM) Workshop explic- concerned with managing risk and ment of a baseline threat model. itly addressing the issue.5 In 2009, a preventing the developed function- So, while efforts to integrate position paper on formal methods ality from being misused—is often security practices in agile method- and agile software development was unfortunately not as highly valued ologies are well underway, there published in Computer to highlight and therefore isn’t addressed in is a risk that many actual devel- the debate.4 early sprints. Consequently, it’s nec- opment efforts will ignore these While we don’t really hold the essary to integrate security activities efforts or that development teams view that agility may be used by in the agile development process lack the knowledge and training to “unscrupulous” developers, in and its practices. actually create secure products in general we do believe that agile de- One way to address security the short term. However, agile prin- velopers could benefit from some early in an agile development con- ciples such as communication and training on formal methods, at least text is by using evil stories. This is practices such as pair programming in reading formal specifications, an agile adoption of misuse cases to may help disseminate knowledge even if developers don’t apply the describe the functionality that an at- throughout the development team. approach rigorously. tacker would be able to exploit. The Certification remains a potential development then takes on two di- issue, as it requires detailed and SECURITY mensions: to implement user stories sometimes formal documentation. Can agile methods produce secure and to avoid implementing evil sto- The approach discussed in Sulli- software, perhaps in cases in which ries. Another practice that integrates van’s article6 can help by mandating the security properties have been security principles in agile develop- activities that need to be included formalized? Opinions regarding ment is a technique called protection in every sprint and by requiring this question remain split. While poker, in which security risks are a (scope-reduced) final security agile methodologies such as Scrum quantified by the agile team. This review at the end of each sprint.

OCTOBER 2014 87 SOFTWARE TECHNOLOGIES

so if a large body of assembler code The transformation process will typ- Formal Informal can be replaced by a smaller amount ically include the following stages: program implementation of high-level language code—­ specication ideas preserving its correctness without • formal specification; seriously affecting performance— • elaboration of the specification; then the potential savings in the • dividing and conquering to form of software and hardware handle the general case; Program 1 maintenance costs are very large. • recursion introduction; A sound agile approach to soft- • recursion removal, if an itera- ware evolution that is based on tive solution is desired; and Program n transformation theory and associ- • optimization, if required. ated with the industrial-strength FermaT program transformation Subspecifications can be ex- Figure 1. Transformational system has been developed. The ap- tracted and transformed separately programming method of algorithm proach to agility and evolution via at any stage in the process. The derivation. transformation involves four stages: main difference between this ap- proach and the invariant-based EVOLUTION • Translate the assembler to Wide programming approach (and similar Real software systems continually Spectrum Language (WSL). stepwise refinement methods) is that evolve. Their evolution is often in • Translate and restructure data loops can be introduced and manip- response to the evolution of their declarations. ulated while maintaining program environments: new functionalities • Apply generic semantics- correctness with no need to derive are added and/or existing ones are preserving WSL to WSL loop invariants. Another difference removed due to the need to address transformations. is that at every stage in the process, changes in business requirements • Apply task-specific operations we’re working with a correct pro- and/or economic forces, and new for migration (translate the gram: there’s no need for a separate platforms are developed and/or new high-level WSL to the target “verification” step. These factors technologies are introduced. This language) and analysis (apply help ensure that the method is capa- necessitates a paradigm shift in slicing or abstraction operations ble of scaling up to the development the way software systems are and to the WSL to raise the abstrac- of large, complex software systems. will be developed, in which agility tion level even further). The algorithm derivation method (without sacrificing trustworthi- has been applied to the derivation of ness) is required. Martin P. Ward and Hussein Ze- the “polynomial addition” problem de- Software migration, especially as- dan’s “Deriving a Slicing Algorithm scribed by Donald Knuth, which is a sembler migration, is an important via FermaT Transformations” fully complex linked list algorithm that uses type of evolution. More than 70 per- explains WSL, the transformation four-way linked lists.9,10 The derived cent of all business-critical software theory, and how program slicing source code turned out to be more currently runs on mainframes, and can be defined as a transformation than twice as fast as Knuth’s code. more than 10 percent of all code cur- within the theory.7 This approach This method was also used to rently in operation is implemented in enables a formal agile methodology derive a polynomial multiplication assembler, which totals around 140– to be applied to real software, aiding algorithm using the same data struc- 220 billion lines. the software evolution process. tures. It started with a trivial change However, the pool of experienced As Figure 1 shows, the transfor- to the formal specification (replace assembler programmers is decreas- mational programming method of “+” with “*”), and the transforma- ing rapidly. As a result, there is algorithm derivation8 starts with a tional derivation was guided by the increasing pressure to move away formal specification of the result to same informal ideas as the addition from assembler, including moving be achieved, together with some in- algorithm. The result was an effi- fewer critical systems away from formal ideas as to which techniques cient implementation of polynomial the mainframe platform, so the will be used in the implementation. multiplication, which worked the legacy assembler problem is likely to The formal specification is then first time. Another paper uses this become increasingly severe. transformed into an implementation method to derive an implementation Analyzing assembler code is by means of correctness-­preserving of program slicing from the formal significantly more difficult than refinement and transformation definition of slicing, defined as a analyzing high-level language code, steps, guided by the informal ideas. program transformation.8

88 COMPUTER e believe that a com- 2. J.P. Bowen and M.G. Hinchey, Addison-Wesley, 1997. bination of formal “Formal Methods,” Computing 10. M. Ward and H. Zedan, “Provably W and agile approaches Handbook, 2nd ed., vol. 1, CRC Correct Derivation of Algorithms is a worthwhile goal in the appli- Press, 2014, pp. 1–25. Using FermaT,” Formal Aspects of cation of software engineering 3. R.M. Hierons et al., “Using Computing, vol. 26, no. 5, 2014, methodologies to real computer- Formal Specifications to Support pp. 993–1031. based systems. Of course, a note Testing,” ACM Computing of caution is in order.5 The use of Surveys, vol. 41, no. 2, 2009; heavyweight formal methods with doi:10.1145/1459352.1459354. Jonathan P. Bowen is a professor full program proving is likely to be 4. S. Black et al., “Formal Versus of at Birmingham difficult and not worthwhile in a Agile: Survival of the Fittest,” City University, UK. Contact him at typical agile setting. On the other Computer, vol. 42, no. 9, 2009, [email protected]. hand, this is a rare approach, largely pp. 37–45. due to scaling problems and the dif- 5. P.G. Larsen, J. Fitzgerald, and Mike Hinchey is director at ficulty of using the available tools. A S. Wolff, “Are Formal Methods Lero—the Irish Software Research lighter touch with the use of formal Ready for Agility? A Reality Centre and a professor of software specification is much more typical Check,” Proc. 2nd Int’l Workshop engineering at the University of Lim- when applying formal methods to Formal Methods and Agile erick, Ireland. Contact him at mike. improve early understanding, guide Methods (FM+AM 10), vol. P-179, [email protected]. the programmer, and perhaps aid 2010, pp. 13–25. proper testing. In this context, com- 6. B. Sullivan, “Streamline Security Helge Janicke is a reader in com- bined use with agile development Practices for Agile Development,” puter science and head of the is more likely to succeed, if applied MSDN Mag., vol. 23, no. 12, 2008, Software Technology Research Labo- judiciously with experienced engi- pp. 52–55. ratory at De Montfort University, neering judgment. 7. M.P. Ward and H. Zedan, UK. Contact him at heljanic@dmu. There are additional issues to “Deriving a Slicing Algorithm via ac.uk. consider in areas such as safety or FermaT Transformations,” IEEE security-related systems, but the Trans. Software Eng., vol. 37, no. 1, Martin Ward is a reader in the Soft- combination of formal and agile 2011, pp. 24–47. ware Engineering Department at De techniques has potential even in 8. M.P. Ward and H. Zedan, Montfort University, UK and CTO of these cases. An agile approach is “Provably Correct Derivation Software Migrations Ltd. Contact especially applicable in response of Algorithms using FermaT,” him at [email protected]. to evolving systems; for example, Formal Aspects of Computing, where the software may need to vol. 26, no. 5, 2013, pp. 993–1031. Hussein Zedan is Dean of Graduate be completely upgraded to a new 9. D. Knuth, The Art of Computer Studies & Research at the Applied language. By adding a formal foun- Programming, Vol. 1: Science University, Bahrain. Contact dation, agile evolution can be Fundamental Algorithms, 3rd ed., him at [email protected]. applied in a sound manner to large software systems with suitable for- malized transformation rules.

Acknowledgments thanks Museophile Lim- ited for financial support. Mike Hinchey acknowledges support from Science Foun- dation Ireland under grant 10/CE/I1855.

References 1. J.P. Bowen and M.G. Hinchey, “Ten Commandments of Formal Methods…Ten Years Later,” Computer, vol. 39, no. 1, 2006, pp. 40–48.

OCTOBER 2014 89