BUILDING AGILE DOCUMENTATION WITH
THE INTENTIONAL GENERATIVE PATTERN MECHANISM (IGPM)
A Thesis
Submitted to the Faculty of Graduate Studies and Research
In Partial Fulfilment of the Requirements
for the Degree of
Doctor of Philosophy
in Electronic Systems Engineering
University of Regina
by
Nawapunth Manusitthipol
Regina, Saskatchewan
March, 2010
Copyright 2010: N. Manusitthipol Library and Archives Bibliotheque et Canada Archives Canada
Published Heritage Direction du 1*1 Branch Patrimoine de I'edition 395 Wellington Street 395, rue Wellington Ottawa ON K1A 0N4 Ottawa ON K1A 0N4 Canada Canada
NOTICE: AVIS: The author has granted a non L'auteur a accord exclusive license allowing Library and permettant a la B Archives Canada to reproduce, Canada de repro* publish, archive, preserve, conserve, sauvegarder, con communicate to the public by partelecommunic telecommunication or on the Internet, distribuer et vend loan, distrbute and sell theses monde, a des fins worldwide, for commercial or non support microforn commercial purposes, in microform, autres formats. paper, electronic and/or any other formats.
The author retains copyright L'auteur conserve ownership and moral rights in this et des droits mor; thesis. Neither the thesis nor la these ni des e> substantial extracts from it may be ne doivent etre irr printed or otherwise reproduced reproduits sans s without the author's permission. UNIVERSITY OF REGINA
FACULTY OF GRADUATE STUDIES AND RESEARCH
SUPERVISORY AND EXAMINING COMMITTEE
Nawapunth Manusitthipol, candidate for the degree of Doctor of Philosophy in Electronic Systems Engineering, has presented a thesis titled, Building Agile Documentation With the Intentional Generative Pattern Mechanism (IGPM), in an oral examination held on February 19, 2010. The following committee members have found the thesis acceptable in form and content, and that the candidate demonstrated satisfactory knowledge of the subject material.
External Examiner: Dr. Pierre Bourque, £cole de technologie sup6rieure
Supervisor: Dr. Luigi Benedicenti, Software Systems Engineering
Committee Member: Dr. Cory Butz, Department of Computer Science
Committee Member: Dr. Raman Paranjape, Electronic Systems Engineering
Committee Member: Professor Kenneth Runtz, Electronic Systems Engineering
Chair of Defense: Dr. A. Hayford, Department of Sociology and Social Studies ABSTRACT
Changes are inevitable in software development, especially considering today's fast- paced and ever-changing business demands. This research investigates the problem and proposes a method called the intentional generative pattern mechanism (IGPM) to improve the responsiveness to change or software process agility.
Normally, agile processes and/or approaches are used to improve process agility. One of the suggestions made by these processes is to reduce the amount of the documentation created during the development. Producing less documents should enable cheap modifications to software since there is no need for the synchronization between code and documents. However, many researchers argue that the lack of adequate documentation limits the use of agile processes only to small projects. The above arguments and other similar concerns in the literature indicate that agile processes may not be the ideal solution to achieve true agility.
This research revisits the problem by investigating the source of the problem - the sources of changes. The result is a set of characteristics necessary to a truly agile process.
A number of existing techniques are found to be possible realizations of those characteristics, except for one - documenting software easily and thoroughly or Agile
Documentation.
Further review confirms that there is no existing documentation technique suitable for
i Agile Documentation. A method for software documentation, the Intentional Generative
Pattern Mechanism (IGPM), is proposed as a technique for Agile Documentation.
In the IGPM, a pattern is created to capture developer intention. The pattern encapsulates software development actions and a label, which describes the intention for performing those actions. More complex intentions and an entire software are naturally documented by a group of connected patterns or a pattern network.
Patterns are executable. When executed, a pattern performs its encapsulated actions. An execution of a pattern network generally results in the development of a software by generating the code. This code generation allows easier synchronization between the documents and the code.
A programming language, called the pattern language, is created to implement the IGPM.
Experiments using the language show that it is possible to document thoroughly with less effort. This proves that the IGPM is a promising choice for Agile Documentation and a truly agile process can be created.
ii ACKNOWLEDGEMENTS
1 wish to express my sincere appreciation to Dr. Luigi Benedicenti for his guidance and advice throughout this research and in the writing of this thesis.
I would also like to thank my internal committee: Dr. Raman Paranjape, Dr. Cory Butz and Prof. Ken Runtz who spent time reading this thesis and provided valuable comments.
Most importantly, this thesis would not have been possible without a TRLabs
Telecommunication Research Graduate Scholarships.
iii POST-DEFENCE ACKNOWLEDGEMENTS
I would like to express my thanks to Dr. Pierre Bourque from the Ecole de technologie superieure as my external examiner and Dr. Alison Hayford from the Department of
Sociology and Social Studies as the chair of the defence.
iv DEDICATION
Indescribable appreciations to mama, papa and sis for your love, guidance and support. A special thank to Lek for your help, patience and companionship.
v TABLE OF CONTENTS
ABSTRACT I
ACKNOWLEDGEMENTS Ill
POST-DEFENCE ACKNOWLEDGEMENTS IV
DEDICATION V
LIST OF TABLES XI LIST OF FIGURES XII
LIST OF APPENDICES XIII
LIST OF ABBREVIATIONS XIV
CHAPTER 1 INTRODUCTION 1 1.1 Motivation 1 1.2 Objectives 4 1.3 Original Contributions 5 1.4 Document Overview 6
CHAPTER 2 BACKGROUND 8 2.1 Software Development Process (SDP) 8 2.1.1 Process as a Framework of Activities 10 2.1.2 Activities and Practises 11 2.1.3 What Exactly is a Software Development Process? 12 2.2 Purposes of the Software Development Process 13 2.2.1 Developing Software 13 2.2.2 Insuring Quality 14 2.2.3 Increasing the Odds of Success 16 2.2.4 Additional Comments 26 2.3 Software Process Adoptability 26 2.3.1 Project and Organization Size 27 2.3.2 Experience 27 2.3.3 Technical and Legacy System Constraints 28
vi 2.3.4 Business Environment 28 2.3.5 Requirements 28 2.3.6 People 28 2.3.7 Quality and Reliability Constraints 30 2.3.8 Project Resources 31 2.3.9 Process Infrastructures 31 2.3.10 Culture 32 2.3.11 Additional Comments on Software Process Adoptability 32 2.4 Software Process Agility 33 2.4.1 Definition 33 2.4.2 History 33 2.4.3 How is Process Agility Useful to Software Development? 34 2.4.4 Why is Agility Important? 34 2.4.5 Achieving Agility - The Agile Approaches 35 2.4.6 Additional Comments on Process Agility 36 2.5 Conclusions 36
CHAPTER 3 EXISTING SOFTWARE DEVELOPMENT PROCESSES 37 3.1 Introduction 37 3.2 Traditional Software Development 37 3.2.1 Common Characteristics of the Traditional Software Development 38 3.2.2 Traditional Software Development Processes 43 3.3 Agile Software Development 55 3.3.1 Common Characteristics of Agile Development 56 3.3.2 Agile Software Development Processes 58 3.4 Problems of Agile Processes 78 3.4.1 Minimalism 78 3.4.2 Lack of Adoptability 81 3.4.3 Lack of Practicality 81 3.4.4 Failure to Work 82 3.5 Conclusions 84
CHAPTER 4 SEARCHING FOR THE TRULY AGILE PROCESS 85 4.1 The Sources of Changes 85 4.1.1 Business-Condition Changes 88
vii 4.1.2 Changes in Business Needs 88 4.1.3 Requirement Changes 89 4.1.4 Technological Changes 90 4.1.5 Management and Organizational Changes 91 4.1.6 High-Level-Solution and Architectural Changes 92 4.1.7 Implementation Changes 92 4.1.8 Test-Result Changes 93 4.2 Dealing with Changes 94 4.2.1 Prevention 96 4.2.2 Elimination 97 4.2.3 Reduction 98 4.2.4 Treatment 99 4.2.5 Toleration 100 4.2.6 Redirection 101 4.3 A Truly Agile Process 101 4.3.1 Risk-Based Upfront Effort 102 4.3.2 Risk Management 104 4.3.3 Accepting and Managing Changes 105 4.3.4 Effective Communication 108 4.3.5 Customer Relationship 109 4.3.6 Management Style 110 4.3.7 Summary and Additional Comments 110 4.4 The Unsolved Problem - Agile Documentation Ill 4.5 Conclusions 113
CHAPTER 5 THE INTENTIONAL GENERATIVE PATTERN MECHANISM (IGPM) 114 5.1 Agile Documentation 114 5.1.1 Roles and Types of Documentation in Software Development 114 5.1.2 Documentation for Agile Processes 116 5.1.3 Possible Solutions 117 5.2 Candidates for Agile Documentation 119 5.2.1 Domain-Specific Languages (DSLs) 119 5.2.2 Model-Driven Developments (MDDs) 120 5.2.3 Language Manipulators 121 5.2.4 Additional Comments 122
viii 5.3 Realizing Agile Documentation 122 5.3.1 Natural Documentation 123 5.3.2 Code Generation 126 5.3.3 Graphical Representation 129 5.3.4 Summary and Additional Comments 129 5.4 The Intentional Generative Pattern Mechanism (IGPM) 129 5.4.1 The Core Concepts 130 5.4.2 A Pattern 133 5.4.3 Advantages of the IGPM 135 5.4.4 An Example Pattern 137 5.4.5 The IGPM for Agile Documentation 139 5.5 Conclusions 139
CHAPTER 6 IMPLEMENTATION THE PATTERN PROGRAMMING PARADIGM 140 6.1 Pattern Programming Paradigm 140 6.1.1 Artifact Properties 141 6.1.2 Delayed Computation 142 6.1.3 Computation Priority 142 6.1.4 Conflict Detection 143 6.1.5 Intention Tracking 143 6.1.6 Text Manipulation 143 6.1.7 Additional Comments 145 6.2 Implementation of the Pattern Programming Language 145 6.2.1 Curry Virtual Machine (Curry VM) 146 6.2.2 Regular Parser (RegParser) 151 6.3 The Pattern Programming Language 151 6.3.1 Basic Elements 151 6.3.2 Ports 154 6.3.3 Patterns 164 6.3.4 ActionRecord 167 6.3.5 Text 167 6.3.6 Intentional Tracking 169 6.4 Examples 171 6.4.1 Example #1: Patterns as Artifacts 171
ix 6.4.2 Example #2: Patterns as Documents 175 6.4.3 Example #3: Patterns as Low-level Intentions 181 6.5 Conclusion 188
CHAPTER 7 CASE STUDY 189 7.1 Introduction 189 7.2 Problem 190 7.2.1 Requirements and General Planning 191 7.2.2 Infrastructure 191 7.3 The Development of the G2H Delivery System 192 7.4 The Implementation 199 7.4.1 Programmer A 199 7.4.2 Programmer B 208 7.5 The Final Product 211 7.6 Discussion 213 7.6.1 Patterns for Documentation 213 7.6.2 Code Generation 216 7.7 Change 1: Add Customer's Phone Number 219 7.7.1 Problem 219 7.7.2 Response 219 7.7.3 Discussion 220 7.8 Change 2: Revert Address Numbers 221 7.8.1 Problem 221 7.8.2 Response 222 7.8.3 Discussion 226 7.9 Conclusion 227
CHAPTER8 CONCLUSIONS AND FUTURE WORK 228 8.1 Conclusion 228 8.2 Future Work 231
LIST OF REFERENCES 233
x LIST OF TABLES
Table 2.1: Some Definitions of Software Development Processes in the Literature 9
Table 3.1: Goals and Practises of "Technical Solution" Process Area 46
Table 3.2: Twelve practises of extreme Programming (XP) 60
Table 3.3: Nine Principles of the Dynamic Systems Development Method (DSDM).... 68
Table 4.1: Solutions for Software Process Agility 95
xi LIST OF FIGURES
Figure 3.1: CMMI Stage and Continuous Representations 44
Figure 3.2: Cleanroom Life Cycle - the Pipeline Process Model 48
Figure 3.3: RAD Process Model - the Parallel-Iterative Development Model 51
Figure 3.4: Four Phases of the Rational Unified Process (RUP) 53
Figure 3.5: Five Phases of eXtreme Programming (XP) 62
Figure 3.6: Scrum Process Framework 65
Figure 3.7: The Dynamic Systems Development Method (DSDM) 69
Figure 3.8: Process Framework of the Adaptive Software Development (ASD) 72
Figure 3.9: The Crystal methodologies are named by colour 74
Figure 3.10: Process Framework of the Feature-Driven Development (FDD) 76
Figure 4.1: A Dependency Model of Software Development 86
Figure 4.2: Simplified Change-Control Process 106
Figure 5.1: Patterns 132
Figure 5.2: A pattern network allows full traceability 134
Figure 5.3: The "ReadOnlyField" pattern 138
Figure 6.1: Rewrite UserPath pattern 176
Figure 6.2: Result and Tracking of the Rewrite_UserPath 180
Figure 6.3: Options' properties 184
Figure 7.1: Page's components added by patterns 196
Figure 7.2: G2H's Delivering System 212
Figure 7.3: G2H project's requirements and tasks 215
xii LIST OF APPENDICES
APPENDIX A THE CODE EXAMPLES 244 I Example #1 244 II Example #2 245 III Example #3 246
APPENDIX B INFRASTRUCTURE PATTERNS FOR THE CASE STUDY 250 I Introduction 250 II Brief Description 250 11.1 Database Patterns 250 II.II HTML Patterns 251 II.III PHP Patterns 251 II.IV PHP Database Patterns 252 III Code 253 III.1 Database Patterns 253 III.II HTML Patterns 268 III.III PHP Patterns 279 III.IV PHPDB Patterns 282
APPENDIX C THE CASE STUDY PATTERNS 293 I General G2H Patterns 293 II Programmer A's Patterns 300 III Programmer B's Patterns 308 IV Result Programs 311 V Intentions of Result Programs 314
xiii LIST OF ABBREVIATIONS
4GL Forth-Generation Language A&D Analysis and Design Agile RUP Agile Rational Unified Process AM Agile Modeling AOP Aspect-Oriented Programming API Application Programming Interface ASD Adaptive Software Development CASE Computer Aided Software Engineering CGI Common Gateway Interface CMM The Capability Maturity Model CMM-SW The Capability Maturity Model for Software CMMI The Capability Maturity Model Integration COCOMO Constructive Cost Model COTS Commercial Off The Shelf CSS Cascading Style Sheets DFD Data-Flow Diagram DSDM Dynamic System Development Method DSL Domain-Specific Language FDD Feature-Driven Development GNU GNU's Not UNIX GPL General-Purpose Language HOOD Hierarchy Object-Oriented Programming HTML HyperText Markup Language IDE Integrated Development Environment IEEE The Institute of Electrical and Electronics Engineers IGPM The Intentional Generative Pattern Mechanism IP Intentional Programming ISD Internet-Speed Development
xiv KPA Key Process Area LOP Language-Oriented Programming LP Lean Programming No BDUF No Big Design UpFront OCL Object Constraint Language OOP Object-Oriented Programming OSSD Open Source Software Development p. at page PHP Personal Home Page PP Pragmatic Programming pp. between pages PROLOG PROgramming in LOGic PSP Personal Software Process RAD Rapid Application Development RIPP Rapid Iterative Production Prototype ROI Return Of Investment RUP The Rational Unified Process SBD Scenario-Based Development SDP Software Development Process SEI Software Engineering Institute SPICE Software Process Improvement and Capability dEtermination SQL String Query Language TSP Team Software Process UML The Unified Modeling Language VDM Vienna Development Method XP Extreme Programming YANGNI You Are Not Gonna Need It
xv xvi CHAPTER 1
INTRODUCTION
This chapter provides an introduction to this research and this report in general. In the first section, "1.1 Motivation," the background to the research is discussed. Then, the targets that this research aims to solve are provided in section " 1.2 Objectives." Next, the proposed contributions are briefly described in section "1.3 Original Contributions." The last section "1.4 Document Overview" contains information about the organization of this document.
1.1 Motivation
By nature, developing software is the work of making use of digital-world abstract functionalities for real-world applications. Software developers work mostly with this abstraction. Due to this abstraction, software is complex, intangible and invisible [1][2],
In most software projects, a team of many developers is required to develop a single complex software. Since it is intangible, works of each developers in the team commonly overlap. The overlapping requires high co-operation among the team members. In other words, the team must efficiently communicate their thoughts and other abstract ideas about the part of the software they are dealing with (this includes the requirements from the customers). However, efficient communication is difficult because software is invisible (hard to depict and describe). The very high failure rate (31% fail and 53% over budgeted [3]) of software projects is likely a result from these three characteristics of
1 software. In short, software development is a difficult activity because software is
abstract.
To solve or ease the above problem, a number of techniques and processes have been
introduced. Basically, thoughtful planning, which should be strictly followed, is
suggested to overcome the complexity of the software. Although there can be many other
activities, planning usually includes task decomposition and, partly, analysis and design
(A&D). Task decomposition breaks down the complexity. A&D aims to define high-level
solutions resulting in the construction of the architecture and design models (making the software more tangible) [4][5][6]. Documentations are also suggested as a way to
improve communication. There are many forms of documentations, e.g., textual,
documentation, specification, models and so on. These documentations describe or
visualize abstract information about the software that is much easier to understand than
the software itself (the software code) [7]. Both planning and documentation have been
intensively studied, which results in many approaches to "thoughtful planning"
(scheduling, estimation, risk management, configuration management are just a few examples). In addition, tens (if not hundreds) of documentation techniques (requirement
specifications [8][9], flow charts [5], data-flow diagrams (DFD) [10], the UML [11], just
to name a few) have been introduced during the few decades of computing.
Today, software development is facing even bigger challenges. Software projects are
getting bigger because people are expecting more from software. The requirements,
which are the inputs of the software project, are harder to describe [12]. As a result, they
are likely to change when the customers (who provide the requirements) realize that the
2 product software is not desirable (because the requirements are wrongly specified). Even
worse, requirements are also often changed because of the rapidly changing market
situation is rapidly changing [13]. Therefore, process agility has become one the most
"talked about" and one of the most coveted characteristics of modern software
development processes [14]. Simply, software process agility indicates how well a
software process can cope with changes [14]. Software processes or techniques that are
highly responsive to changes are considered agile.
In 2001, a group of software developers, writers and consultants referred to as the "Agile
Alliance" emerged. The movement is often called the "Agile Revolution." The alliance
blamed on traditional means of software development, especially in term of the upfront
effort for both planning and design as well as the intensive use of documentations. Those
processes or techniques are labelled "traditional," "inflexible" or "plan-driven" (as they
define and follow plans) and "heavy" (as they produce documents that are not working
software). The alliance argued that planning and documentation degrade the development
and are not agile mainly because planning resists changes while documentation slows the
response of change since they require extra effort to update. The group also suggests a set
of different processes collectively called Agile processes, claiming that these processes
have better response to change by reducing planning and documentation.
Although Agile processes work in some cases, they have many limitations and problems
of their own. One of the main complaints about the Agile processes is that they are not
adoptable. For example, most Agile processes are only suitable for small projects [15]
[16]. This is likely the result of the absence of planning and documentations [17].
3 Moreover, many Agile processes are incomplete; they provide very brief descriptions
(e.g., Adaptive Software Development (ASD) [13], Crystal Development Methodologies
[18]) which makes it difficult to adopt them.
Much effort has been spent to solve this underlying problem of process agility. Some attempts have been made to improve the agility of the traditional processes (e.g., Agile
RUP [19]) or to improve adoptibility of the Agile processes [20][21], Many have tried to overcome limitations of Agile processes by creating hybrid processes between traditional and Agile processes [15]. So far, the ultimate solution has yet to be found [20],
To conclude this section, the following are the research motivation in the short form:
1. Developing software is a difficult and complex task.
2. Today's business software process needs to be agile to cope with rapid changes.
3. The new approach to process agility is necessary as existing processes are not
adequate. Many of them are too heavy, too light, not flexible or too flexible
(chaos). Many have many usage limitations, or provide incomplete specifications.
1.2 Objectives
The ultimate goal of this research is to find solutions for the process-agility problem.
That is, to find a way achieve agility without having so many limitations, especially in terms of adoption. To reach such a goal, a set of recommendations and/or software development methods will be formulated. These recommendations may be realized using existing techniques or practises. For the ones without existing realizations, techniques will be developed and implemented.
4 1.3 Original Contributions
To understand the problem, an investigation into the sources of changes was conducted.
The results are recommendations in the form of characteristics that a true agile process should possess ("4.3.7 Summary and Additional Comments"). Those characteristics are:
1) Iterative development framework that allows feedback,
2) The utilization of risk management,
3) The appropriate level of stability and upfront effort,
4) The management of changes,
5) Effective means of communication, and
6) Documenting software easily and thoroughly (or "Agile Documentation").
Existing techniques and practises found in many processes are evaluated for the purpose of finding a suitable or acceptable realizations of those characteristics. There are existing realizations for all agile characteristics, except for the last one - "Agile Documentation."
A software-documentation method, "the Intentional Generative Pattern Mechanism"
(IGPM), was proposed for Agile Documentation (See "5.4 The Intentional Generative
Pattern Mechanism (IGPM)"). In the IGPM, a pattern is used to document software development actions by labelling them. The label (the pattern's name) represents or documents the intention for performing those actions. Patterns can be connected together and form a "pattern network" to document more complex intentions or an entire software.
In addition, a pattern is executable; actions of the pattern will be performed when a pattern is executed. This results in the development of software or the generation of code.
5 Since patterns can naturally document an entire software and can be used to automatically generate the software code, they allow software documentation to be done easily and thoroughly, which makes the IGPM a perfect choice for Agile Documentation.
The following are the original contributions of this research in a short form:
1) The identification of characteristics for a truly agile process,
2) The identification of existing software development techniques that realize agile
characteristics,
3) The Intentional Generative Pattern Mechanism for agile documentation, and
4) The pattern programming language as an implementation of the Intentional
Generative Pattern Mechanism.
1.4 Document Overview
This document is organized into eight chapters covering the definition and backgrounds of the problems, the reviews of existing methods and their limitations, to the reinvestigation of the problem and the proposal of the solutions. The following is a quick summary of the chapters:
Chapter 1 provides an introduction and motivation of the problem, objectives of the
research and a summary of the solutions (the original contributions).
Chapter 2 provides background knowledge about the software development process
in general. Chapter 2 also covers more-emphasized discussions of process adopt
ability and process agility.
6 Chapter 3 provides executive summaries of existing processes for both traditional and
agile as well as the reviews and concerns about Agile processes.
Chapter 4 revisits the problem of software process agility as well as the
characteristics of a truly agile process.
Chapter 5 covers the search for techniques for agile documentation and the develop
ment of the Intentional Generative Patterns Mechanism (IGPM).
Chapter 6 provides details about and the development of the pattern language as an
implementation for the IGPM.
Chapter 7 presents a case study of using the pattern language to improve process
agility.
Chapter 8 is the conclusion of the report and the possible future work.
7 CHAPTER 2
BACKGROUND
This chapter provides the necessary fundamental background information. It begins with the basic understanding of the software development process in general, such as what it is, what it is composed of, and what it is for. Then, the discussion will focus on desirable characteristics of software processes that this research aims to improve: the software process agility and adoptability.
2.1 Software Development Process (SDP)
The term "Software Development Process" is a combination of three generic terms. Its definition can be derived from these terms. As a results, there seems to be no widely- accepted definition for the term. The Table 2.1 below shows some of the definitions. This part discusses about the definition of the word "Software Development Process" that will be used in the scope of the thesis.
First, consider the meaning of the word "process." The Merriam-Webster dictionary [22] defines the word "process" as
(noun) - a series of actions or operations conducing to an end; especially : a continuous operation or treatment especially in manufacture.
8 Table 2.1: Some Definitions of Software Development Processes in the Literature Source/Reference Definitions of Process or Software Development Process
Merriam-Webster a series of actions or operations conducing to an end; especially : a On-line dictionary continuous operation or treatment especially in manufacture [22] [software development] IEEE-STD-610 A sequence of steps performed for a given purpose, for example, [23] the software development process Hammer and A collection of activities that takes one or more kinds of inputs and Champy (1993) creates an output that is of value to the customer [24] Olson et al. (1989) A set of actions, tasks, and procedure that when performed or [25] executed obtain a specific goal or objectives. More specifically, a software process is a software development process SEI-CMM [26] The set of activities, methods, and practises used in the production and evolution of software.
9 so we can simply define "Software development process" as
A series of actions or operations conducing to develop software
This definition is simple, but it is far from accurate. In the literature, researchers and practitioners have provided many different definitions for "Software Development
Process" as well as the elements inside it. We will decompose a SDP into smaller elements; then, we will define each element using definitions found in the literature.
Finally, we will use those elements to redefine the "Software Development Process."
In brief, a software development process is described as a framework of actions or activities which are performed, sometimes, in a serial (linear) fashion but they are, in practise, usually performed in more complicated fashions. Also, the description of each action is usually divided into activity and practise.
2.1.1 Process as a Framework of Activities
A software development process has evolved considerably from simple serial assembly lines; the concept of increased added values of a software product, as its development process is enacted, is still valid. Software process activities, on the other hand, do not necessarily occur in series. For example, the spiral development process suggests an incremental development cycle in which tasks are re-performed over and over again during the process enactment [27]. Therefore, a software development process is described as a framework of activities rather than a series of activities. A structure in which all the actions or activities are put together is called "Common Process
Framework" [28], "Process baseline" [29], or "Process Life cycle" [30]. In this thesis, it
10 is referred to as "Process Framework."
The literature shows that a process framework tells us "what to do" and "when to do it."
In other words, a process framework explains how all the activities are put together. This observation supports Leon Osterweil's idea that a process is a kind of program [31].
Process frameworks, found in many processes, are defined based on similar strategies.
These strategies are called "Software Process Model" (or simply "process model") [28].
Examples of process models are "the Waterfall model" [32], "the Prototyping model"
[24], "the Iterative model" [33], "the RAD model" [34] and "the Evolutionary model"
[27][35].
2.1.2 Activities and Practises
As discussed above, a software development process is comprised of a set of actions or activities. In many models, each of these activities is separately described as two separate entities divided by their levels of abstraction: the high level, which defines the rough targets or the goals of what the actions will produce ("what to do"), and the low level, which specifies the detailed procedures ("how to do") including preconditions, the use of tools, and so on. An example of this division is as follows: "Gaining agreement on the problem definition" is "what to do" and "simply listing the problems and see if everyone agrees" [2] is "how to do." This separation allows practitioners to customize or tailor their process since the detailed procedures may be adjusted while still producing artifacts as described in its "what to do" counterpart.
Since an exact line dividing the two levels ("what" and "how") is blurry, the names used
11 to refer to these two levels differ depending on the source. For example, "activity" and
"method" [28], "key process area (KPA)" and "practise" [26][29][36] and "phase" and
"task" [30] have been used by different authors to refer to the high level and the low level
descriptions, respectively. Since some of these terms also have other meanings of their
own, the terms "software process activity" (or "activity" in short) and "software process
practise" (or "practise") are preferred in the scope of this thesis.
It is clear that a process should provide a full framework description, though it is not
quite clear whether it should provide a complete set of detailed practises. Some
processes, such as the Rational Unified Process (RUP) [37] and the Capability Maturity
Model (CMM) families [26][36][38], come with a full description of all practises
(although many of them are optional). Adaptive software development (ASD) [13], on
the other hand, provides detailed descriptions of some practises (e.g., "formal technical
review") but only brief detail for others.
2.1.3 What Exactly is a Software Development Process?
Thus far, we can see that an SDP is composed of a framework of activities and/or
detailed practises for developing software. A SDP can be redefined as follows:
Software Development Process is a framework of activities and possibly detailed practises used for developing software.
Although the definition is now more accurate, it is still too general, and, as such provides
nothing useful. For the definition to be useful, more factors must be considered. First,
SDPs are part of software engineering and every engineering discipline should concern
12 itself with efficiency and quality. Secondly, many SDPs do not focus only on development issues but also include other aspects, for instance, business aspects
(customer relationship [39]) and management aspects (e.g., planing and monitoring [40]
[41]). Considering all these factors, the definition of the SDP can be expanded as follows:
Software Development Process is a framework containing activities and possibly detailed practises used for developing and managing an effective production of a quality software.
2.2 Purposes of the Software Development Process
From the definition stated above, a SDP should describe how to develop software, it should also ensure that the product software is of the desired quality; and it should be efficient enough for the project to finish on time and within budget to be successful.
These three purposes will be further discussed in what follows.
2.2.1 Developing Software
Clearly, developing software is the ultimate target of every SDP. A SDP, therefore, should provide a framework of activities and/or practises used to develop softwares.
At a minimum, the framework contains a set of activities that is carried out to develop software from the gathering of requirements (knowing what is to be produced), analysis and design (finding out how), to the implementation or coding (actually producing the software). These activities are essential for developing software.
In this thesis, those activities that aim directly to develop software will be referred to as
"common activities." Furthermore, "supportive activities" will be used to refer to
13 activities that do not directly focus on the development but on other aspects, for example, to insure the quality of the product and to increase the probability of the project's success.
Every SDP should, at least, contain a description of a process framework and all the common activities. This fact helps distinguish process from other architecture artifacts, such as software techniques (e.g., COCOMO [40]), methodologies (e.g., object-oriented analysis and design [42] and Formal [43]), principles (e.g., Agile modelling [44]) or process improvement models (e.g., CMMI [38] and SPICE [45]).
One good example of this distinction is the CMM family (CMM-SW [26] and CMMI
[36][38]). The CMM does not have a clear description of a process framework. It also does not focus directly on common activities. Instead, its key process areas (KPAs) aim to improve supportive activities and in turn to improve desirable qualities of an existing process. For example, the "Software Project Planning" process area aims to improve repeatability [46]. Note that although "Requirement Management" and "Technical
Solution" target the requirement gathering and analysis activities, they aim only to improve those activities by describing what the ideal target activities should meet.
2.2.2 Insuring Quality
As an engineering discipline, a software development process should provide guidelines or create an environment that enables the development team to produce software of a desired quality level. A SDP should help the development team fulfil the expectations while ensuring that those expectations are technically sound and economical.
In his book [28], Pressman has categorized software quality in two different dimensions or aspects: the production aspect and the product aspect. In the production aspect, there
are two kinds of quality: the quality of design (or quality of the solution) and the quality
of conformance (or quality of the implementation). In the product aspect, software
qualities are grouped into explicit qualities (functionality, features and so on) and implicit
qualities (unmentioned general constraints such as maintainability, reliability, usability
and so on).
The Quality of Design and the Quality of Conformance
Pressman further defines the quality of design as "the characteristics that designers
specify for an item" or the quality of the design specification, which is the measure of
how well the proposed design solution is able to fulfil the requirements or to meet the
needs of the customers. The quality of conformance, on the other hand, is defined as "the
degree to which the design specifications are followed during manufacturing" or the
quality of the realization of the specifications. The definition of the quality of
conformance is self-explanatory.
To insure the quality of design, the specification should be tested against the
requirements. Some examples of these activities are the acceptance test in extreme
programming [39]; these activities are also known as "validation tests." To insure the
quality of conformance, on the other hand, the implemented code will be tested against
the design specifications. The best example of these kinds of testing is the unit testing
[47]; these activities are commonly known as "verification tests."
The Explicit and Implicit Qualities
Referring to Pressman's definitions, the difference is that explicit qualities are mentioned
15 while the implicit qualities go unmentioned. Being mentioned or unmentioned depends largely on conscious decisions of the stakeholders. These conscious decisions are largely subjective, which makes this distinction unclear. To find a clear distinction between the two, it is helpful to consider "measurability" (ability to be measured). If a quality is unmeasurable or difficult to be measured, it is usually difficult to specify and hard to prove that the quality is of the desired level and that is why it is usually left unmentioned.
Measurability, thus, appears to be a good indicator of the distinction mentioned and unmentioned qualities. Therefore, in this thesis, we will refer to explicit qualities as measurable qualities and implicit qualities as unmeasurable qualities.
Implicit qualities are often very abstract and subjective; everyone seems to have their own idea of what they are. Example of these quality are "performance" and "usability."
Since they are not easy to measure and always left unmentioned, they normally appear to be different in everyone's mind. The differences often result in surprise and unacceptable products (from the customers' perspective), as well as costly reworking (from the developers' perspective). Furthermore, many of these qualities are expensive to implement and especially to add at a later stage. Therefore, it is very important to ensure that the desired level of each quality is clearly defined, mentioned and accepted by all stakeholders. In other words, implicit qualities should always be mentioned. As an example, expandability can be very costly to add into the product at a later stage.
2.2.3 Increasing the Odds of Success
Software development is one type of constructive activity. Most constructive activities require material costs to be paid as an investment expense. Although developing software
16 does not require such advance material costs, it does require some investment expenses, mostly for the development team (e.g., effort and time). Often, the inability to produce the desired or acceptable product means the loss of the investment. Additional to the economic loss, the team often loses their reputation as well as their self-confidence.
These losses are sometimes worse than the economic loss because it may lead to future economic loss or even jeopardize the business. Therefore, it is vital for an SDP to provide the development team the means to create an appropriate environment or facility that increases the probability for the project to be successful (or decreases the probability for the project to fail).
There are many different strategies to increase the probability of success. The following are four such strategies.
Synchronize the Development Team
People are undoubtedly the most important resource for most organizations, especially for creativity-oriented organizations like software development teams. As many have pointed out, talented people are very valuable to the team [48], However, having talented people in the team cannot be achieved entirely through an SDP. An SDP should provide an environment where the individual skills of the team's members are synchronized and complement each other similarly to the resonance of harmonic waves; in other words, an
SDP should contain guidelines to create effective teamwork. A development team in which its members' skills complement each other is referred to as a "jelled" team [49].
A number of factors are involved in creating a jelled team. Some of these factors are not controllable through SDPs. Individual preferences, cultural backgrounds, communication
17 skills, and personalities-to-roles appropriateness are examples of these factors [48][50].
From the literature, two common tactics used in SDPs to synchronize the team are
(1) specifying "who does what and when" and (2) improving communications amongst team members.
Roles and Responsibilities A SDP should ensure that everybody in the team knows what to do, when and by who.
This should cover "Who does what" and "When to do what."
Specifying "Who does what" is assigning a role to each individual so that everyone in the team knows who is responsible for any particular task. This usually involves two activities: assigning roles to tasks and individuals to roles. Most SDPs found in the
literature provide descriptions of roles and responsibilities (defining roles and tasks), for example, ScrumMaster [51]. Roles and responsibilities are defined based on activities in the processes which make the defining an essential part of the process. Assigning people to the roles to responsibilities, on the other hand, is left for the adoption of the process by a project, which is part of the process tailoring. This job is often performed by the leader of the team, but sometimes a volunteering system many be used. Some processes even specifically suggest the latter approach, e.g., Extreme Programming [39],
Similarly to specifying "Who does what," specifying "When to do what" also involves process description and tailoring. In the process description, "When to do what" is expressed in the form of the process framework. On the other hand, activities such as
planning and scheduling are performed during the adoption of the process.
18 Knowing "Who does what" and "When to do what," allows the team members to perform and respond appropriately. It tells the team members who is responsible for any potential problems and whom to turn to. It also provides the path or steps that the team members should follow and perform together synchronously. When performed appropriately, this knowledge will enable individual skills of the team to complement each other and likely provide effective results. In other words, the team becomes a jelled team.
Effective Communication In addition to specifying roles and responsibility, efficient communication between team members is another factor necessary for a team to become a jelled team. The team members, as well as other stakeholders, should be able to express their thoughts to others.
From the literature, efficient communication involves the following factors: (1) individual communication skills, (2) the communication media and (3) communication environments.
Individual Communication Skill Individual communication skill is understood as how well a person can express concepts in a clear and concise way. Since most individual skills are not directly controllable through an SDP, there seems to be no SDP that improves communication skills directly.
To improve team-member communication skills, an appropriate training program may help. Screening criteria that covers communication-skill assessment of new employment may also be used. However, some SDPs provide activities that improve the skill indirectly or reduce the effect of the lack of such a skill.
An example of the first case is the pair programming of XP [52], Pair programming suggests that all the coding is performed by pairs of programmers. The pairing up
19 promotes more communications between the programmers; increased communication
hopefully means more experiences gained and enhanced communication skills. Also, the
pairs are given more opportunity to learn about each others' communication preference
(e.g., terminologies, backgrounds and so on).
Another case involves specifying what information needs to be passed on in certain
situations or steps. The passing of information may be performed with or without specified targets such as by oral or written informing and permanent documentation,
respectively. Specifications are usually defined and described as parts of the process
framework or process activity descriptions. If this technique is used appropriately, the
necessary information will be passed on. However, specifying too much or too little
information to be passed on may result in inefficient or useless communication,
respectively. Some flexibility might be used to reduce the effects of improper
specifications.
Communication Media Communication media are forms of information representation or materials that hold
information [53]. Different types of media have different qualities that affect the emphasis of information aspects, (i.e., the way they are produced, maintained and so on).
Appropriate use of communication media greatly supports efficient communications.
Communication Environments Communication environments are environments where communications occur. They may
be places (e.g., the customer's site), atmospheres (e.g., contract negotiations),
communication channels (e.g., over the phone) or events (e.g., weekly meeting).
Communication environments affect a number of different communication factors. Such
20 factors include tone, information accuracy, clarity and so on. Most SDPs do not explicitly specify the proper communication environments, although some may be specified implicitly. However, a few SDPs do suggest or even specify the communication environments. For example, planning game activities should be performed in a single physical location (the developers' place) where customer representatives and programmers can define, organize, estimate and voluntarily assigned user-stories. In his book, A. Cockburn provides an interesting discussion about communication environments [54].
Controlling the Direction of Development
A SDP description should include a way to ensure that the development is going forward without leaving anything out. In other words, the team should do everything they are supposed to do and should not do anything they are not supposed to do.
In most cases, this is done by identifying and specifying the development steps that the team can follow. These steps enable the team to progress the development smoothly without forgetting any jobs. The team can also check what they have done against the specifications.
Since a SDP description contains the development steps in the form of the process framework and activities, using a SDP is the first step in controlling the direction. These activities usually cover what, when and who. The framework should also cover what to do in case something goes wrong. Selecting an appropriate SDP depends on many different factors. This topic will be discussed later.
21 In order to control the direction of the development, a SDP is selected, customized or
tailored to fit the project. Then the project plans and schedules are developed, followed
and (if needed) adjusted.
Although a process framework only reflects the SDP concepts or principles that the
process designers focus on or care about, it is written without concern for adopting in a
project and an organization. Therefore, process framework must be customized or
tailored to suit the project and the organization. Some SDPs provide guidelines or
sometimes process variations can be used for the tailoring of the process, for example, the
feasibility study of the Dynamic System Development Methods [4].
After tailoring, planing and scheduling will be conducted at the beginning of the project.
In most cases, the scope of the project is, firstly, defined from the requirements. Then, the
scope will be analyzed and estimated against the team and the project characteristics.
Other risks (e.g., technical feasibility) may also be analyzed as a part of risk management
activities. The product is, then, broken down to smaller tasks that. These relatively
smaller tasks are individually estimated and assigned to the team members. Deadlines of
each tasks as well as project milestones are defined. In short, plans and schedules should
be defined and followed. They may be adjusted later as seen appropriate. The above
process of scope analysis, estimation, task decomposition, planning and scheduling are found in many existing SDPs (e.g., RUP [55] and Cleanroom [56]).
Increase Visibility
According to his article [57], Watts S. Humphrey argues that one of the big problems in
managing software process is the lack of management visibility. It is tough to see where the project stands, he said. Since software is an abstract artifact, it is challenging to visualize the desired product and the being-developed product. It is also difficult to measure the real status of the project such as its progress, problems and so on.
A SDP should provide a way to ensure that the team is obtaining accurate data about the development status. It should include activities or practises that describe how accurate data can be obtained and how to make use of said data. The following are examples of such activities:
• Requirement management - Describing how the team can obtain the most
accurate information about the product to be produced (e.g.,[58], [12]).
• Risk management - Describing how foreseeable risk can be identified and
managed (e.g., [59]).
• Change management - Describing how to handle change (e.g., [60], [14]).
• Process monitoring - Describing how to measure the project's progress using the
schedules.
A SDP can also suggest an appropriate way to represent the software being produced,
"use only code as the representation of the software" [13] is one such example. If the software is appropriately represented, it may better reflect the status of the developer and may make measuring progress easier (See "Communication Media" on page 20).
Development-Effort Optimization
In order for a project to be successful, the development team should deliver a desirable product on time and on budget. One way to achieve this is to be as efficient as possible.
23 In other words, the development effort should be optimized. It is worth emphasizing that
the optimization should be done without or with less sacrifice to the product's quality.
There are two common ways to optimize the effort: (1) reduce unnecessary work and
(2) increase productivity.
Reduce Unnecessary Work We should first define "necessary work." Necessary work is the set of activities required
for a project to succeed. It may directly be part of the final product or a necessary step to
produce part of the final product.
Explicit Necessary Work It is easier to identify the first type of necessary work. Sub-components of the final
product should be identified first; then, the work of producing each of these components
is necessary. If the team wrongly identifies these sub-components and the work of
producing the components was already performed, the team will need to rework it to fix
those components. Rework can result in high costs later in the development [14],
To reduce the amount of rework, some SDPs suggest the prevention of rework. In other
words, the team should prevent reworks from occurring or let them occur as little as
possible. Basically, the team should first identify the final product by doing requirement
management. Then, they should separate the final product into a set of sub-components
(this is done in analysis activities). Finally, the team should produce those components
correctly (design activities facilitate this). The efficiency of each, as well as the
combination of these activities, is essential to prevent or reduce the amount of reworks.
The Rational Unified Process (RUP), for example, prefers to focus on the requirement
24 analysis by navigating beyond the requirements (provided by the customers) into the real intentions and business values of those requirements [61]. On the other hand, CMMI suggests that the team should perform thorough analysis of all possible solutions so that the right sub-components can be identified [36].
Some modern SDPs take a very different approach by trying to reduce the efforts to rework instead of preventing them. An example of this type of SDP is eXtreme
Programming (XP); XP suggests to make changes easy by using "code as documentation" (also called "self documentation" or "natural documentation"), "code standards" and "collective code ownership" principles [16].
Implicit Necessary Work The second type of necessary works are the works that do not directly produce the final product parts but that are necessary for the production of the parts. Identifying these works is relatively harder than the first type and largely depends on the framework of the
SDP.
Activities such as planning and scheduling facilitates the optimization by controlling the direction of the development (see the previous section). Since the planning activities somehow depend on the framework of the SDP used, the choice of the SDP used is an important factor. Some activities or practises are defined as necessary in some SDPs but defined as unnecessary in others. For example, producing written documentations is considered necessary in CMMI [36], while they are considered optional in Extreme
Programming [62],
25 Increasing Development Productivity Increasing development productivity is a straightforward way to optimize the development effort. Techniques commonly used are reuse, design patterns [63][64], domain-specific language (DSL) [65], graphical modelling [6] and others.
2.2.4 Additional Comments
Good SDPs should satisfy all of the above purposes; they should provide a guideline to develop software; they should ensure that the product is of acceptable quality; and, they should also increase the odds of success of the project. To conclude, this subsection provides common purposes of using SDPs in a software project.
2.3 Software Process Adoptability
Every SDP has limitations (constraints or pitfalls) [1], These limitations prevent the SDP from being used in certain scenarios or reduce the performance greatly [15]. For example, eXtreme Programming (XP) requires that the customer representative be at the development site [66], This requires that the customers must be willing to spend time at the development office [21]. Since using a SDP out of its limitations may result in the failure of the project, it is important to determine if a process is suitable to a particular scenario (the combination of a project and an organization) [15].
When a process has no limitations against a development scenario, it can be adopted by the scenario. The less limitations a process has, the wider range of scenarios that the process can be adopted to. It is clear that some SDPs are more adoptable than others. To measure that, a set of characteristics of software development scenarios should be defined as the evaluation criteria. Then, "Software Process Adoptability" can be measured by30
26 determining the set of characteristics of scenarios in which a process can be adopted to.
The following is the list of those characteristics and the justification for how they can be used as the evaluation criteria for determining process adoptabiltiy.
2.3.1 Project and Organization Size
The size of a project is often associated with the technical complexity, while organization size is associated with management complexity. For example, XP's code standard requires that all code must be written in the standard so that it can be easily read by the whole team [66]. The bigger the team, the harder to compromise every team members' coding styles [21].
2.3.2 Experience
The organization's experience on similar projects normally allows them to foresee and to be prepared for difficulties commonly associated with these projects. The team may even have reusable code for the previous projects. Some processes require planning ahead, which will be more suitable in scenarios where the developers can foresee the development steps and possible problems. For example, RAD requires the decomposition of the requirements into a set of software components before developing each of the components independently [14]. If this decomposition turns out to be inappropriate, the system may not function as expected. Experience can, hopefully, reduce the chance of inappropriate decomposition.
27 2.3.3 Technical and Legacy System Constraints
These constraints largely involve development freedom. Some processes, for instance, prefer or require a development paradigm over the others. For example, the rational unified process (RUP) prefers the UML and the OOP paradigm and may not be suitable to, for example, a declarative programming language such as PROLOG. Legacy system constraints involve existing code.
2.3.4 Business Environment
The business environment is one of the factors for requirement changes. Some processes are more adoptable to a rapidly changing or chaotic environment (e.g., eXtreme
Programming [66]) while some are for the more stable business or market environment
(e.g., the Cleanroom Development [56]).
2.3.5 Requirements
As with development input, well-understood requirements make development easier.
Tasks such as analysis, task decomposition and distribution, designing, planing, progress tracking, and configuration management, and many others are much easier to perform when the requirements are well-understood; this is the opposite in the case of the ill- understood requirements. In the latter case, more effort should be spent to better understand the requirements. SDPs that promote or allow technique such as the prototyping technique will be more adoptable to the latter type of requirements [28].
2.3.6 People
There are three common categories of people involving in software development:
28 customers, developers and managers.
Customers
Commitment and Collaboration: Some processes such as XP and DSDM require a large degree of customer commitment and collaboration throughout the development [39] while some processes require much less collaboration [15]. For example, in RUP, the customers will be involved only at the beginning and the end of the development [55].
Knowledge about software development: In virtually every process, the customer dictates the requirements. Writing the requirements needs the understanding of both the business and the software development. Some processes follow the customer requirements without question. To be successful, this type of processes requires the customers who understand software development, e.g., XP [39]. On the other hand, some processes suggest that the developers can help to redefine and improve the requirements.
For example, the Scenario-Based development suggests that the developers should visit and observe the real working environment where the software will be used to help redefine the requirements [58], Another example is the construction of the vision document in Rational Unified Process [37][61].
Developers
Skills: As many have pointed out, different processes require different types of people
[15][18]. A. Cockburn has proposed five different levels of developers based on their technical skills [15].
Personality: Teamwork is very important in software development [49][13]. Different processes have different ways to deal with teamwork issues. This requires people with
29 different personalities. Many Agile processes, for example, make use of "Pair
programming" practise in which programming is always done by a pair of programmers.
Therefore, the programmers who prefer working alone may not be comfortable and,
therefore, will be less productive. Desirable personality traits include amicability and
good communication skills.
Expertise: People have different expertise; in other words, Each person may be good at
in different areas. Software processes address the use of expertise differently. Examples
of various ways of using personnel expertise in SDPs are: the surgical team [ 1 ] and pair
programming [39]. The surgical team method supports the idea that people should be
assigned to work in their area of expertise, while "pair programming" promotes the
circulation of tasks and spreading expertise [67].
Managers
In different SDPs, project managers have different roles or tasks. These differences
require people that have different experience, energy, personality and skill sets. For
example, in Rational Unified Processes (RUP), the manager (at the beginning) works
with the customers and the development team to develop the problem scope and all initial
project planning. Then, the manager will develop the software development plan (more detailed). The manager also monitors the progress [55] while, in extreme Programming
(XP), the manager's responsibilities are more similar to project co-operator and an XP coach [52][68] (less planning and scheduling as well as tracking tasks).
2.3.7 Quality and Reliability Constraints
In some projects, quality and reliability issues are critical (for example, the production of
30 medical-related applications). Some processes make use of widely-accepted quality assurance techniques so they can be used for those projects. On the other hand, some projects do not require high degrees of quality and reliability. Rather, they prefer quick delivery or a small development budget and acceptable quality. For example, Cleanroom development aims to produce high-quality products [56] while Rapid Application
Development (or RAD) focuses mainly on shrinking the development time [69][34],
2.3.8 Project Resources
As found in the literature, project-resource constraints cover two aspects: availability of the resource and the resource priority. The number of developers required in a project varies in different processes. Time and budget are also factors. Some process frameworks take a longer time to be performed or may have steeper learning curves. Some practises or methods may take a long time to master. The second aspect involves the priority between project resource and other elements such as functionalities. Many Agile processes embrace the idea that the product functionalities can be varied. In the Dynamic
Systems Development Method (DSDM), the amount of time and budget is fixed while the functionality is adjusted accordingly [69]. This idea may not suit most projects.
2.3.9 Process Infrastructures
Many processes require appropriate infrastructure to be effective. Tools and operational environments are commonly required infrastructures.
Tools: In this context, tools refer to development tools, such as, modelling, testing and version-controlling tools. Although some of these tools may not actually be required, they
31 are highly recommended. For example, RUP suggests using UML [61], which means that
UML tools are needed but, in case of a small project, the tools may not be required as the developers can manage the small amount of hand-drawn diagrams [55].
Operational Environments: Some Agile processes require a specific-work setting. For example, XP highly suggests that all the programmers should work in the same room
[16]. Another example is the Crystal development, which highly recommends that the team should have means for communication such as white boards or bulletin boards [18].
2.3.10 Culture
In different assorted, people treat each other differently. These differences may be the reason a practise works in one culture but does not work in another culture. For example, pair programming may not work in the east where arguments with older colleagues (even by a few years) or people in the higher ranks are considered to be disrespectful [50].
2.3.11 Additional Comments on Software Process Adoptability
Good SDPs should have as few limitations as possible. In other words, good processes should be adoptable to many types of software development process scenarios. Such a process does not seem to exist [1]. A good example of the conflict is "requirements" and
"size and complexity;" processes that are suitable to fast-changing requirements are often not appropriate to large or complex projects [20].
32 2.4 Software Process Agility
2.4.1 Definition
According to Gage Canadian Dictionary [70], the term agility means "the agility to move quickly and easily; activeness; liveliness; nimbleness." "Software Process Agility" should, therefore, mean something similar to "the degree in which a software process can quickly and easily change its direction." This is similar to the definition found in the literature [19] that "Agility, for a software-development organization, is the ability to adapt and react expeditiously and appropriately to changes in its environment and to the demands imposed by this environment." In short, process agility is about responsiveness to changes.
2.4.2 History
Software process agility has been around for decades as software processes provide mechanisms for reacting to or reducing the effect of changes (e.g., feedback and change management) [4]. Nonetheless, agility was recently brought to the software community's attention around the end of the last decade. A group called the "Agile alliance" heavily criticized the software development approaches of the time as poorly responsive to change. The term "heavy" and "inflexible" are used to describe the traditional approaches. The group also presents the new approach for achieving agility based on a set of basic principles called the "Agile Manifesto" [71].
The group proposed four essential values called "Agile Manifesto" (the history about the construction and reasons behind it are described in details in Appendix A [18]) as follows:
33 Individual and interactions over processes and tools (J) Working software over comprehensive documentation (2) Customer collaboration over contract negotiation (3) Responding to change over following a plan (4)
2.4.3 How is Process Agility Useful to Software Development?
Process agility helps increase the odds of success by reducing unnecessary work. When change occurs, the direction of development is changed. Necessary work may become unnecessary and vice Versa. Being agile means that the project can adapt to the new status more quickly. For example, no further unnecessary work (that used to be necessary) are performed and the work that becomes necessary (that used to be unnecessary) can be identified and performed in timely manner. In essence, in achieving process agility, the process optimizes the effort by reducing unnecessary work, the result of later changes.
2.4.4 Why is Agility Important?
An analysis of the literature reveals two main reasons why software process agility is an important or desired characteristic: (1) increasing software complexity and (2) rapidly changing demands.
Software is becoming more complex as people expect software to do more. The complexity (together with intangibility and abstraction) makes it difficult for customers to specify their wants [12]. In other words, it becomes harder to define the accurate requirements. As a result, the requirements are often redefined or, sometimes, refined during the development as the customers gain more understanding or simply "just realize" that the previously given requirements are not the right ones.
34 The rapidly changing nature of today's Internet-based economy demands flexibility and speed from software developers. It is very logical that software requirements quickly change to respond to ever-changing demands.
2.4.5 Achieving Agility - The Agile Approaches
From the perspective of the agile alliance, process agility is more than an effective response to changes. It also encompasses the philosophy derived from the agile manifesto. From that, the alliance defines twelve principles for achieving agility [71]:
1) The highest priority is to satisfy the customer through early and continuous
delivery of value software.
2) Welcome changing requirements, even late in development. Agile processes
harness change for the customer's competitive advantage.
3) Deliver working software frequently, from a couple of weeks to a couple of
months, with a preference to the shorter timescale.
4) Business people and developers must work together daily throughout the project.
5) Build projects around motivated individuals. Give them the environment and
support they need, and trust them to get the job done.
6) The most efficient and effective method of conveying information to and within a
development team is the face-to-face conversation.
7) Working software is the primary measure of progress. Agile processes promote
sustainable development.
8) The sponsors, developers, and users should be able to maintain a constant pace
35 indefinitely.
9) Simplicity - the art of maximizing the amount of work not done - is essential.
10) Continuous attention to technical excellence and good design enhances agility.
11) The best architectures, requirements, and designs emerge from self-organizing
teams.
12) At regular intervals, the team reflects on how to become more effective, then
tunes and adjusts its behaviour accordingly.
2.4.6 Additional Comments on Process Agility
Process agility ultimately involves responsiveness to changes. It helps increase the possibility for a software project's success. Process agility is an important characteristic of every modern software process because today's market demands are rapidly changing.
The importance of process agility is obvious, but the approaches for achieving agility is questionable. See section "3.4 Problems of Agile Processes" for further discussion.
2.5 Conclusions
This chapter provides general discussion about the software development process and two desirable characteristics, namely, process adoptability and agility. The next chapter will provide reviews and summaries of existing software development processes.
36 CHAPTER 3
EXISTING SOFTWARE DEVELOPMENT PROCESSES
This chapter provides surveys of existing software development processes (SDPs). From the literature, existing SDPs can be categorized into two general categorizes: traditional and Agile processes. Common characteristics of processes in each categories as well as summary of some processes are provided.
3.1 Introduction
At around the beginning of this decade, there was a movement often referred to as "agile revolution" started by a group called the "agile alliance" [71]. The group criticized the old idea of software engineering (which they called "traditional approaches") and proposed the new idea focusing on the process agility or a quick response to changes that occur during the development. This movement has arguably marked the new age of software engineering. These approaches are collectively called the "Agile process."
This section aims to discuss existing approaches for both the traditional and the Agile processes. Firstly, common characteristics of both approaches are discussed. Then, the executive summary of some processes are provided.
3.2 Traditional Software Development
In the early days of computing, software development was not considered to be an
37 engineering activity (the development was considered art or craftsmanship of some kind
[41]). In his book, Winston Royce described what is known as the waterfall model for
software development [32], The model gave hope that software development and
management can be done systematically (as an engineering discipline). Since then, many
other techniques and practises have been proposed. Many of those techniques have failed.
However, some have survived and become the best means to develop software. Since
most of these techniques aim to make software development systematic, they share many
similarities. These techniques and approaches are called "Traditional software
development approaches."
Since the traditional approaches were introduced and refined by many contributors spanning over several decades, it is, in some cases, hard to pinpoint exactly how each
activity was originally proposed and by which contributors. It is also tough to describe
each technique in detail since there are large variations within each technique. This section will, therefore, provide brief descriptions of what traditional approaches look like.
What follows is a few examples of traditional processes discussed in more detail.
3.2.1 Common Characteristics of the Traditional Software Development
System Engineering
The traditional approaches consider software engineering as part of a system of engineering and consider software as part of the whole system product [28]. Software projects are sometime referred to as "computer-based" systems.
Modelling
The traditional approaches utilize a number of modelling techniques. Modelling allows
38 abstract entities such as concepts, ideas, and software to be embodied as more tangible entities. It also makes software more visible and, ultimately, shareable. In other words, modelling enables developers to make sense of the software they are developing [4],
Examples of modelling techniques are system modelling [28], flowcharts, data-flow diagrams [10], entity relationship diagrams [72], Hatley-Pirbhai modelling [28], the UML
(a widely accepted standard for software modelling) [73] and many others.
Formality
To be systematic, traditional approaches favour formality. Plans and scheduling, require ments, designs, testing, risk management, configuration management and other activities are formally and comprehensively performed, reviewed, signed off and well-recorded.
Specifications and Documentations
Engineering often happens in top-down fashion where high-level solutions are specified.
This specification will be passed on to other team members and the lower-level tasks will be performed by many people (as it has more detail). The specifications are preferred, in many cases, to be complete and stable. Examples of specifications are system specification [28], formal specifications in Cleanroom development [74].
Traceability
Traceability is a very important characteristic for traditional processes [75]. Regardless of its domain of focus, the traditional processes encompass a collection of top-down and bottom-up to allow tracing and navigating through all the production elements [28].
Planning
Traditional approaches also make intensive use of planning. Planning provides a road
39 map of the development where everyone is aware of what is coming [41]. There are many kinds of plans. Examples of planning are SQA plan [28], risk management (plan for things that may happen) [28][59] and others.
Estimation
Estimation of size, cost and development time are often required prior to the planning and scheduling. The result of estimation will be used in planning activities such as risk management and scheduling (the estimation of time). Software-development estimation usually begins with software decomposition and software size estimation. The estimated size will later be used for cost and time estimates. Most of the estimation techniques are empirical and numerical-based. It makes intensive use of statistic from past data. A powerful example of estimation techniques is COCOMO [40].
Development Process Model
A variety of process frameworks or process models are used in the traditional processes.
The following are some process models and their short descriptions:
1. Waterfall Process Model [32] - Activities (planning, requirement, analysis and
design, implementation, and testing) are performed in linear fashion (finish one
first, then begin another). Some of the variants of the waterfall model are:
(1).the waterfall with feedback (e.g., the V Model) [76] - the development and
testing are performed in parallel
(2). the overlapped waterfall model [4] - consequences activities may be
performed before the previous ones finish, and
(3). the waterfall with prototype. - Rapid prototypes are created in order to
40 understand the requirement better and, sometime, to verify feasibility (e.g.,
Scenario-Based Development [58]).
2. Iterative Process Model [33] - The development is decomposed into iterations.
Each iteration produces deliverable increments. This implies that the requirements
and scope of the system should be well understood. In each iteration, activities are
performed in linear fashion. Example processes that utilize iterative development
are the rational unified process (RUP) [55] and RAD [69]. A variation of the
iterative model is the Rapid Application Development or RAD Model. RAD
suggests that if the development is well decomposed, each the iteration can be
performed in parallel [28].
3. Evolutionary Process Model - The main idea of the evolutionary process model is
that software evolve or grows over time [27], Evolutionary process model are
iterative. The first iterative may produce the prototype or mock-up of the system.
In the later iterations, the prototype is refined. More functionalities are added in
the product at later iterations.
Requirement Gathering
Based on the process models, most traditional approaches suggest that the requirements gathering should be performed only at the beginning and stabilized throughout the project
(although many processes accept requirement changes but the changes are considered costly). The requirements should be well understood and clear [30]. Some processes suggest that the development team should comprehend the real needs (usually refers to the business needs) of the customers (the vision document in the RUP [55][77] and field study [58] rather than just the requirements specified by the customers. The requirements
41 should also be formally recorded (requirement specification) [14].
Additionally, rapid prototyping is common [69]. Prototyping aims for better comprehension of the requirements (especially for human and machine interaction) [58].
Many suggest that the prototype should be thrown away because it is usually built rapidly without proper design [14].
Development (Analysis, Design and Implementation)
Most traditional processes encourage the development of architecture (high-level solutions) that are traceable to the requirements. They also suggest that all or a significant amount of detail of designs should be done prior to development (this statement is valid in each iteration of the iterative or evolutionary model). The effective design should be traceable to the architecture. The designer should utilize abstraction and modularity (high cohesion and low coupling) to allow changeability and expandability [14].
Some processes intensively utilize reusable components (also abstract and modular). For example, RAD [14] and RUP [77], Another important technique is the formal method
[30]. The formal method is not actually a method for analysis and design but the method for creating requirement and design specifications (Z [78], VDM [79]). It can also be used as a software-development technique by continually refining the specification until there is enough details to produce code. The specification may be represented using a mathematical or similar representation [14] (OCL [80]), which is consistence, complete and unambiguity [14].
42 3.2.2 Traditional Software Development Processes
The following are some of the traditional software processes that are still well known as
they appear in recent literature.
The Capability Maturity Model (CMM) and the Capability Maturity Model Integration (CMMI)
The Capability Maturity Model (CMM) [26] is a process assessment and improvement
frameworks. It contains information and guidelines for organizations to evaluate and
improve their process. The Capability Maturity Model Integration (CMMI) is the
successor of the CMM. The CMMI allows the integration of multiple disciplines of the
CMM. Since the differences between the CMM and the CMMI are not important for the
discussion in this Thesis, the term "the CMMI" will be used (except explicitly specified)
to represent both the CMM and the CMMI.
The ultimate target of the CMMI is to improve process capability maturity. The process
capability maturity (or process maturity) indicates that the process is capable of reliably
producing desired products. The more mature processes, the more likely to produce
desired products. In CMMI, the process maturity is divided into five levels: 1) Initial,
2) Managed, 3) Defined, 4) Qualitative Managed and 5) Optimizing.
Only the CMMI has two different representation models: stage and continuous. In the
stage mode, sets of process areas are required to be implemented (the goals are satisfied)
in order to achieve maturity levels (similar to the CMM) (see the Figure 3.1 below). As
mention above, there are five maturity levels as mentioned above. On the other hand, the
continuous model is divided into six capability levels: 0) Incomplete, 1) Performed,
43 CMMI3" Stage Representation
5 Optimizing Organizational Innovation and Deployment OC Process Mgt Casual Analysis and Resolution CAR Support
3 Defined Requirements Development Engineering Technical Solution TS Engineering Product Integration PI Engineering Verification VRT Engineering Validation VAL Engineering Organizational Process Focus OPF Process Mgt Organizational Process Definition OPD Process Mgt Organizational Training OT Process Mgt Integrated Project Management IPM Project Mgt CMMPM Continuous Representation Risk Management RSKM Project Mgt Decision Analysis and Resolution DAR Support Integrated Supplier Management ISM Project Mgt Organizational Environment for Integration OEI Support Integrated Teaming rr Project Mgt
Proem Areas
Figure 3.1: CMMI Stage and Continuous Representations
44 2) Managed, 3) Defined, 4) Quantitatively Managed and 5) Optimized. Each capability level is achieved by satisfying goals of all process areas (or all process areas in the selected disciplines) [38][36] of that particular level.
Table 3.1 shows an example of a process area called "Technical Solution." Its goals and practises are also shown.
Although the models are not actually a process, they provide activities (called process areas) with goals and practises that an organization should adopt in order to improve its process maturity. The CMMI does not specify any process framework to be used (this is a common misconception about the CMMI [81]). The CMMI address process framework in level 3 (the defined level) where whatever process model are in place, it should be documented and standardized (Organizational Process Model Definition [38]).
Nonetheless, the CMMI are often considered as software process because of its comprehensive details of activities and practises (in the form of process areas). The process areas are described in very comprehensive detail as descriptions (approximately
500-page documents [38][36]).
There are other software-process assessment and improvement models. Some examples are ISO 9001:2000 for Software [14], SPICE (ISO/IEC 15504) [45] and ISO 12207 [82],
They are all similar in concept.
Cleanroom Development
Cleanroom development is an approach that aims to produce extremely high-quality software and was originally introduced by Mills [74]. The philosophy behind the
45 Table 3.1: Goals and Practises of "Technical Solution " Process Area
Goal 1 - Select Product-Component Solutions Practise 1.1 - Develop Detailed Alternative Solutions and Selection Criteria Practise 1.2 - Evolve Operational Concepts and Scenarios Practise 1.3 - Select Product-Component Solutions Goal 2 - Develop the Design Practise 2.1 - Design the Product or Product Component Practise 2.2 - Establish a Technical Data Package Practise 2.3 - Design Interfaces Using Criteria Practise 2.4 - Perform Make, Buy, or Reuse Analyzes Goal 3 - Implement the Product Design Practise 3.1 - Implement the Design Practise 3.2 - Develop Product Support Documentation
46 approach is to avoid the defect detection and removal by writing develop each increments
"right the first time" and verifying their correctness before the testing. [83]. In other words, rather than producing a working software then detecting and removing the defect as seen in other approaches, Cleanroom development suggests the development should be clean or defect free [56].
In Cleanroom, the requirements are gathered and analyzed to define system specifications
(inputs and outputs) using formal representation. The software will be incrementally developed (design) using the "Box Structure Specification" technique (or the formal method) [74]. At the same time, user-function test cases are developed. Then, the designs will be verified for the correctness using a technique called "correctness verification"
(similar to formal review) and the user functions will be statistically tested to validate the requirements before being integrated into the whole software.
Pipeline Process Model The Cleanroom process model is a pipeline of user-function software increments that is accumulatively integrated into the final product [83] (see the Figure 3.2 below). Each iteration encompasses development and certification of both user functions and the designs simultaneously. As an iterative development model, the model enables early and continual quality assessment and user feedback and avoids the risks associated with late integration. With the iterative model, systematic management and the incorporation of requirement changes are possible. It also facilitates process improvements as the development progresses.
Formal Method and Quality Assurance Cleanroom adopts the formal method as its development method. The requirement and
47 Customer Requirements f
Specification
Function Usage
Incremental Development Planning
Functional User Specification Incremental Specification Development Plan l Formal Design I ! Random Correctness j ! Testcase Verification I Generation
Source Test Code Case
Statistical Testing
i Interfall Times y (Failure Data) Improvement 1 Feedback I , Quality j Certification I Model !
Mean Time To Failure (MTTF) Estimates
Figure 3.2: Cleanroom Life Cycle - the Pipeline Process Model
48 design specifications are represented using formal language. This allows correctness to be
verified in a formal review activity conducted during every iteration of refinement (in the
box-structured specification). This intensive use of reviews supposedly guarantees the
correctness of the software. Therefore, there is no or little need for unit testing.
Furthermore, since Cleanroom makes intensive use of design and code reviews, the
resulting product should be defect free. Therefore, the testing is performed only to
validate software requirements. Cleanroom suggests statistical testing. The development
team must first define every possible way that users may use the software. Then, a usage-
probability distribution is determined. Test cases are generated for each stimulus
(samples of inputs or events) according to the usage-probability distribution.
Rapid Application Development (RAD)
Rapid Application Development or RAD is a software development process originally
proposed by James Martin (initially called Rapid Iterative Production Prototype or RIPP)
[69], RAD emphasizes a short development cycle. Such a target can be achieved by using
rapid prototyping, CASE tools and component-based construction approaches, the
parallel-iterative develop model, and time boxing (postponing the functionalities to the
next version in order to maintain the speed and the dead line) [28] [84].
Rapid prototypes are created for the purpose of jump-starting design and flushing out
user requirements. They are also used as the proof of concepts and as a talking point for
refining requirements. Unlike others, RAD prototypes are also used as the feature light
versions of the finish product that will be iteratively expanded to the actual final product.
To build the prototypes fast, RAD encourages the utilizations of CASE tools (virtual
49 IDE, and high-level or domain specific programming) and reusable components. RAD recommends small teams that consist of experienced versatile and motivated members who are able and willing to perform multiple roles [69],
Each of the RAD iteration consists of four life cycle stages: requirement, planning, modelling, construction and implement. The iterations can be performed in parallel to achieve high-speed development (see the Figure 3.3 below).
One adoption limitation of RAD is that the requirements of the project must be well understood. The system must be decomposable or can be properly modularized.
Moreover, RAD is not suitable for the systems that are life critical and/or require high- performance or reliability [85].
Rational Unified Process (RUP)
RUP is developed by Ivar Jacopson, Philippe Kruchten and others at the Rational
Software Corporation, currently a division of IBM. The rational unified process tries to prevent software project failures by tackling the problems that cause software project failures [86] by encompassing six industrial best practises: iterative development, requirement management, component-based architecture, visual modelling software, software quality verification, and controlling software changes [87],
RUP uses the iterative development model with four milestones or four phases: inception, elaboration, construction and transition [55], The phases are not simply a renaming of the classic phrases of the waterfall process model. However, each phase represents the mitigation of risks: business risks, technical risks, logistical risks and risks associated
50 roodaiinjj ^ Reqiureme Business Data | modatinfl | | modeling f J Proc* t> [ Busines Data i j Uodaiingl modelin | modeling f P? Applicaliofj | Procesi ganaralio | Data ' Modeling I Tasting li Planningi modelinb f I tufnovarl Application Deployment | Process generatiojn ^
'Modeling Testing ^ I turnover Application | generation |
Testing & turnover
60 - 90 days
Figure 3.3: RAD Process Model - the Parallel-Iterative Development Model
51 with the logistics of deploying. There can be one or more iterations in order to mitigate those risks. In each iteration, there are nine disciplines or workflows that the team may perform. The Figure 3.4 shows amount of work that is likely to be performed in each discipline or workflows of each iteration.
An essential precept of RUP is to identify and attack the risks that may jeopardize the success of the project. A vision document is created to capture the real needs of the customer (rather than just the requirements proposed by the customers). A business case is, then, created and used to make an accurate assessment of the return on the investment
(ROI) and the justification of the project. The project risks are identified and issues are established, assigned, and tracked to mitigate those risks [37],
RUP suggests the reduction of development risks by: developing software iteratively
(allows user and technical feedback), managing requirements (using Usecases [88][77]
[2]), using component-based architecture (using the object-oriented analysis and design
[61][42][89]), visually modelling the software (using the UML [6]) and controlling changes to software [55].
The Personal Software Process (PSP) and The Team Software Process (TSP)
PSP and TSP are developed largely by Watt Humphreys while he worked at the Software
Engineering Institute (SEI) [48]. The core methodology of these two processes is that developers (as in PSP [90]) and the development team (as in TSP [91]) estimates and plan their work before they start every job. After the job, they gather data that can be used to evaluate their performance and improve the method and their productivity. The data is also used it to make better future plans. The rationale behind the methodology is
52 Phases Workflows Inception Elaboration Construction Transition Business Modeling Requirements
Analysis & Design Implementation
Test Deployment Configuration & Change Management Project Management
Environment Iterations -* Initial Elabtt Elab#2 Const #1 Const #2 bonat #3 Tran #2 Tran #3
Business Technical Logistical Risks Risks Risks
Figure 3.4: Four Phases of the Rational Unified Process (RUP)
53 that engineers (developers) are rarely trained in sound development (e.g., proper design before coding), planning, and quality methods [48].
PSP, TSP and CMMI can be used together. PSP focuses on the improvement of personal skills, while TSP covers how to build effective team work and CMMI governs the organization and process. Both of them accelerate the adoption of the CMMI [92][93].
Scenario-based development (SBD)
Scenario-based development [58] is a software development process proposed by Mary
Rosson and John Carroll that focuses on usability engineering. The process makes intensive use of user interaction "scenarios" as software representations. A user- interaction scenario is a story about people and their activities. This description of people using technology is vital in discussing and analyzing how the technology reshapes their activities.
The result of this process is not a working software, but rather a set of usability specifications used for the development the user-interaction part of the actual software.
Therefore, SBD is not actually a software development process. What makes SBD worth noting is the method used to understand the actual requirements of the project. SBD suggests that a field study should be conducted in order to gain insight into the current practise at the actual work place. SBD argues that this insight cannot be gained by just asking the workers. Although the workers are the experts in the work they are doing, but some expertise is often held as tacit knowledge that they do not realize what they hold.
This tacit knowledge can be very valuable to the project [58].
54 There are many others traditional approaches that do not qualify as software development processes. Instead, these are software development methodologies. These methodologies do not provide a software-development framework, but focus on specific aspects such as the management, requirement, analysis, design and so on. Examples of these methodologies are HOOD [94], The Booch Method [95] and Aspect-Oriented
Programming (AOP) [96], to name just a few.
3.3 Agile Software Development
As mentioned, the movement called the "agile movement" by the Agile alliance marks the new era of software engineering. The alliance and their supporters criticize the traditional approaches for being heavy, inflexible, and unresponsive to change. The group proposed four essential values called the "Agile Manifesto" (the history about the construction and reasons behind it are described in detail in the Appendix A of the book
[18]) as follows:
Individual and interactions over processes and tools (1) Working software over comprehensive documentation (2) Customer collaboration over contract negotiation (3) Responding to change over following a plan (4)
The group also proposes a set of software development processes that honoured these values. These processes are called the "Agile Development Processes" (so in short "Agile processes"). These processes have many similarities and differences. Similar characteristics of Agile processes are discussed in the following section while the differences that make each Agile processes unique will be discussed in each individual summary of the subsequent section ("3.3.2 Agile Software Development Processes").
55 3.3.1 Common Characteristics of Agile Development
Based on the twelve principles of the agile approach (Section "2.4.5 Achieving Agility -
The Agile Approaches"), a number of Agile processes have been introduced. These approaches prefer to be minimalist. Most Agile processes have similar characteristics such as emergence, light, a close relationship with customer and adaptability. Practises commonly found in Agile processes follow.
Welcome Changes and Allow Feedback
To welcome changes and allow feedback, Agile processes suggest iterative development with small iteration and customer improvement.
Agile processes prefer iterative development [51] - short iterations. The iterative development allows feedback so that modification can be made and the direction of the development can be adjusted (the principle #2). Short iterations enable the development direction to be adjusted more often so that the resulting software will maximize customer satisfaction (the principle #1) [16].
Customer involvement is essential in Agile processes. Agile supporters (so called
"Agilists") believe that it is hard to describe software requirements accurately and that customer feedback is the best requirement to ensure that their needs are met. In addition, continuous contact with the customers enables the team to obtain the requirement changes as soon as possible [13].
Be Light
Agile processes avoid producing documentations to reduce modification efforts to the
56 minimum [18]. Agilists believe that, since documents are not the working software, they
have no value to the customer. As a result, it is a waste of time to produce and maintain
them [13]. Robert Martin stated in his book [97], "Produce no document unless its needs
is immediate and significant." Agile processes prefer to use code as the representation of
software (self-document) and verbal communication as the most effective way to
communicate [97],
Less Upfront
Agile processes avoid doing work for the future because they believe that changes will
occur and the effort for those works will be wasted. Any kind of upfront effort, therefore,
is put to a minimum.
Minimum Planning Agile processes avoid overhead planning in order to promote flexibility (the principle
#2). Planning implies that the plan must be followed, which promotes inflexibility.
Agilists believe that it is not possible to plan software development in advance.
Therefore, there is no need to do it as the direction of the development should be
constantly adjusted as development progresses [98].
Emergent Design Agilists believe that solutions to complex problems can only emerge from the interactions
between individuals (agents) that work closely together as a self-organized team rather
than the solutions obtained from sophisticated methodology (referred to as "processes" in
Agile Manifesto (1)), overall design (architecture) or tools. In other words, complex
solutions should emerge from simple rules [13]. (See [39] for more information on
emergent designs).
57 Emergent designs start with a simple design for a small part of a complex problem. These are later expanded step-by-step to cover the whole problem. The simplicity means to optimize development effort since, at every point of the development, the software does not contain extra elements that may not be used in the future (as the requirement may change). Since requirements can change at any time, the overall, or up-front design (as opposed to the emergent design), may be useless and the effort spent to produce it will be a waste. There are two rules indicating this preference of agile approach: (1) YAGNI or
"You Aren't Going to Need It" [97] and (2) "No BDUF" or "No Big Design Up-Front"
[66].
Less Management Control
In most Agile processes, the responsibilities of project mangers are limited as they become coaches rather than managers [68]. The development teams should have freedom as they can show off their creativity. In Agile processes such as Scrum [99], the main responsibility of the project manager is to monitor the development progress as the developers report back to him or her during a scrum daily meeting [51].
3.3.2 Agile Software Development Processes
At the time the agile manifesto was proposed and signed, there were seven recognized
Agile processes and methodologies: Extreme Programming (XP) [66], Scrum [99],
Dynamic Systems Development Method (DSDM) [51], Adaptive Software Development
(ASD) [13], Crystal [18], Feature-Driven Development (FDD) [13] and Pragmatic
Programming (PP) [51]. Later, the literature shows that there are more processes and methodologies that can be recognized to be agile including: the Rational Unified Process
58 (RUP) [19], Open Source Software Development (OSSD) [51], Agile Modeling (AM)
[44] and Lean Programming (LP) [100][101]. The following are short summaries of some
Agile processes.
Extreme Programming (XP)
XP is a very light-weight software development process that aims at small-sized development teams in an operation of ill-understood and/or rapidly-changing requirements [16]. Introduced in 1999 by Kent Beck and Ron Jeffries [66], [39], XP is the most recognizable Agile process because its controversial approaches to achieve agility putting its methodology to the extreme. Page 22 to 25 of [51] by Pekka
Abrahamsson and et al. offer a clear picture of the history of the XP practises.
XP has four core values: simplicity, communication, feedback and courage [66]. For simplicity, XP team will do the simplest job to solve today's problems and pay little or no attention to future problems (known as "You Are Not Gonna Need It" or YANGNI principle [52]). XP prefers direct communication (such as oral conversation) rather than formal communication because the former requires less effort and resources than the latter (e.g., creating documentations). In addition, XP uses iterative development models.
Thus, functionalities of the software product will be iteratively added. Elements that provides the most value to the customer will be the first to be implemented and delivered.
The idea is to deliver the product (even the part of it) as fast as possible to enable customer feedback. Finally, XP teams have the courage to refactor/rework their code when they discover that it does not meet requirements, or it is in-expandable or intelligible [52], To realize these values, XP suggests twelve practises. The Table 3.2 below provides brief descriptions of those practises.
59 Table 3.2: Twelve practises of eXtreme Programming (XP)
XP Practises Description Planning game The customer decides the scope (a set of functionalities) of releases (in form of stories) based on business values and development time estimation provided by programmers. Small release Apply evolutionary process model with the smaller release. Metaphor The shape of the system is defined by a metaphor or set of metaphors shared between the customer and programmers. Simple Design Use the simplest design that works. Tests The programmers write unit tests before the code and perform the tests at all time. The customer writes acceptance tests for the stories of each iteration. Refactoring The design of the system is evolved though transformations of the existing design that keep all the test running. Pair programming All code is written by two programmers at one computer/mouse/keyboard to allow continuous review (feedback). Continuous All code is integrated very often (no more than a few hours) and the integration built system must pass all tests. Collective The code is owned by everyone in the team. Therefore, any ownership programmer can improve any code anywhere in the system at anytime if they see the opportunity. On-site customer A customer sits with the team at full time. Coding standard All programmers in the team use the same style of coding. 40-hour a week Avoid working overtime by negotiating with the customer when the project is behind schedule.
60 Development Framework XP's development framework is composed of six phases: exploration, planning, iterations to release, productionizing, maintenance and death phase as shown in the figure 3.5.
The Exploration phase typically takes a few weeks for customers to provide requirements of the first release. Concurrently, the project team becomes familiar with the technology, tools and practises they will use on the project. In the Planning phase, the project team spends several days working with the customer to prioritize the capabilities needed for the first release. The developers estimate effort required and the team distributes the tasks. The Iterations to the Release Phase goes through several iterations to produce the first release. Each iteration takes one to four weeks and, at the end of each, the functional tests are executed. When the last iteration is completed, the software is ready for productionizing. In the Productionizing phase, the software is almost releasable. The team conducts additional performance testing and checking to ensure the release meets customer requirements. New changes may be introduced here and the decision must be made whether they should be included in the current release. In the Maintenance phase, the team produces new iterations of the software product to implement changes from new feature requests. If no new features are requested, the development will move to the
Death phase when work is finalized (e.g., manual or training).
Management XP does not offer comprehensive project manager guidelines [102]. Most of the management is performed by the customers and the developers. There is no overall planning. For instance, the customers write user stories (the requirements) and prioritize them. This prioritizing is the only form of planning in XP. Both iteration and release plan
61 EXPLO PLANNING ITERATIONS TO o z UJ RATION PHASE RELEASE PHASE N o PHASE CONTINUOUS Z X UJ REVIEW o I- (0 H 2 < < < O UJ X =J o a. a < STORIES o REGULAR FOR NEXT PAIR PROGRAMMING oc UPDATES ITERATION a.
PLANNING ANALYSIS DESIGN FOR TESTING TESTING 4 1STORIES CONTINUOUS FEEDBACK INTEGRATION Priorities Effort • estimates
TEST Collective SMALL j UPDATED FINAL 1 CODEBASE RELEASE! RELEASES RELEASE
CUSTOMER APPROVAL
Figure 3.5: Five Phases of eXtreme Programming (XP)
62 are based on the customer prioritizing. Scheduling, on the other hand, occurs based on estimates from the developers where user stories are signed up and estimated. Further more, XP strongly prefers small teams rather than large ones.
Development XP offers a number of concrete guidelines for development. Some examples are: (1) customers write and prioritize user stories as requirements, (2) metaphors are used to represent the architecture, (3) prototyping is encouraged, (4) emergent designed and test- driven development should be used, (5) pair programming is required for all coding,
(6) the customers write the acceptance tests and so on [39].
In XP, requirements are gathered in the form of user stories, which describe how the user plans to use the software. In the planning game, the customers, or users, then, prioritize the stories based on return values. The development will be performed based entirely on the priorities, It is, therefore, vital that the development will not depend on architectural dependencies between each story.
XP suggests emergent design and test-driven development. The coding starts with implementing a set of unit tests. Then, the actual coding begins with the simplest design possible in order to pass the unit tests. After the code has passed the test, the programmers look for ways to improve the design (called "smelling the code"). Next, the refactoring will be performed by removing duplication, improving communication, simplifying, and adding flexibility. The code will be tested every time it is refactored to ensure the functionalities and external behaviours [66][52],
XP also suggests avoiding modelling and documentation. In addition, metaphor and
63 modelling are done roughly to be used as a vehicle for coding. No model will be kept as documents. In fact, various documentations rather than code and verbal conversation should be avoided. Code is the only accepted documentation. In order to make code more readable, XP suggests it should be written in a standard style agreed or by every team member. This is called "code standard" practise, which is the standard that must be followed in order to keep the code readable [52].
Scrum
Scrum, named after an activity in rugby, was developed by Ken Schwaber and Jeff
Sutherland [103][4] (although some credit two Japanese developers, Takeuchi and
Nonaka [51 ][ 103]). Scrum focuses on management aspects rather than development aspects [102], Its principle is that software development is chaos and cannot be fully controlled. Accurate estimation can only be achieved with empirical data from the previous iteration (data from the same project and development team) [99].
Development Framework The scrum framework consists of three phases: pre-game, development and post-game phases respectively (sometimes, referred to as pre-sprint, sprint and post-sprint stages).
The target of the pre-game is to capture scope, requirements, vision (anticipated ROl, release and milestones) from the customers, and the product backlogs, which contain a set of the product features. Dependencies between each backlog will be found. The backlog drives team activities. The Figure 3.6 depict the development framework for
Scrum.
The development phase (sprint) takes approximately 30 days. Before each sprint, some of the product backlogs will be selected to be the release or sprint backlogs. The
64 > Product Sprint Backlog Backlog Daily Executable Increment Scrum Post-Sprint Demonstration 30-Day and Meeting Sprint
Sprint Standards, Techniques Planning Process, Guidelines, Meeting Development Tools
Figure 3.6: Scrum Process Framework
65 development will be performed in order to implement the backlogs. The daily scrum meeting will be conducted by the project manager. The target is to continually track of the progress of the scrum team. The meeting also serves as a planning meeting. Questions such as "What has been done since the last scrum?" and "What will be done next?" will be asked and answered. Moreover, the members can also ask the rest of the team for advice about problems they may face. The backlog will be stable during the sprint (no changes are allowed from outside the team) [104].
At the last day of the sprint, the result product (the working product increment) will be presented to the customer in an informal meeting. The customers and the team will then decide what to do next. New backlogs may be introduced to the product backlogs. Unless the product is accepted by the customers to be completed, the new set of sprint backlogs will be selected and the new sprint begins.
Management Ken Schwaber said that "Scrum relies on self-commitment, self-organization, and emergence rather than authoritarian measures" [13]. The manager, which is called
ScrumMaster, does not have authority to control the team members but, instead, the team manages itself. The manager's main responsibility is to track the progress of the development [51].
Development Scrum does not specify any development practise [102]. Both top-down design (as in the tradition approaches) or emergent design (as in XP) may be used. In Scrum, the design documents are not visible in the management level. This means that the team members are free to choose whether to produce the design documentation or not.
66 Dynamic Systems Development Method
The Dynamic Systems Development Method or DSDM was introduced in 1995 by the
DSDM Consortium [105][106] as the development standard of the Rapid Application
Development model or RAD model. The standard proposes a set of recommendations for
RAD projects [106]. The DSDM is more concerned with products than the activities that produce them. Thus, the standard specifies the products to be achieved, leaves the techniques to be used and ensure their notations are open and flexible. Nine Principles of
DSDM can found at [51] (Table 3.3).
Development Frameworks As a standardized RAD, DSDM uses prototype-based iterative models. DSDM consists of five phases: feasibility study, business study, functional model iteration, design and build iteration, and implementation (Figure 3.7 below).
A feasibility study is conducted to determine if the project is suitable for DSDM. The study is vital because DSDM, just like RAD, is not for every project. For example, the team and the customers must be ready; the requirements must be well understood; the customers must also accept time boxing. The technical risks are also be considered in the feasibility study [51].
In the business-study phases, the essential characteristics of business and technology are analyzed. High-level descriptions of the problems are presented (Models and diagrams are accepted) [51].
In the functional model iterations, rapid prototypes are created and inspected by the customer. Necessary changes may be made to ensure accurate functional models. Non-
67 Table 3.3: Nine Principles of the Dynamic Systems Development Method (DSDM)
I. Active user involvement is imperative. II. DSDM teams must be empowered to make decisions. III. The focus is on frequent delivery of products. IV. Fitness for business purpose is the essential criterion for acceptance of deliverable. V. Iterative and incremental development is necessary to converge on an accurate business solution. VI. All changes during development are reversible. VII. Requirements are base-lined at a high level. VIII. Testing is integrated throughout the life-cycle. IX. A collaborative and co-operative approach between all stake holders is essential.
68 Phase 1 Feasibility
Phase 2 Business Study
Agree Schedule Implement
Create Identify Rain Functional Function Model Functional; -« Review Implement Users Prototype Prototype
User approval / Review Prototype guidelines
Phase 3 * Phase 5
Identify Design Prototype
Review Agree Design and Design Schedule Build Prototype
Phase 4 Create Design Prototype
Figure 3.7: The Dynamic Systems Development Method (DSDM)
69 functional requirements are also listed and a risk analysis is performed. In the design and building of iterations, the functional prototype is further developed as a design prototype.
The customers then inspect the prototype. Next, necessary adjustments are performed. In the implementation phase, the design prototype is transformed into a full software product (e.g., customer training, the user manual, and other employment activities) [51].
Management From the DSDM principles, DSDM teams are empowered to make almost every decision.
Therefore, the manager has little control over the team. User involvement is a vital factor in DSDM. The users are required to review all the prototypes and provide feedback. As
RAD and DSDM also adopt a time-boxing approach where the functionalities are adjusted to fit the development time and resource [69][ 107]. Furthermore, the feasibility and business study enables the team to identify and manage risk as well as understand the actual need of the customers.
Development In terms of development, DSDM derives many characteristic from RAD. For example, the utilization of tools and rapid prototypes that will not be thrown away. Tools, such as configuration tools, are strongly recommended to allow all modifications to be reversible.
Testing is integrated throughout the development rather than a separate activity. In addition, the requirements are based at the high level, which allows some stability and flexibility. Additionally, frequent delivery is preferred because it allows rapid feedback, which leads to adjustment.
Adaptive Software Development
Adaptive Software Development, or ASD for short, is a derivative of "RADical Software
70 Development." Both were proposed by James A. Highsmith. ASD focuses mainly on the problems in developing complex, large systems. The method strongly encourages incremental, iterative development with constant prototyping. ASD is about "balancing on the edge of chaos". Its aim is to provide a framework with enough guidance to prevent projects from falling into chaos without interfering with emergence (the emerging of solutions from self-organized team) and creativity [13].
Development Framework As depicted in figure 3.8, ASD consists of three phase cycles: speculation, collaboration and learning. "Speculation" is a "plan" with some degree of acceptance for uncertainty.
"Collaboration" emphasizes the essential role of teamwork in high-change systems.
"Learning" stresses the need to acknowledge and react to mistakes and understand that requirements may change during development.
In the "speculate" phase, the project is initiated and the adaptive cycle planning is conducted. A set of release cycles (software increments) for the project will be defined based on user project initiation information, such as the customer mission statement and project constraints (such as delivery date).
In the "collaboration" phase, the team is motivated to work together in a way that promotes their abilities and creativity. There are some guidelines for "formal technical review" provided as follows: [14]; (1) criticize without animosity, (2) assist without resentment, (3) work as hard as or harder than they do, (4) have the skill set to contribute to the work at hand, and (5) communicate problems or concerns in a way that leads to effective action.
71 Adaptive Lifecycle Learning Loop
I \\2 11 Adaptive Concurrent Final Project Quality Cycle Features Q/A & Initiation Review Planning Development Release
Speculate Collaborate Learn
Figure 3.8: Process Framework of the Adaptive Software Development (ASD)
12 In the "learning" phase, the team learns about themselves and their work as development progresses. Learning will help improve their level of understanding. ASD teams learn in four ways: (1) focus group (the customer gives feedback on the software increments that are being delivered), (2) formal technical reviews (ASD teams review the software component that are being developed and learn from those reviews), (3) self-monitoring
(the team learns from monitoring their performances) and (4) project status (learning about the status of the project) [13].
ADS proposes very few practises. Example of those are: (1) iterative development,
(2) feature-based planning and (3) customer focus group reviews [51]. Some principles such as how the team should do in the formal technical reviews. [13] provides a general philosophy of building an adaptive organizational culture.
Crystal Family of Methodologies
The Crystal family of methodologies is proposed by A. Cockburn and J. Highsmith. As the name implies, Crystal is a set of methodologies that can be selected (within specific frameworks) for each individual projects (also called as a kit of methodology parts [18]).
There are four types of crystal methodologies named by colour: Crystal Clear, Crystal
Yellow, Crystal Orange and Crystal Red. However, only two of them are described here
(Crystal Clear and Crystal Orange).
The colour in the name indicates suitable projects in term of size and criticality (figure
3.9). The darker the colour, the heavier the methodology. There are four steps of project size that Crystal are suitable to: 6, 20, 40 and 80 peoples. There are also four levels of criticality: Comfort (C), Discretionary money(D), Essential money (E) and Life (L) [18].
73 L6 L20 L40 L80 < \ l O l i C E20 E40 E80
D6 D20 D40 D80
C6 C20 C40 C80
Clear Yellow Orange . Red J
Figure 3.9: The Crystal methodologies are named by colour
74 Crystal Clear is a light-weight methodology for small-size projects (for D6 project category), while Crystal Orange is for middle-size projects (for D40). Both methodologies share the same policy standards: (1) incremental regular delivery, (2) progress tracking by milestones based on software deliveries and major decisions (as opposed to written documents), (3) direct user involvement, (4) automated regression testing of functionality, (5) two user viewings per release and (6) tuning of the product and methodology at the beginning and in the middle of each increment.
More details of Crystal Methodologies can be found in [18], [51] and [13].
Feature-Driven Development
Feature-Driven Development (FDD) was developed by Jeff DeLuca and Peter Coad [13].
FDD is a process-oriented method for developing business critical systems. The FDD focuses on the design and building. FDD is one of the few approaches that was empirically designed for developing large systems. Additionally, one of the characteristics that makes FDD unique from other Agile process is that FDD is not as code-centric as other methods. Some modelling is required in FDD. Another unique characteristic of FDD is the code ownership in which the programmers will own the class or will be responsible for the class (this opposes the collective ownership found in most
Agile processes where the code belongs to the entire team). Details about FDD can be found at [108], [109] and [13].
FDD uses a five-phase iterative model. Figure 3.10 depicts the development framework of Feature-Driven Development (FDD).
75 Develop Build Plan * Design Build an a by by by Overall Features Feature Feature Feature Model List
Figure 3.10: Process Framework of the Feature-Driven Development (FDD)
76 Develop an Overall Model: the shape of the system is modelled. Too much detail analysis aims to be avoid. The target is to provide a big picture of the system.
Build a feature list: further modelling and customer inputs will be used to develop feature lists.
Plan by Feature: The feature set is used by a development planning team to formulate an iteration plan. This involves the organizing and prioritizing of the feature list to a release feature list.
Design by Feature: The chief programmers develop design packages based on the release feature list. The packages should consist of modelling components that will describe the objects to the point where transformation to code can be performed. The chief programmers manage the refinement of the packages by the entire development team.
Implement by Feature: The detailed design is implemented. Then the code is inspected, unit tested, and integrated.
Why FDD agile? FDD looks like a traditional iterative process. However, it does not have an explicit feedback loop like other agile frameworks. When asked, Jeff DeLuca replied that the customers (called the domain experts) get involved in two phases: phase 1 and phase 4. In phase 1, the domain experts participate in the construction of the overall model and, in phase 4, the domain experts provide more detail and review the design as it progresses.
77 Other Agile Processes and Methods
There are other less-known Agile processes such as Lean Programming (LP) [100][101]
(claims to derive from Lean and Just-In-Time Manufacturing [110]), Internet Speed
Development (ISD) [111] (claims to derive from Microsoft's "Synch-and-Stabilize"
technique [112]) and Open Source Software Development (OSSD) [51]. These processes share many similarities to the better-known processes, particularly XP (e.g., Code-
centric, iterative-development and so on). There are also other agile methodologies that
do not qualify as software processes, but that claim to support other Agile processes. The
best-known ones are Pragmatic Programming (PP) [51][113] and Agile Modelling (AM)
[44][ 114].
3.4 Problems of Agile Processes
The Agile alliance has pointed out the importance of process agility. They also offer solutions: the agile approaches and the Agile processes. Although most people agree on
the importance of process agility, not everyone thinks these solutions will really lead to actual agility [14]. The essential idea behind those solutions is to be light (less plan, design, documents), to avoid upfront work, and prefer a self-organized team. Some
researchers say that the approaches do not really work because they are too minimalistic
[21]. Others argue that the approaches are not useful because they work only in very specific cases (particularly small projects) [20]. This section presents some complaints and concerns about the agile approaches.
3.4.1 Minimalism
Agile processes suggest that the documentation should be created as little as possible
78 (See "Be Light" page 56). Overall design and overhead planning should also be avoided or performed at a minimum (See "Less Upfront" page 57). It is reasonable to assume that a minimum amount of documentations will reduce the amount of modification effort.
Since the effort for creating and maintaining documents does not translate to values for the customer, as the documentations are not the working software [97][18]. It is also apparent that planning for rapid-changing requirements is a waste of time [97]. Agile opponents, however, argue that documentation and planning are there for reasons.
Documentation
Documentation allows effective communication about the software [115]. Effective communication synchronizes the development team (See "Effective Communication" on page 19). Documents, which usually contain high-level human-friendly information about the software, are more readable than code. Software codes are usually very long and full with much low-level information [97], The low-level information is required for a computer to understand and function correctly. However, for humans trying to make sense of the code, those details can be overwhelming. Therefore, code is not a good communication media [17]. The Inadequate amount of documents will affect the communication among team members.
Project Visibility
Documentation does not only improve communication between developers. It also enables managers to gain insight into development so the project does not run off course.
In other words, documentation improves project visibility [116], Project visibility is a vital factor of successful project management (especially middle and large size projects)
[57] (See "Increase Visibility" on page 22).
79 Planning
There are also number of reasons for planning. A plan allows the manager to control the
direction of development (See "Controlling the Direction of Development" on page 21).
There are many types of planning. Activity planning, such as configuration-management
or quality assurance planning, enables the team to specify steps for those activities so that
the team can perform them accordingly. Furthermore, risk management, which is a form
of planning, allows the team to be aware of possible threats to the project. Without proper
planning, the team is unable to see the overall picture of the project and is unaware of
what they may face; as Gery Derbier wrote in [68] "Short iterations can make you short
sighted."
Design
Agile processes prefer emergent designs over overall designs ("Emergent Design" page
57). Agile opponents claim that emergent designs do not produce quality products as the
solutions are not the result from an analysis of the whole problem [55]. Looking at the
whole picture allows the organization a better understanding of the solutions which helps
create well-structured programs with fewer errors and increased maintainable [117].
Emergent design may also be redundant. As the development progress, the design often
needs to be adjusted to help integrate new solutions (from the new requirements). In some cases, the adjustment is difficult and much effort may have to be expended [118].
The effort for that adjustment can be eliminated by identifying dependencies between
each of the elements, developing an overall architecture derived from the analysis of the
whole problem [55] and enabling reason while managing change [119].
80 3.4.2 Lack of Adoptability
Adoptability
First, many agile processes are simply difficult to adopt [120]. Some do not offer enough guidelines for adoption [102]. For example, Scum provides management guidelines but not development guidelines. Likewise, FDD focuses on the development aspects.
Moreover, ASD barely provides detailed process descriptions, but does provide a philosophy of agile developments [51].
Limitations
Agile processes also have limitations. The best examples are the project size and complexity. Nearly all Agile processes are only suitable to be employed in small projects
[102][104], Scaling up the agile processes is difficult [20] because of the lack of adequate documentation and upfront effort [21].
XP, DSDM, FDD and Crystal Clear require the customer to be involved with the projects as they are required to provide constant feedback [39][13]. The constant feedback is essential because there are no persistent and clearly written requirements. This means that the processes are not suitable for the project that the customers are unwilling or unable to fully participate in such activities. Moreover, most agile processes can only be used in organizations where all team members work in the same physical space as verbal communication is required to compensate for the lack of documentation [18].
3.4.3 Lack of Practicality
High Interdependency
Many agile practises are highly dependent. The absence of one practise can easily result
81 in the failure of the project [66][21]. A straightforward example is "Code standard" practise in XP. Here, the team must agree on and strictly follow standard coding style.
Failure to do so will result in code that is not readable by others. This, in turn, disables effective refactoring (improving the design of the existing code). Refactoring is an essential mechanism for emergent design and emergent design is an important ingredient for agility because it allows the reduction of upfront design [39] ("Circle of Life or Circle of Snakes?" in [21] provides more details).
High Discipline
Since there is less management control, developers must consistently monitor and discipline themselves. This does not always happen in practise since "consistency in action is a human weakness" [18]. Roger Pressman writes [14] "If process models are to work, they must provide a realistic mechanism for encouraging the discipline that is necessary, or they must be characterized in a manner that shows tolerance for the people who do software engineering work." An example is "Coding Standard" practise in the eXtreme Programming (XP); here, the programmers must consistently use the same coding standard so that the code will be readable by all team members. While the rationale behind the practise make sense, the enforcement can be hard or impossible because programming styles are like accents in natural language - they are tough to change.
3.4.4 Failure to Work
There are examples of successful projects that use agile processes [118][121 ]. However, there are some reasons why agile opponents think that those successes do not prove that
82 the approach works.
Questionable Success
One of the first selling points for Agile processes is the success of a payroll project called
C3 ("Chrysler Comprehensive Compensation" [118]). C3 is a payroll software project (a very common type of software projects for business). eXtreme Programming (a popular
Agile process) was first tried on this project by the inventor himself, Kent Beck [16] and his team. The project has been highly advertised as an indicator that XP works [54][13].
However, after four years, the project was not completed and was finally terminated in
February, 2000 [122][21].
Other successful XP projects are small projects [104][15][20], which are what XP is designed for. However, success of small projects does not really indicate that XP works.
See "Other Approaches should Work Too" on page 84.
Success but Not Really Agile Development
DSDM and FDD are, sometimes, considered to be non-agile processes (even by the concept of Agile processes from the agile alliance). They both support the use of documentations and expenditure of some effort upfront. DSDM is considered by many to be non-agile. The full DSDM can be heavy [123][ 124], In the case of FDD, there are reports that FDD works in many projects [125] but, again, FDD looks more like an iterative traditional processes (i.e., the Rapid Application Development (RAD),
Cleanroom or the Rational Unified Processed (RUP)) with an exception that the customer is involved during the design [13]. In fact, DSDM was developed to be a standard process implementing the RAD model [106]. Hence, it derives a number of characteristics from
83 RAD. This basically indicates that "being light" and "less upfront" is not really a way to achieve agility the Agile alliance claims.
Other Approaches should Work Too
This claim comes from the fact that agile processes are only suitable for small projects.
There are success stories from small projects using agile processes [104][15][20]. The question is "Does a small project need a software development process at all?" Watt
Humphrey wrote if the team members are enthusiastically committed, any processes should work for small projects [17].
3.5 Conclusions
In this chapter, existing processes are presented. The processes can be categorizes into two groups: traditional and agile processes. Common characteristics of both groups were presented. Traditional process are formal, heavy, and plan driven, while Agile processes are light, avoid upfront work, and prefer self-organization team. A summary of each process is also provided. After that, criticisms against the Agile processes are discussed.
84 CHAPTER 4
SEARCHING FOR THE TRULY AGILE PROCESS
The Agile alliance stresses the importance of process agility. They also offer some solutions: the agile approaches and the Agile processes. Although most people agree on the importance of process agility, not everyone believes these solutions will lead to actual agility [14]. There are also arguments opposing those claims even with logical analysis and some evidences indicating that agile processes (as presented by the agile alliance) are a means to achieve process agility (as discussed in the previous chapter). These debates adequately show that agile processes are not effective enough and that there is room for improvement. In this chapter, the software process agility problem will be re-investigated in order to develop a set of recommendations or desirable characteristics for achieving true agility.
4.1 The Sources of Changes
The problems of Agile processes (as discussed in the previous chapter) show that the
Agile processes do not effectively achieve agility. A question, then, arises "What is the right way to deal with the problem of software-process agility?" Perhaps the only way to find out is to investigate the source of the problem - the sources of changes. In this section, the sources of software changes will be examined with the hope to gain more understanding about the problem and, perhaps, the solution.
85 o Business
H Q
Business & Customers © v needs Managers and >_ , , t . ( Technology organizational v Requirements personnels O High-level O solution or Architecture & Components Developers o ^ O o Testers
Figure 4.1: A Dependency Model of Software Development
86 Before investigating the sources of changes in software development, we must first define the basic assumptions of software development projects for the purpose of identifying dependencies among software components, participants and other factors.
First, there are three groups of participants in a software project: customers, developers and managers. The customers have the need for, use and/or benefit from the software.
The customers usually know very little about software development. The developers, on other hand, create the software. The developers, in most cases, learn about the details and the needs for the software from the customers. The developers also include testers who inspect the software to ensure quality. The last group of participants is managers.
Managers facilitate development. This group may include project managers, human- resource personnel, financial-support personnel, and so on.
Second, a simple software development model is defined (as depicted in the Figure 4.1).
In that figure, customers consider the business condition and see the need for the software product. They, then, define a set of requirements that they propose to the developers. The developers, then, use their skills and experience in software development to select software development technologies and techniques to analyze the requirements and identify high-level solutions and/or architecture for the software. Next, the developers implement the design and all the components needed to source codes. After that, the codes will be tested to see if everything runs effectively.
This simple model does not mean to represent the waterfall process model (as it may appear). Instead, it shows the activities of producing one product (artifacts) from other artifacts. In other words, the model shows participants, activities and theirs dependencies.
87 There are, generally, eight situations where change may occur:
4.1.1 Business-Condition Changes
Business conditions typically involve market competition and demand changes. For example, the software being produced may become less useful because the competitors release a new version of their software that shares similar or greater functionalities.
Business changes are out of the customers and developers' control since they do not have influence over the business condition. Nonetheless, some business changes are foreseeable. An experienced businessperson with sufficient and accurate data may be able to predict the direction of the market, trends of the new and existing demands, or even the movements of the competitors. Moreover, changes that are completely unforeseeable to one company are likely to be unforeseeable to the competitor as well.
4.1.2 Changes in Business Needs
Business needs are a set of needs required by the customers in order to be competitive in the market. Business needs usually depend on business conditions and the customers' business strategies. If these two factors are to change, business needs are likely to change.
As mentioned above, it is hard to control the business condition. On the other hand, the customer has full control over their business strategy. This means that it is possible for the customers to stabilize business aims or strategies. That, in turn, enables business needs to be stabilized to a certain extent.
For example, business conditions may be that the competitors' prices are lower. With the
88 aim to be the leading manufacturer, the strategy is to reduce the internal cost. One way to do this is to computerize the internal processes, such as the payroll process. Then, the need is to have the payroll system that is fully integrated to the human-resource, manufacturing and finance departments. This system will hopefully reduce costs.
From the above example, business needs can be partly stabilized (less changes) by the stabilization of the business aims and strategies.
4.1.3 Requirement Changes
Software development requirements are a set of descriptions about the software to be built. In order to write these descriptions, the writers (the customers or the analysts) should understand the needs for the software and how the software can satisfy those needs. This means that the writers need to understand the capabilities of computer software in general.
The requirements will need to be modified when: (1) the business needs have changed,
(2) the interpretation of business needs and the role of software (in satisfying the needs) have changed and (3) the already written requirements have been found to be incorrect.
Changes from the first factor - business needs - can be reduced by the stabilization of aims and strategies. This issue is discussed in the above section. When business changes occur, the requirements must be reviewed, which leads to necessary modifications.
Otherwise, the resulting product will no longer satisfy customer needs. The needs should be constantly monitored so that the requirements can be appropriately adjusted.
89 Changes from the second factor - the interpretation - can be avoided by thoughtfully analyzing the needs. In particular, what software can or cannot do to satisfy the needs.
Some of the main problems are: (1) the customers do not really understand what they want, (2) the customers do not really know how to specify the requirements and (3) they usually do not know what computer software can do for them [14], (4) the analysts usually do not clearly understand the business needs [88] and (5) the communication between the customers and the analysts is often inefficient.
Changes from the third factor - unclear requirements - can be avoided by carefully developing the requirement specifications. Properties of good requirements can be found in [30] and properties of bad requirements can be found in [8].
It is possible to stabilize requirements, although not completely. In fact, some agile processes, such as DSDM and FDD, suggest the stabilization of high-level requirements.
4.1.4 Technological Changes
Technological changes involve the changing of or changes in tools or techniques, or the reusable components. Some examples are changes in programming languages, library or application programming interface (API), the integrated development environment
(IDE), purchased component of the shelf (COTS), the database server, and so on.
Usually, technological changes can be avoided by simply ignoring the changes, e.g., to continue to use Java 1.3 instead of updating to 1.4 or 1.5. However, avoiding is not an option in some cases, for example, when these changes are parts of the requirements or the result of the analysis. For instance, the newly released version of the web server
90 "Apache" should be used instead of the old one because it has better (easier to employ, more stable and better performance) connectivity to the Java container "Tomcat."
If it is not possible to avoid technology changes (for whatever reason), modularity (e.g., high-cohesive and low-coupling design) in both architectural and detail designs, often, helps reduce the effect of such changes [14].
4.1.5 Management and Organizational Changes
Examples of management and organizational changes are planning or scheduling changes, resource or personnel changes, and process changes. Management changes also include the support changes (such as the financial support). There are many factors associated with these changes. Many of the factors may be external ones (in the process's perspective) which are hard to control, e.g., changes in budget or policy, or unexpected incidents of power outage.
Planning and scheduling changes may also be the result of incorrect estimations, which are hard, but possible, to avoid [48][90], When the estimation is not accurate (as many believe [126][104]), iterative development allows the schedules to be constantly adjusted.
The personnel changes may occur in the form of personnel replacement (someone quits and someone else replaces them) or personnel reinforcement (more people are added to the project) [1]. The book [48] by Watts Humphrey provides a number of techniques that may be used to avoid personnel changes. A number of techniques may help reduce the effects of these changes, such as improving communications by producing design documentation [115] or pair programming [97].
91 4.1.6 High-Level-Solution and Architectural Changes
The developers identify high-level solutions and construct the architecture to solve the problems (as stated in the requirements). As expected, these changes are affected by the requirement changes. They are also affected by other factors.
The high-level solutions will need to be changed if they do not or no longer solve the problems. The architecture will need to be modified if its realization is unmanageable, unfeasible or impractical. This may be caused by poor, outdated, ill-communicated requirements or an inexperienced architect. If the initial solutions are well and thoughtfully defined, it is possible that they can to be stable throughout the development.
High coherent and low-coupled design makes the architecture more manageable [127].
Using proven solutions (or design patterns) can reduce the risk of having infeasible or impractical solutions [63], In addition, flexibility and expandability allow for the stabilization of the architecture and high-level solutions [55][41].
In the case where the system architects are not familiar with the problem that the software aims to solve, techniques such as prototyping can help them gain a better understanding of the problem.
4.1.7 Implementation Changes
To implement software, coding is performed. This involves the realization of high-level solutions, architecture and design and low-level solution specification. Therefore, the code must be modified when those artifacts change. However, this is not a concern in this subsection. The implementation changes are the changes that occur from the software
92 implementation. The interpretation and the realization of high-level solutions may be initially incorrect and necessary to rework accordingly. In other words, the implementation changes occur when low-level solutions do not reflect the high-level solutions or do not actually solve the problems or, even worse, create more problems
(reliability, efficiency and so on).
Faulty implementations can be avoided by putting more effort into understanding the high-level solutions and in generating the low-level solutions (low-level designs or code).
Another significant way to ensure the conceptual ideas and knowledge about high-level solutions are well communicated.
4.1.8 Test-Result Changes
It is a normal part of development process that changes are needed in order to correct the code after tests have detected defects in the software; in other words, testing usually identifies defects and triggers other changes. This is actually a positive thing when defects are identified because they can be fixed early in development.
In the context of this subsection, test result changes refers to another kind of change - change from the initially undetected defects found after the test cases have been modified
(because the test cases are found to be inappropriate). To avoid these changes, the test cases (both for verification and validation) should be correctly defined the first time (or as close as possible). This is harder than it sounds but, at least, some techniques may be employed and a reasonable amount of effort may be expended in order to produce a set of effective test cases.
93 4.2 Dealing with Changes
This section discusses an attempt to find solutions to software process agility problem based in the examination of the source of changes.
When facing a problem, there are two general approaches to resolve the problem: proactive and reactive [128]. The proactive approach attempts to stop the problem before it occurs or reoccurs. The reactive approach solves the problem after it occurs. The Table
4.1 below provides an overview of dealing with changes using both approaches to the problem originated by the source of changes.
Ideally, both approaches should be used. The possibility of a problem will be low and if it actually happens, it will be properly and quickly responded to. However, there are some situations where the consequences of the problem are irresolvable (e.g., too late to resolve); thus, the proactive approach is the only one that attacks the problem. There are also situations where the problem cannot be attacked proactively. This is because the proactive approach requires a level of prediction that may not be possible. If the problem already occurs, the reaction approach may be used to fix the issue.
Based on the investigation of the sources of changes and the principles of problem solving, a set of solutions are defined in the following table as well as in the subsequence subsections. These solutions are also defined from the perspective of the software development process (less focus on the business or management perspective). Also, keep in mind that properties of software process in general must still be valid (i.e., develop software, ensure quality, increase odds of success, adoptability (See "2.2 Purposes of the
Software Development Process").
94 Table 4.1: Solutions for Software Process Agility
Sources of Changes BC BN Req Tech Management Solutions Implement Test BC BN Reg Reg HLS of of of of of Aims Specify Demand Support Int. Int. Schedule Personnel Construct Construct Int. ht. Construct ht. Competitor Technology
Prevention N/A N/A Upfront Bfort M Satisfied Upfront Bfort
M Elimination N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A O Reduction Monitor O&M Optimize O&M M&D Monitor O&M Optimize 3 Treatment Accept Change Stop Accept Change Ui Toleration React Quickly N/A TB React Quickly Redirection Ignore N/A N/A N/A Ignore
BC = Business condition BN = Business needs Reg = Requirement Int. = Interpret M = Motivate O&M = Optimize and Monitor M&D = Monitor and Document TB = Time boxing
95 4.2.1 Prevention
Preventing a problem is to keeping the problem from occurring or reoccurring. For example, the prevention of catching a cold is to clean hands often. The prevented problem may still occur, but the possibility of it occurring will be reduced.
From "the sources of changes," mistakes are one of the factors for changes in software development. Mistakes can be prevented by being thoughtful when creating any software artifacts. To get the high-level solutions and the architecture right, for instance, effort should be spent to understand the requirements and care should be taken to formulate the solution in order to fulfil the requirements. Similarly, to get the design right, the team should understand the architecture as well as the requirements before carefully constructing the designs. This concept can also be applied to business needs, requirements, implementations and test cases.
Attempting to make something correctly the first time is difficult; however, the best or reasonable effort may be spent beforehand. However, as there are other factors rather than mistakes (e.g., business condition changes), changes may still transpire. Spending too much effort on building something that might not be used can be a waste as changes may still happen (agile alliance pointed out this fact [13]). Therefore, the effort spent to prevent mistakes should be justified. As such, the team should consider the possibility of changes, the value and effect of the modification in order to find the reasonable point (or sweet spot). "Risk management" may be helpful in this situation [59],
Another change prevention is impeding personnel and support changes. To limit the
96 "personnel" changes, the organization must keep the team members motivated and satisfied. Many factors may be involved, for examples, pay, benefits, challenges [129], freedom and working environment to name a few (more information in [48]).
To prevent "support" changes, the team could keep the customer satisfied. This may
involve, for example, a good relationship, delivering the increments on time and returning the business values that customers enjoy.
In conclusion, the development team should perform risk management and spend upfront effort (e.g., for the architecture) based on the result of the risk management. The upfront effort should, at some level, prevent mistakes from occurring. Some efforts should also be spent to ensure that the communication between team members is properly performed.
To prevent personnel changes, members should be kept satisfied and/or motivated.
4.2.2 Elimination
Eliminating a problem is the solution that attempts to remove the source of the problem for once and for all, so that it will not turn into a problem later. This means that there will be no more problem because the source is eliminated.
It does not seem possible to eliminate any sources of changes in software development.
We cannot eliminate business demands, competitors, business aims or strategies or the requirements. We cannot eliminate the use of technology or the activity of defining software solutions, or producing code or test cases. Therefore, the elimination of changes in software is not realistic.
97 It is noteworthy, however, that Cleanroom development attempts to eliminate the need for unit testing using correctness verification but it still requires user-function testing (See
"Formal Method and Quality Assurance" on page 47).
4.2.3 Reduction
If a problem is unpreventable and its consequences unbearable, some steps may be taken in advance to reduce its negative effects. Consider the use of safety belts or airbags. We may not be able to perfectly prevent car accidents from happening or eliminate the need for transportation, but we can reduce the consequences of the accident with safety belt and airbags.
Continuous Monitoring Using techniques such as the Iterative Development Framework [33] and the Design for
Changes [41] help reduce the effect of changes. Iterative development frameworks that allows feedback can reduce the effect, since the team will acknowledge changes immediately and a reaction can be taken promptly.
Thorough Documentation
Some changes, such as personnel changes (e.g., a team member quit) cannot be prevented or eliminated. One noteworthy effect of these changes is that the team member takes the knowledge of the software when he or she leaves. Documentation allows such knowledge to be persistently recorded and helps to reduce the effect of such changes. However, personnel turnover is unpreventable or even unpredictable (although some are trying
[129]). Therefore, the team needs to document every part and every aspect of software so, when unexpected changes occur, the team will have enough information to respond.
98 Documentation also improves project visibility. The visibility enables effective progress tracking, which allows the project manager to monitor and recognize schedule changes easier [57].
Developing for Changes
The number of techniques found in the literature shows that software can be built in the ways that allow changes to happen at a certain level. If needed, relatively less modification (rework effort) may be required. When changes occur, in most cases, the design must be modified (reworked). The rework effort can be reduced if the design is made to be easily changed. This is called "design for changes" [41]. Abstraction, information hiding, modularity, high-cohesive and low-coupling designs are examples of techniques found in the literature [41][14], With those techniques, it is possible to prepare for change and the cost of reworking can be reduced.
Be Optimized
Being optimized means that there should be no unnecessary effort is expended. Consider the agile approaches on this issue; Agile processes spent little or no effort preparing for the future because it has no immediate value today.
4.2.4 Treatment
This solution simply means that modification should happen in response to change. A process should allow modifications. There should be, at least, an entry point for changes
(or means to recognize and manage changes if they occur). For example, customer feedback, market, progress tracking or technology monitoring and so on. Some examples of entry point of changes are customer feedbacks and progress monitoring.
99 Since modifications can result in even more changes, they should be performed carefully
and in a manageable manner. Configuration management or change control techniques
can be used [14].
Furthermore, embracing changes should not mean to encourage changes (as some agile
processes suggest such as XP). This because modifications mean rework and rework
means waste of effort.
4.2.5 Toleration
In some cases, changes may already occurs but, for some reason, modification of the
code to react to those changes is not an option. Toleration of a problem means that the
problem will continue to occur, but its effect will be reduced.
One way to reduce the effects of the problem after it happens is to react fast, effective,
and in a manageable way. Change management and configuration management enables such reaction so they must be employed.
Change management is a plan describing what the team would do when changes occur
[14]. Like other plans, it should provide concrete, yet flexible steps for managing change.
If the team members are aware and prepare for the change, they can respond quickly and effectively to those changes. The sub-section "4.3.3 Accepting and Managing Changes"
provides an example of change management mechanisms.
Configuration management is a mechanism for managing software sub systems [14].
When changes occur, modifications are made in response. The modifications create
100 different configurations. The management of these configurations allows the team to identify changes later on, to revert the changes, to mix and match different configurations, to isolate changes to be separately reviewed and many other operations.
One of the wildly-used techniques for configuration management is version control [130].
4.2.6 Redirection
When nothing else can be done to cope with a problem, it may not be a problem at all
[128], We can "just accept" certain problems. Some changes are not preventable and they cannot be eliminated. When they occur, there may be nothing the development team can do about it. For example, financial-support changes, business-demand changes or very late requirement changes are all unpredictable.
If there is no appropriate or economical way to respond to the changes, the process should have a mechanism in place to help choose whether to ignore the changes. Change control techniques [14] describes such a mechanism.
The team may also choose to postpone the changes for later releases rather than ignore them completely. This is called "Time Boxing" [28].
4.3 A Truly Agile Process
In the previous section, the root causes of the software process agility problem have been investigated and analyzed. The results indicate that there is something that we can do to tackle the agility problem. This sector discusses elements or characteristics that a truly agile process should have.
101 4.3.1 Risk-Based Upfront Effort
The development team should spend some effort upfront minimizing mistakes ("4.2.1
Prevention"). However, changes may still occur from other factors or simply because the team may not have adequate or accurate information. Therefore, the upfront effort must be spent in moderation so that the total effort can be optimized ("Be Optimized" on page
99). Risk associated with that particular activity can be used to find such moderation.
Risk-Based Planning
Planning provides a road map or guidance for development. Since changes commonly occur, the agile alliance suggests that planning should be minimized. However, in spite of change, there are many advantages to planning, for example, improving business concepts, keeping the development on track and increasing the odds of success [131].
Therefore, planning must be performed in moderation [14][13] and changes to the plan should also be allowed to occur. Risk analysis can also be used to find the sweet spot or the balance point.
Consider the example of "Travel Planning" as a metaphor for a risk-based upfront effort.
A travel plan to a foreign country (e.g., never visited, different language and culture) should be relatively well-detailed. Accommodation and the means of transportation should be specified and even reserved in advance. If there are other risks, such as plane tickets or immigration permissions that cannot be easily changed later, a detailed itinerary may be required. On the other hand, a travel plan to one's hometown or a neighbouring city does not need to be detailed. The traveller can easily find the way around his or her own hometown.
102 In addition, the "Treatment" solution (the Section 4.2.4) suggests that there should be a way to recognize and accept changes. This means that any plan should be subject to change as development progresses. Progress tracking and risk monitoring are examples of the entry points for altering a plan.
Risk-Based Requirement Gathering
The development team should ensure they identify the right requirements. The team should understand what the value of the product to the customer really is and how the requirements reflect that value. There are many advantages of this. First, the team may offer adjustments to the requirements that better reflect the values (as the team has better knowledge about computer software and software development) [77]. Second, the knowledge gained from those values should provide better understanding about the project's business risks (e.g., knowing the competitor of the customers). Third, discussing the business values and needs with the customers allows the customers to review their business needs and strategies.
Spending the time and effort to gather requirements should result in strong foundations for the project and ensure some level of stabilization. Processes such as the Rational
Unified Process (RUP), Cleanroom, the Dynamic Systems Development Method
(DSDM) and the Feature-Driven Development (FDD) encourage stability of high-level requirements. However, in some cases, low-level requirements may also need to be stabilized. Risk analysis should be performed to identify what part of the requirements should be stabilized.
103 Risk-Based Analysis, Design and Implementation
Overall analysis and design produce better quality with more optimal effort than emergent design [21] (See "Design" on page 80). However, if the requirements are not stable, effort spent on overall analysis and design may be wasteful [13]. Therefore, it is critical to identify what is worth doing while balancing the upfront with emergent efforts.
For example, low-risk-of-change high-feasibility-risk high-dependent requirements are worth spending time and effort to analyze and design.
4.3.2 Risk Management
Risk management enables a team to be aware of problems that they may face so that they can mitigate or prepare for it. Many researchers, such as Barry Beohm and Tom
DeMarco, agree that risk management is the essential key for agility [15][116]. Risk management generally includes the following activities: risk identification, risk analysis, risk mitigation and risk monitoring [59][14].
Risk Identification
There are many ways to identify risks. [59] discusses techniques for identifying risks.
Some of the techniques are: checklist, interview, meeting, routine input, survey and working group.
Risk Analysis
Risk analysis enables the team to manage uncertainty. Risk analysis may include: grouping, determining the risk drivers or sources, estimating the risk exposure, and identifying risk rankings.
104 Risk Mitigation
Risk mitigation is a set of activities performed to eliminate or reduce the effect of the risks. There are many existing strategies to mitigate risks (prevention, elimination or reduction solutions). For example, using iterative development mitigates the technical risk and integration risk [55]. Prototyping helps mitigate the risk associated with requirements. Moreover, persistent document reduces the effect of personnel changes.
Design for flexibility or extensibility reduces the effect of changes.
Risk Monitoring
During development, new risks may occur or the low exposure risk may turn to be more dangerous. Risks should be constantly monitored. In other words, in order to be agile, risk management is needed.
4.3.3 Accepting and Managing Changes
It is clear that software development processes should allow changes to be recognized and managed. In agile processes, developers are encouraged to change the code whenever they see fit. There are no concrete guidelines of how changes should be handled. In contracts, there is a traditional method for handling changes found in [14]. The simplified version of the method in the book is depicted in Figure 4.2 below. This change management techniques can be utilized to improve process agility.
Change Recognition and Entry Points of Changes
First, changes should be recognized or identified. There should be a mechanism to continually monitor and recognize changes. Customer feedback, reviews and testing are common entry points. Iterative development allows customer feedback, reviews and
105 Charge Recognition
Figure 4.2: Simplified Change-Control Process
106 testing to be continually performed. Since changes are considered to be a software development risk, performing risk management should help to identify possible changes.
Change Evaluation and the Decision to Modify
Changes are costly, especially later in the development [66]. Later in the development, there are already a number of finished components that must be reworked as well as potential new work. As a result, the cost of change must be evaluated so that appropriate decisions (whether the modification are worthwhile) can be made.
First, the developers should identify the necessary modifications for responding to the change as well as the consequences of that modification.
In order to identify the modifications needed and all of their possible consequences, the developers must understand the software as a whole. In small, recently written software, developers may be able to recall both concept and details from memory. For other cases, documentation can facilitate the activities as it helps the developers understand the software. If used appropriately, documentation can also show dependencies between components. These dependencies can be used to identify possible consequences of modifications.
Finally, the cost of changes and their benefits are compared to see if the modifications are worthwhile. If the modification is too costly but provides little value, it may be rejected.
In the case that the modification is costly but results in higher value to the customer, the team may decide to make the change. The deadline may be extended or some functionalities may be postponed or cancelled (in case the deadline cannot be changed).
107 Perform the Modification and Ensure the Expected Effects
The modification is performed and its effect are evaluated. In essence, changes are applied and a set of regression tests are performed to ensure expected consequences.
Integrate the Modification and the Management of Changes
The modification is, then, integrated into the system; this should be performed in a manageable manner. For example, changes may enter the development at the beginning of iterations. This allows development to be stabilized in one level. The resulting stability allows the team to be more focus. The "Synch-and-Stabilize" approach, used by
Microsoft in the successful production of Windows operating system, is evidence that some level of stabilization is important [112].
Configuration management techniques, such as architectural base lining, can greatly improve manageability. Other techniques, such as automatic integration (e.g., GNU Make
[132] and Apache Ant [133]), reversible integration, variation control and version control are strongly recommended by the Rational Unified Process (RUP) and the Dynamic
Systems Development Method (DSDM).
4.3.4 Effective Communication
The search for process agility in section "4.2 Dealing with Changes" shows that documentation is an important mechanism for being agile in three ways: (1) reduce mistakes by improving communication, (2) reduce the effect of personnel changes by persistently capturing information about the software, and (3) evaluating changes.
First, to improve communication, the documentation should be precise and easy to read.
108 It should also cover every aspects of the software (e.g., structure, behaviour, requirements, deployment and many more).
For the second and the third, documents should be easy to create and maintain because they will be used when unexpected changes occur. As such, the documents should capture as much information about the software as possible.
In practise (considering existing methodologies), documents are costly to create and maintain. They also require extra care. The Agile alliance pointed out that this extra care makes the process heavy and not agile. The group also suggested that oral documentation and code standard (as self-documentation) is the way to make the documentation agile.
Nonetheless, many claims presented in the section "3.4 Problems with Agile Processes" argue that since memory and oral documentation is not persistent, using code standard is not possible in a large team and code is still difficult to read even when standardized.
This conflicting (the demand to use less and more documentation at the same time) should be further studied. A documentation technique that is agile will undoubtedly improve process agility.
4.3.5 Customer Relationship
It is always the best idea to keep the customer satisfied. One way to do that is given them what they want (e.g., quality software with low price and delivered on time).
Agile processes suggest that the way to achieve customer satisfaction is by requiring close customer relationships and proactive customer involvement. The customers are
109 asked about the requirements not only at the beginning, but throughout development [13]
[39]. Some processes (such as XP, Crystal Clear, DSDM, FDD) need constant contact with the customer to maintain agility because using oral documentation rather than written documents allows the process to be light. Some process, such as XP, goes further by requiring that the acceptant tests are written by the customers. Customer involvement is beneficial to the development as far as the customers are willing and capable.
Therefore, a high-degree of customer involvement should be considered as an optional part of a true agile process, rather than an essential element.
4.3.6 Management Style
The solutions do not point to any specific management style. However, two characteristics are suggested. Firstly, the manager should monitor all possible changes.
The monitoring should include the progress of project to increase the project visibility - a vital element for a successful project management [57]. Moreover, management should create a working environment that keeps the team members motivated and satisfied in order to prevent personnel changes.
4.3.7 Summary and Additional Comments
The characteristics of a true agile process point to software developer processes with:
1) Iterative development framework that allows feedback,
2) The utilization of risk management,
3) The appropriate level of stability and upfront effort,
4) The management of changes,
110 5) Effective means of communication, and
6) Documenting software easily and thoroughly (or "Agile Documentation").
The rational unified process (RUP) and the Dynamic Systems Development Method
(DSDM) have most of these characteristics. Popularity and the large number of success stories [134][107] of these two processes are strong evidence demonstrating that the underlined characteristics are keys to achieving true agility. The only issue that needs further study is documentation -"Agile documentation."
4.4 The Unsolved Problem - Agile Documentation
In the previous section, a set of characteristics as well as the techniques and practises required for a process to be agile are defined and discussed. Some of these techniques are ready to be used or require small adjustments. The biggest unsolved issue (because there seems to be no appropriate technique or practise) is the need for a technique for producing and maintaining documentations with agility or the "Agile Documentation."
The Agile alliance points out that documentation makes a process heavy and slow to respond to change [18]. This is a legitimate statement since the code (working software) and the documents are independent entities that must be updated independently. The synchronization is critical, as many also point out, because inaccurate documentation provides incorrect information about the software [135]. In other words, extra efforts must be spent to synchronize the code and the documents. However, the solutions of process agility discussed in the previous section show that documentation is a vital ingredient for achieving process agility. Documentation improves communication,
111 reduces the effect of unexpected changes and improves progress visibility. It also provides useful information for the evaluation of changes and necessary modifications.
This conflicting demand was also recognized by other, such as Scott Ambler, who proposes a set of principles and practises for documentation in agile processes called
"Agile Modelling (AM)." The core idea here is that the team should be thoughtful about every model or document they make so that only necessary documents are created. The principles, such as "Understand your models," "Choose the Right Artifacts" and
"Modelling with Purposes," are example of AM's principles [44],
Another technique that may be used for agile documentation is using information in documents to create or generate working software. These techniques are collectively referred to as "Generative Programming" (GP) [136]. Examples of these techniques are model driven development (e.g., executable UML) [80], domain-specific languages or
DSL (e.g., 4GL[28], SQL [72], MatLab [137]), reuse (e.g., Components, Design Patterns
[63][64]) and others (e.g., AOP [96], IP [138], LOP [139] and CO2P2S [140]).
A quick look shows that the techniques mentioned above are inadequate or inappropriate to be Agile documentation. For example, Modelling with Purposes requires prediction or evaluation for the needs of each model [44], However, since documentation is most useful when unexpected changes occur (See "Thorough Documentation" on page 98), these unexpected changes are not easy to predict or evaluate. Therefore, software documentation should be done thoroughly rather than selectively.
Modelling techniques for the model-driven development (i.e., the UML) focus solely on
112 the business and development models. As a result, the models are only high-level
representations of the software, but do not cover developers' intentions. Domain-Specific
Languages (DSL) represent a developer's intentions but are limited to specific domains.
Design patterns capture intentions and can be used in any domain, but they are not automatic, they require manual updates [140]. Code templates can only be used to quickly generate code but still require manual maintenance. Aspect-Oriented
Programming (AOP), Intentional Programming (IP) [138] and Language-Oriented
Programming (LOP) [139] allow automatic code generation and the capturing of
intentions; they can be used in any domain, but cannot be used to manipulate business or development modelling (high-level representation). They are only capable of
manipulating target programming languages.
The next chapter will cover a more detailed review of those limitations. It also covers an attempt to develop a technique for Agile Documentation.
4.5 Conclusions
In this chapter, the sources of changes (the origin of the software process agility problem) are investigated in an attempt to solve the problem using basic problem solving
principles. The results are the required characteristics for a truly agile process.
Fortunately, most of these characteristics can be found in existing software development processes. The only characteristic that seems to have no satisfying realization is Agile documentation. To realize agile documentation, software process must provide techniques to create and maintain software documentation thoroughly and effectively.
113 CHAPTER 5
THE INTENTIONAL GENERATIVE PATTERN
MECHANISM (IGPM)
In the previous chapter, arguments against agile approaches show that the approaches are not the perfect means to achieve agility. The problem of process agility was, then, revisited and the characteristics of a true agile process were identified. A number of existing techniques can be used since they have those characteristics, except for the means of documentation - Agile Documentation. There seems to be no existing documentation techniques that can be used for agile documentation.
This chapter discusses the search for the appropriate realization for Agile Documentation.
It starts with a review of how documentation is used in software development. After that, desired characteristics of agile documentations are defined. Then, existing documentation methodologies will be evaluated against the desired characteristics. Finally, a documentation methodology called the "Intentional Generative Pattern Mechanism"
(IGPM) is proposed as a technique for Agile Documentation.
5.1 Agile Documentation
5.1.1 Roles and Types of Documentation in Software Development
In the context of software development, there are many different types of documentation for different usages but they can be categorized in to two very broad classifications:
114 requirement and design documents.
Requirement Documents
Since the input of every project is the requirements, the first type of documents to be created are requirement documents (also called "requirement specifications"). As the name implies, the requirement documents capture requirements. These documents will be used by the developers as the main source of information about the system being developed. In many situations, these documents are used as the business contract between the customers and the developers. In this case, the documents must be described precisely and will be referred to as "requirement specifications."
Design Documents
When developers develop software, they use their experience and creativity to try to fulfil the requirements. Sets of solutions (how they intend to solve the problem) are formulated.
Usually, the very first solutions are brief and abstract. The result of these solutions are the high-level structure or behaviour models of the software (the architecture or design) that hopefully solve the problem. These high-level structures or behaviours will later be refined until the solution is implemented and code is written based on the design.
Capturing Designs The developers may choose to create design documents to capture the high-level structure or behaviours of the design. The design documents can be compared to a map that is the simplified representation of the actual landscape. These documents may be distributed so the implementation (or the further refinement) can be performed by other developers.
Additional design documents may be created as the development progresses.
115 Capturing Intentions Designs are results or solutions that developers create. The design may or may not clearly
reflect the intentions of the developers. For example, a developer may initially intent to
use n as a parameter of a function to pass on a natural number (i.e., 1, 2, 3,...). He or she
also wants a means to pass on a null or un set value. Since the programming language
used (let say, Java) does not allow variables of primitive types (e.g., int, double, boolean)
to be set to null, the programmer decides to use the value -l as the way to tell that
function that the parameter is not set. What is recorded in the design document will be
just that a parameter n is int. Extra explanation may be required if the programmer wants
to record his or her intention. They may, for instance, add a short comment to the
function.
To summarize, there are generally two types of documents in software development:
requirement and design documents. Design documents capture high-level information of
the software and/or developer intentions.
5.1.2 Documentation for Agile Processes
According to the analysis of the sources of changes, documentation is necessary for:
(1) reducing mistakes, (2) reducing the effect of personnel changes and (3) is used in the
evaluation of changes.
In the first case, both types of documents are used. The requirement documentation
improves communications between the customers and the developers while the design documents improve communication between developers. That, in turn, reduces the
possibility for mistakes. This means that the documents must cover all aspects of the
116 software so they can be used to communicate everything about the software.
For the second case, the design documents are used. If the documents are persistently stored, the knowledge about the software will still be preserved in the organization after developers leave the team. This helps reduce the effect of personnel changes.
Finally, both types of documents are used in the evaluation of changes. The developer can use the information in the requirement and design documents to identify the modification and its effect. Without these documents, the developers have to recall the knowledge about the software from memory or from the code by performing reverse engineering from the code. This takes time and effort and may not fully recall all the necessary information.
In the second and the third purpose, documentation is used as a safety net when unexpected changes occurs. Since it is nearly impossible to predict unexpected changes, the team must document everything about the software. This is a significant problem since doing so results in a large number of documents, which require more time and effort to create and maintain as the Agile processes indicate.
5.1.3 Possible Solutions
One of the possible solutions is to find a way to automatically synchronize the document and the code. This is possible by using one thing to generate or represent another.
Use Code to Generate or Represent the Documents
Code contains too much information that is needed to be made sense of. Since making
117 sense of code is not an easy task for computers, synchronization by generating documents from code is not simple. A good example of these techniques is Pretty Printing [5], which displays code in a more readable way (using font, colour or shapes).
Another possible choice is to allow documentation to be embedded into the code so that synchronization can be done more easily (still a manual job). Although it is easier to update the documents, there are still many limitations, including the structural constraints of the code. Since document must be located near its described code, it must follow the structural constraints of the language. For example, documents of two public classes in
Java must be in two different files. It is impossible to document two classes in one place.
Examples of these documentation techniques are Java Doc [141] and Doxygen [142],
Use Documents to Generate or Represent Code
Another way to achieve automatic synchronization is to use information from the document to generate code. The core idea of these techniques is to create a mechanism to allow more information to be added to documents (high-level of abstraction but less detail). Extra information will be added into the document until there is enough information to generate code. If the appending of the extra information is done easily, transparently or automatically, the synchronization will be easy and inexpensive. This type of technique is collectively called "Generative Programming" (GP) [136].
The following section provides a review of existing generative programming techniques that may be used for agile documentation.
118 5.2 Candidates for Agile Documentation
Generative programming (GP) is a software-development technique that allows developers to develop software using high-level notations [136]. Since high-level notations can be used as documents, generative programming is a possible solution for agile documentation. The core idea of GP is to create a mechanism that allows more information to be added to the high-level notations. Here, extra information will be added into the document until there is enough information to generate code. If the addition of the extra information is done easily, transparently or automatically, the synchronization will be easy and inexpensive. GP techniques can be categorized into three categories. The following is a review of generative programming in each category.
5.2.1 Domain-Specific Languages (DSLs)
A domain-specific language (DSL) (as opposed to general-purpose languages or GPLs) is a high-level programming language designed to be used in a specific domain [136][65].
Extra information added to refine DSL (in order to have enough details to generate code) are assumed knowledges from its specific domain. For example, in SQL "select * from table/" means to query data of all fields from a database table named "tablel." All actions of actual querying are transparently specified.
There are many forms of DSLs, such as textural languages (SQL [72], HTML [143],
GNU Make [132] and Matlab [137]) and graphical languages (LabVIEW [144] and
Electronic Workbench [145]).
Because of the smaller-focused area, DSLs tend to be more reflective to the developers'
119 intentions than general-purpose language. Because of the smaller-focused area, DSLs are only useful within their own domain. They have little use outside their own domains. As a result, DSLs are not a preferred option to be used for agile documentation. Families of
DSLs may be needed in order to cover all aspects of software.
5.2.2 Model-Driven Developments (MDDs)
GP techniques in this group utilize modelling artifacts as documents. They do so by allowing the developers to specify more information into those modelling artifacts until there is enough information to generate code. A well known example of these techniques is the executable Unified Modelling Language (the executable UML) developed by the
Object Management Group (OMG) [80].
MDD can be a good choice for agile documentation because the modelling artifacts are initially designed for software documentation. However, there are some problems that makes MDD not the ideal choice.
Firstly, they are not intentionally reflective. This is because modelling artifacts are designed to represent business and development concepts. This problem may be solved by developing modelling artifacts for capturing intentions directly.
Another problem is that an MDD is usually composed of multiple modelling methods.
This is because modelling artifacts are usually represented graphically and graphical representations are not as expressive as the textual ones. The Number of possible graphical notations (e.g., rectangles, circles, ovals) are not comparable to the number of words that can be used in textural notations. To solve this problem, the model should
120 allow some concepts to be expressed by textural notations. For example, the executable
URL uses the Object Constraint Language (OCL) to better express model constraints.
The last problem involves multiple modelling methodologies (e.g., Class, State, Usecase,
Node diagrams). As graphical models are inexpressive, families of modelling methodologies are used so that they can cover every aspect of the software. The problem here is that it takes more time and effort to learn each methodology. Moreover, multiple modelling methodologies also require links between each methodologies so they can be used to represent the same software.
5.2.3 Language Manipulators
Language Manipulators, as the name implies, manipulate programming languages. They allow abstract concepts to be created and mapped to actions of manipulating code in target programming languages. Many have use these techniques to implement Design
Pattern [64] (as found in JBuilder and Eclipse, for example).
In essence, language manipulators allow developers to group related pieces of code and label them. The label acts a natural documentation of that piece of code. An entity of a label and the piece of code represented by the label are called by many names including
"macro" or "template" (in C++ [127]), "pattern" (in JBuilder and Eclipse), "generative patterns" (in CO2P2S [140]), "intention," "syntax" (in Intentional Programming (IP)[ 138] and Language-Oriented Programming (LOP) [139]) and "aspect" (in Aspect-Oriented
Programming (AOP)[96]).
A problem found with these techniques is that they target the manipulation of code.
121 Therefore, they cannot be used to manipulate higher level elements like designs or requirements. This problem may be solved by developing a language manipulator that can manipulate more abstract artifacts. Another solution is to use these techniques to manipulate design and requirement specifications.
5.2.4 Additional Comments
A number of different generative-programming (GP) techniques have been reviewed and evaluated. There is no single technique that is perfectly suitable to be used for Agile
Documentation. Some GP methods automatically add domain-specific predefined- information to the high-level notations. Hence, they can only be used within specific domains. Some GP techniques neglect the developers' intentions and focus only on the designs. Other GP methods capture the intentions and can be used to manipulate low- level artifacts (i.e., the code). Nonetheless, the concept of using high-level information to generate code is a promising way to inexpensively document software.
5.3 Realizing Agile Documentation
Considering the analysis in the previous section, none of the existing techniques seems satisfactory as a possible implementation for Agile Documentation. However, in those techniques, some interesting concepts are presented. With some adjustments, those concepts can be combined to develop a single methodology or mechanism that may become a possible realization for Agile Documentation. This section presents some of those concepts that inspire the development of the Intentional Generative Pattern
Mechanism - a proposed solution for Agile Documentation.
122 5.3.1 Natural Documentation
To allow documentation to occur with less effort, the document should not require extra effort to create or maintain. This means that the documentation should be an integrated part of the software. This way, the documentation is created with the code and changes with the code (See "Use Code to Generate or Represent the Documents" on page 117).
To understand how this can be done, consider the action of naming a variable or function with a meaningful name [135]. Calling a variable by what it represents naturally documents the variable. In the scope of this thesis, this will be called "natural documentation," as the name of the variable naturally documents the variable; similarly, naming a function or class with meaningful names.
Expanding the concept further, explicit documentation techniques such as comments can be replaced by first wrapping a piece of code into functions, which are named to document that piece of code - this is a part of the technique called Refactoring [146].
Consider the following code listing as an example.
1 Random Randomer = new Random(); 2 int[] Values = new int[100]; // 100 random number 3 II Creates random numbers that are smaller than 100 4 for(int i - 0; i < 100; i++) 5 Values[i] - Randomer.nextlnt(100); 6 7 int Sum = 0; 8 // Calculate the mean of the values 9 forfint i = 0; i < 100; i++) 10 Sum += i*Values[i]; 11 12 int Mean = Sum/100; 13 Print("Wean of random number: " + Mean);
As you can see, there are a few comments in the above code. The code can be re-written
(or be refactored) as:
123 1 function int[] newRandomints(int Count, int MaxValue) { 2 Random Randomer - new Random(); 3 int[] Ints = new int[Count]; 4 for(int i - 0; i < MaxValue; i++) 5 Ints(i] » Randomer.nextlnt(MaxValue); 6 return Ints; 7 } 8 function int calculateMean(int[J Ints) { 9 int Mean = 0; 10 for(int i » 0; i < Ints.length; i++) 11 Mean +« i*Ints(i]; 12 return Mean/lnts.length; 13 } 14 15 int ValueCount = 100; 16 int MaximumValue = 100; 17 int[] Ints = newRandomints(ValueCount, MaximumValue); 18 int Mean « caicuiateMean(Values); 19 Print("Mean of random number: " + Mean);
Now, the rearranged code needs no comment and still provides important information about what the execution may produce and where in the code those productions occur.1
It is crucial to note that the Creation Of the function newRandomints and caicuiateMean is not done to serve the original purpose of functions - to be reused; instead, the functions are created partly to allow those pieces of code to be named (natural documentation).
Additionally, the separation of work created by the functions allows changes to be applied to it in a more manageable way (i.e., "Developing for Changes" page 99).
The above technique has potential to be Agile Documentation, but it has a few serious limitations.
First, the technique will likely break the program into too many pieces. Just consider the above code, which is a very simple less-than-ten-line code, but requires two functions to document it. If these functions serve other purposes such as reuse, they would be worth it.
However, if the functions do not have other usages beside being a point for documentation, this technique will simply separate the program into too many functions.
1 This particular refactoring technique is called "Extract Method" [146].
124 The cluttering of code may seems harmless, but, for many programming languages, function calls can be expensive, especially with many parameters. In other words, this technique may result in an unexpected performance issue [146].
Second, the separation of work (that allows the piece of code to be named) must follow the syntactical rules of the language. This simply means that code can only be documented as much as the language allows. Although a number of features are available for this purpose (such as functions, classes, packages, closures, aspects and others), there are still parts of the technique that cannot be applied. For example, multiple class elements cannot be grouped together into one entity (which we can attach a name to).
Third, the technique cannot be used with multiple-language-programming frameworks or paradigms. Softwares that requires multiple programming languages are more common than one might think. For example, most Internet applications regularly require the developers to understand and handle at least five languages, e.g., PHP, HTML,
JavaScript, CSS and SQL. Code of these languages are put together by a set of rules.
Natural documentation (by separation and labelling) cannot go across languages.
Lastly, the technique can only document code or low-level designs and not other aspects of the software (such as the requirements) because the technique is done on code; therefore, you need pieces of code to be labelled.
The intentional generative pattern mechanism expands the natural documentation concept into a much greater level in order to overcome the limitations. Further elaborations will be discussed in later sub sections.
125 5.3.2 Code Generation
To be agile, response to changes must be quick and require little effort. Applying changes should occur in a manageable fashion. Code generation allows that. It also helps compensate the negative effects of cluttering code into too many pieces for natural documentation. Code generation also enables reuse in higher degrees than permitted by programming languages.
Since source code is essentially a sequence of characters (a text), changes in the source code are simple changes to a text. Fortunately, many text manipulation tools are already available. The best example is the common gateway interface (CGI) such as the Personal
Home Page (PHP). Some of those ideas can be used for Agile Documentation. Consider the following example:
1 public class Gateway { 2 private String!] Clients; 3 private String!] Servers; 4 5 public int getClientCount() { 6 return this.Clients.length; 7 } 8 public String getClient(int I) { 9 if{(1 < 0) || (I >= this.Clients.length)) 10 return null; 11 return this.Clientsfl]; 12 > 13 14 public int getServerCount() { 15 return this.Servers.length; 16 > 17 public String getServer(int I) { 18 iff(I < 0) || (I >= this.Servers.length)) 19 return null; 20 return this.Servers[I ]; 21 } 22
In the above code listing, the Gateway class contains a list of Client and Server names.
The programmer does not want the list to be modified by objects other than the Gateway so he makes both Clients and Servers private fields (line #2-3). In this programming
126 language (i.e., Java), array is mutable. It is not safe to return the array of either the Clients and Servers fields. Thus, the pair of methods getxxxcount() and getxxx(int) for Client and Server are required to allow access to the Clients and the Servers names (line
#5,#8,#14 and #17).
This "immutable-array access pattern" is very common, but there is no way to quickly apply the pattern without hand coding. Also, there is no way to naturally document them except to put them next to each other. Furthermore, there are a few variations of this pattern, for example, the method getxxx(int) may throw an ArrayoutofBoundException rather than returning nun when the index is out-of-bounds. However, changing an accessible array field to an immutable-array access and from variation to another cannot be done so quickly.
By using code generation technique, the pattern can be applied quickly; changing between variations can also be done with almost no effort. The following code provides an example.
1 function String addMutableArrayField(String VarName, String VarType) { 2 return "public " + VarType + "[] " + VarName + "s;" + "\n"; 3 } 4 function String addImmutableArrayField( String VarName, String VarType) { 5 return 6 "private " + VarType + "[] " + VarName + "s/" +"\n"+ 7 +"\n"+ 8 "public int get" + VarName + "Count() {" +"\n"+ 9 " return this." + VarName + "s.length;" +"\n"+ 10 "}" +"\n"+ 11 "public " + VarType + " get" + VarName + "(int I) {" +"\n"+ 12 " if((I < 0)||(I >= this." + VarName + "s.length)) return null;" +"\n"+ 13 return this." + VarName + "all];" +"\n"+ 14 "}" +"\n"; 15 } 16 Print( 17 "public class Gateway {" +"\n"+ 18 addlmmutableArrayFieldf"Client", "String") +"\n" + 19 addImmutableArrayField("Server", "String") +"\n"+ 20 "}" +"\n" 21
127 In the above code, changing from Immutable to mutable and back can be done by changing between the function addimmutabieArrayFieidf..and addMutabieArrayFieidf...) (line #18 and #19). This way, changes can be applied quickly, more manageable and with little effort. Also, the pattern is naturally documented as immutableArrayField.
Although, the code, is now naturally documented and changes (that involve array fields) can be applied with lesser effort, there are two problems with this method.
First, the code of this code generator is nowhere near as maintainable as the original code
(changes to it would require much more effort); in other words, it is not agile because the programming language that this code is written with is not designed specially for text manipulation. A programming language with a feature for easy text manipulation should be used for this purpose. The following code listing shows how code can become more maintainable if the language is specially designed for text manipulation.
i function String addMutableArrayField(String VarName, String VarType) <${ 2 public <(VarType)>(] <(VarName)>s; 3 }$> 4 5 function String addimmutabieArrayFieidfString VarName, String VarType) <${ 6 private
The second problem with this method is that the product code now become fragments of
128 text. The scattering makes it difficult to debug the final code. This problem can be overcome if the developers can track the origin of each part of the final code.
5.3.3 Graphical Representation
Graphical representation is a better communication medium than the textual counterpart when presenting especially for high-level information. It makes the informations easy to read because it puts the informations in a two-dimensional view. However, there is a limited number of expressions representable by graphic notations. This problem will have less effect if there are simply less expressions to be represented graphically. Therefore, the realization of Agile Documentation should require only a few expressions and all of them should be representable graphically.
5.3.4 Summary and Additional Comments
This section discusses a few concepts taken form existing techniques that can be used for
Agile Documentation. Natural documentation allows documentation to be done with less effort as it is a part of created artifacts. Code generation allows reuse in a higher degree because fragmented code can be reused. Lastly, graphical representation should also be utilized since pictures are often a better communication medium.
These concepts inspire the development of the Intentional Generative Pattern Mechanism
(IGPM) as a realization for Agile Documentation.
5.4 The Intentional Generative Pattern Mechanism (IGPM)
This research aims to solve the software development process agility problem. After reviewing a number of existing techniques looking for possible solutions. Most of sub-
129 problems have techniques that can be used as possible solutions (in certain degree). The only issue that does not have an appropriate realization is Agile documentation or a mechanism or technique to create and maintain thorough documentation with less effort while doing it in a manageable fashion. The Intentional Generative Pattern Mechanism
(IGPM) is developed as a potential candidate of Agile documentation. This Section describes the concepts and advantages of the mechanism and provide a simple example.
It is intentionaly short for conciseness.
5.4.1 The Core Concepts
The Intentional Generative Pattern Mechanism (IGPM) is a proposed solution for Agile documentation. It makes use of a unit called a "Pattern" to capture developer intentions.
The intentions will be used to generate the program code.
Patterns for Intention Capturing
The "pattern" has a very similar meaning to the term "design pattern" in the sense that it captures the intentions of the developers. A pattern encapsulates actions of manipulating software artifacts such as "creating a class," "adding method to the class," "assigning value 5 to the variable n." The pattern's name represents the intention of those actions and naturally documents them.
A pattern should capture one intention. Many patterns may be connected together to create a pattern network, which represents more complex intentions. This network of labelling acts as documentation of the software. The developers will be able to track the intention for creating or configuring an artifact.
130 In IGPM, an artifact is also represented by a pattern. This means that everything in IGPM is represented by patterns, except for primitive values like integers or strings (the code).
Representing an artifact can be seen as an intention to have that artifact. For example, a pattern that represents a Java class captures the actions of creating a piece of code for a
Java class.
Since patterns are abstract entities, they can be used to represent everything. This means that the artifacts may include everything from software development artifacts, business modelling artifacts, requirements, test cases, reasons (textural explanation), qualities
(e.g., reliability, usability) and many others. This is important since an agile- documentation realization should cover every aspect of software and software development.
In IGPM, every aspect of software development can be represented within a single framework - a pattern network. Some patterns may represent high-level intentions or artifacts such as the requirements, stakeholders, sub-systems and so on. These patterns manipulate lower-level patterns. Lower-level patterns manipulate even lower-level patterns. Finally, some patterns manipulate the code directly. In other words, the whole software can be represented by a network of patterns.
The Generative Mechanism
Patterns are executable units. When executed, a pattern performs the actions it contains.
In other words, a pattern network is a program to develop software. Since software can be fully represented by a network of patterns, the software code can be generated by executing the pattern network.
131 [P2| PI
II 01
13 TEXTl h-T1
project Projectl; pattern PI { in port II :P2; out port 01 :T«xt; in port 12 :int; out port 02 :?«xt; P2 aP2 • m in port 13 :T«xt; out port 03 :P5; P3 aP3 :» new(); function render():void { PI aPl :« n«w(); 11.11 :» 12; aPl.Il aP2; 11.12 :« 13; aPl.12 :• aP3.o2; P4 aP4 :« n«w(); aPl.13 :• "Text"; aP4.i2 11.13; 01 :» "String"; 02 :« II.ol; 03 :• aP4.o3; ! )
Figure 5.1: Patterns
132 As executable (and programmable) units, patterns share many similarities with objects in the object-oriented programming (OOP). In fact, the IGPM adopts many concepts from
OOP such as types and instances, inheritance and polymorphisms. This means that patterns can also be reused. Re-usability is a vital quality because it reduces long-term development costs.
When modifications to any pattern are made, the software can be regenerated simply by re-executing the network. Code generation and re-usability reduce costs of modification and synchronization between documents and code. That, in turn, improves project agility.
Graphical Representation
The IGPM is designed to represent graphical notations. This can be done because there are only two operations when creating pattern networks. One is assignment and the other is code template. Fortunately, these two operations can be represented graphically.
Examples of the representation can be seen in the Figure 5.1 and the Figure 5.2 below
(the programming language of the code appeared in the Figure 5.1 will be discussed in
"Chapter 6.3 The Pattern Programming Language ").
5.4.2 A Pattern
A pattern is a programmable unit similar to an object in object-oriented programming
(OOP). Each pattern encapsulates data and computation, The class-object concept as well as inheritance and polymorphisms found in OOP also exist in IGPM.
A pattern has a set of inputs and outputs, and functions. The inputs and outputs can only be primitive-typed objects (strings, numbers or Booleans) and other patterns. Inputs and
133 Requirements
Code
Figure 5.2: A pattern network allows full traceability
134 outputs enable patterns to be configured so that they can be reused in similar contexts.
The inputs and outputs may contains single or multiple values. Multiple-value inputs or outputs are similar to array or list. They may be indexed with numbers, strings or patterns.
A pattern may have a render function. A render function contains actions for manipulating its values. This means that patterns can do two things: (1) manipulate primitive values and (2) manipulate other patterns (such as initiate, execute, set inputs or read outputs). Patterns may also access functions in Java in order to manipulate files, folders or databases (for the purpose of generative software code).
When a pattern is initialized, the creator of the pattern (which is another pattern) will be recorded to allow tracking. This also happens when a pattern input is set (the setter pattern will be recorded) and when strings are manipulated. The recording mechanism enables tracking through a pattern network (the Figure 5.2).
5.4.3 Advantages of the IGPM
Natural Documentation
An artifact name, in which its meaning represents the artifact concept, is considered a natural documentation of that artifact. This is called "natural documentation" because the name documents the artifacts without any additional information. Therefore, natural documentation is a documentation technique that requires little effort to create and maintain. Function, class and variable names are common natural documentations.
Unfortunately, the opportunity to do natural documentation is limited by the modular
135 structure of the development paradigm or the programming language. To be specific, the only place to naturally document an artifact is within the artifact name. This means it is limited to one place per artifact for natural documentation, which is not enough in most cases. Therefore, we cannot naturally document parts of artifacts that we cannot name.
Another problem is that parts of multiple artifacts may be involved in a single concept, but most paradigms or languages do not allow us to name them. This problem is also known as "cross-cut concern" in Aspect-Oriented Programming (AOP) [96].
The IGPM eliminates these problems. A pattern contains a set of developmental actions that do not respect the modular structure of the target programming language.
For example, creating a singleton class usually involves [63]: 1) hiding its constructor
(setting it to private or protected, in Java), 2) creating a hidden class-scoped field of the same type of the class and 3) providing a class-scope public method that will create the object and assign it to the field if this is the first time it is called (checked by the value of the field) or return the field as the only created object if it is not. In this example, we can create a pattern that is responsible for making a singleton class. The pattern should be named as'makeClass_a_Singleton .
Document Every Aspect of Software
Because patterns are abstract entities, they can be used to represent everything. They can represent software structures or behaviours, high-level or low-level artifacts, business or development models and many others. They can also represent developmental actions
(such as creating a file, e.g., a source code file) and other intangible entities such as qualities (e.g., "performance") or disciplines (e.g., "extensibility"). For example,
136 "performance" pattern manipulates some artifacts to increase their performance.
Reason and Dependency Tracking
Since every artifact is created and modified by patterns it can be said that everything in the software happens because of patterns. When a pattern or a string is manipulated, the manipulator pattern is recorded. The recorded pattern will be used for intentional tracking through a pattern network. The developers will be able to track every single piece of code
(a string in IGPM) up to the pattern that created or manipulated that piece of code.
Ultimately, every piece of code can be tracked to the associated requirements.
Automatic Code Generation
Since patterns are executable units representing sets of actions for software development, the software is developed when the patterns are executed. Any changes to the pattern network will be reflected in the resulting software whenever the network is executed.
5.4.4 An Example Pattern
A pattern named ReadoniyFieid in the Figure 5.3 represents the intention of making a field of a class to become read only. This pattern gets a "Class" pattern and a "Field" pattern as inputs. It attaches the field into the class and makes the field read-only by hiding the field (set it to "protected") and creating a "Method" pattern that returns the field value. The pattern "Class," "Field" and "Method" represent the intentions for creating class, field and method in Java, respectively. The pattern ReadoniyFieid is an example of intentional elements that are not usually represented in existing modelling techniques because it is not a software artifact. Rather, it is a feature of an artifact.
137 ri«id • Class Fi«ld \
ltethod
public raturn
Figure 5.3: The "ReadOnlyField" pattern
138 5.4.5 The IGPM for Agile Documentation
Since patterns can naturally document an entire software and can be used to automatically generate software code, they enable software documentation to be done easily and thoroughly, which makes the IGPM a potential candidate for Agile
Documentation.
5.5 Conclusions
This chapter provides a discussion of a search for an appropriate realization for agile documentation. A number of software documentation techniques are reviewed and re evaluated. The reviews show that there is no existing techniques suitable for Agile documentation. Then, a documentation method called the "Intentional Generative Pattern
Mechanism" (IGPM) is proposed for use in agile documentation.
139 CHAPTER 6
IMPLEMENTATION
THE PATTERN PROGRAMMING PARADIGM
In this chapter, the implementation of the Intentional Generative Pattern Mechanism
(IGPM) will be discussed, starting with the need for a new programming paradigm that is required for implementing the IGPM. A number of characteristics unique to the paradigm are provided. The second section describes brief details of how the pattern programming language (PPL) is developed (as a realization of the pattern programming paradigm) using the Curry Virtual Machine and the Regular Parser. Finally, the pattern programming language is introduced. Some examples of how the language can be used as
Agile Documentation will also be demonstrated.
6.1 Pattern Programming Paradigm
With the IGPM, developers program a pattern network to generate the final software code. The kind of programming necessary for creating pattern networks is different from other types of programming. This is because the product of the pattern networks are static artifacts rather than run-time computational results. Some of the unique nature of pattern- networks programming requires a different programming paradigm and programming language. This section discusses the pattern-programming paradigm and its necessity.
140 6.1.1 Artifact Properties
In the IGPM, a pattern network (a kind of program) will be written and used to generate another program code (the final software's source code). Program code is an artifact. Like other artifacts, program code has a set of properties (e.g., variable names, function signatures). These properties must be agreed on by all developers involved in the development.
To better understand this, consider a group of artists designing a coffee mug. The artists must agree upon how big the mug will be (the height and radius). They also have to agree on the picture and colour of the mug. These properties are static; being static, in this context, means that there will be only one value for each of these properties to be accepted and that the values will not be changed after they are accepted. If it is said to be
10cm tall, for an instance, it cannot also be said to be 8cm tall. In the same way, a variable name, for example, must be agreed and accepted by ever developer.
From the nature of the artifact-property designing, a variable representing such property must only be in two states: before and after the decision about it is made. The two-state nature differs greatly from freely-replaceable nature of regular variables (found in the imperative programming paradigm) where a value can be assigned and reassigned to a variable over and over as execution progresses. Therefore, the imperative programming paradigm is not suitable for pattern programming.
The declarative programming paradigm, a programming paradigm used for creating specifications (e,g., HTML), is also not suitable. The declaration of those specifications
141 are location dependent. For instance, an attribute of a table must be at the table tag. In the
IGPM, a pattern that represents an artifact may be passed around and its properties may be assigned by many different intentions, meaning it will be written in many different part of the code. Therefore, the declarative paradigm is not a suitable programming paradigm for the IGPM.
Pattern programming language must provide a natural way to specify the artifact properties. The properties may be specified, configured, and verified. Once the final value is determined, it cannot be replaced.
6.1.2 Delayed Computation
Actions for specifying, configuring and verifying a property's final value may be assigned to the port (which holds a pattern property value) by different patterns at different times and from the different locations in the code. These actions will be executed all at once when the port is asked to determine its final value. This process is called "rendering."
This delayed computation can be implemented using number of existing techniques, such as, instant class (like in Java), closure, delegates and so on. The pattern programming paradigm should have a way for the delayed computation to be created and assigned to ports with reasonable ease.
6.1.3 Computation Priority
Since multiple actions may be assigned to a port, but only one final value will be used, there must be a way to determined priority of the computation of these actions. The priority determination must be logical. The final value must be determined before it can
142 be asserted, configured or used. If the port is a collection, sorting it should have lower priority than appending values into the collection.
6.1.4 Conflict Detection
With multiple actions assigned to a port, there will likely be conflicts between the values these actions specify (consider the "coffee mug's height" example above). The pattern paradigm must have a mechanism to systematically detect and report such conflict with patterns responsible for the conflict so that the conflict can be resolved.
6.1.5 Intention Tracking
The goal of the IGPM is to implement Agile documentation, which is to document the software code. At the low level, pieces of code are to be documented while, at the high level, designs, specifications or requirements are to be documented. This means that the paradigm must provide a way to check which pattern is responsible for the final value of a port or a part of a text generated by another pattern. This way, developers can track from the code upward to the higher-level patterns.
There seems to be no existing programming language that allows this. However, this mechanism is very important for the IGPM to realize Agile documentation and the pattern programming paradigm must have this feature.
6.1.6 Text Manipulation
Code generation is an essential part of the IGPM. It allows changes to be applied to source code faster and in a more manageable way. It compromises and reduces the effect
143 of the code partitioning necessary to allow natural documentation.
Code generation involves heavily on string or text manipulation. There are a few features involving text manipulation that pattern programming paradigms should possess.
Text Composition
Many programming languages (e.g., Perl [147], PHP [148], Bash [149]) designed to target intensive text manipulations, especially those used for the Common Gateway
Interface (CGI), have quite an interesting way of manipulating text by using template-like string literals. The template can be composed with static texts or variable values, for example,'s = " $var is the same with to's = » » + var + "
In PHP, and in a few other languages designed for CGI, there is also a way to capture and compose text from a standard output. For example:
1 Here is the header. 1 Here is the header. 2 —^ 2 Here is the body. 3 Here is the footter. 3 Here is the footter.
The string "Here is the body." will be echoed out but captured and included as a part of the string (together with "Here is the header." and "Here is the footer.").
The above techniques should be employed by the pattern programming paradigm.
Long-String Literal
String literals in the IGPM can be very long and short. For the low-level patterns, string literals tend to be long as they may be used to represent a multiple-line code template.
For the high-level patterns, on the other hands, string literals tend to be short as they are
144 only used to configuration values for other patterns. The pattern programming language
should provide ways to define both short and long-string literals. The different between
both kinds of literals must also be explicit to prevent the unclosed-string-literal problem.
Delimiter Collision
Delimiter collision is a lexical conflict when there is an attempt to include a string
delimiter such as the string boundary token (e,g., double quote character) inside a string
literal. Since the IGPM will be used to create source codes in other programming
languages, it is very likely that string delimiters of the target language will be inserted in
string literals when programming patterns. Moreover, the IGPM may be used to create
code in any language so there is no way to know what string delimiters might be and how
to properly avoid the collision. Therefore, pattern programming language must provide
different ways to create string literals and avoid collisions.
6.1.7 Additional Comments
The IGPM aims to implement Agile Documentation, something no other implementation
exists before it. A number of unique obstructions must be overcome. The pattern
programming paradigm provide a rough picture of the characteristics of the programming
language for the IGPM.
6.2 Implementation of the Pattern Programming Language
This section provides an inside look into the implementation of the pattern programming
language for the IGPM.
145 6.2.1 Curry Virtual Machine (CurryVM)
Curry Virtual Machine is a customizable virtual machine written in Java. CurryVM provides the possibility to implement pattern programming language for the IGPM.
CurryVM can be customized in many aspects. The following are some of the features of
CurryVM and how they are used in regards to the IGPM.
Instructions and Expressions
CurryVM is an intermediate-code virtual machine that executes the non-machine compiled code (compiled code that is not the computer machine code) similar to JavaVM which executes Java byte-code.
Curry's intermediate code is a serialization of Curry Expression objects. Curry
Expressions are the smallest executable unit in Curry. An expression represents a single operation the Curry virtual machine will do (by holding a reference to an instruction to be executed). An expression may also hold parameters and sub expressions needed for execution. This structural nature of the curry expression enables post-compilation processing. The pattern programming language exploits this, in order to implement delayed computation (see "6.1.2 Delayed Computation").
When executed, Curry will ask the expression for the instruction number and look it up in the instruction set. Then, the expression will be passed to the instruction to execute.
Curry has a set of built-in instructions, but the developers may add more instructions by implementing and registering them via the Engine Extension. This process does not be discussed in this thesis as it is outside its scope.
146 Context
Curry executional context encompasses a set of information (such as variables, stackowner and locations) necessary for execution. This information is considered to be internal data so the Context is designed in such a way that it will not be accessible outside the scope of the Engine. Contexts can be nested to represent executional stacks.
Curry contexts hold the variables together with their names so the variables are accessible by name at run-time. This feature enables the more advance feature "Frozen Variable," which is an ability to save variable values with an executable unit created at runtime. See the topic "Executables" on page 149 below.
Type System
The type system in CurryVM is highly customizable. If needed, developers can create a type kind to specify how particular types behave. Type behaviours such as type checking, default value, elements (attributes and operations), instantiation and many others may be customized. CurryVM have eight built-in type kinds implemented by this customizable type system: (1) Java types, (2) array types (fully compatible with Java array), (3) variant types, type types (provide access to a type as a regular object), (4) package types (provide access to a package as a regular object), (5) DataHolder types (data-reference object), (6) executable types (executable objects), (7) enumerator types and (8) object types.
CurryVM also allows type loading and referring to be customized. Developers may specify how a Typespec (which provides the blueprint of a type) can be retrieved.
Typespecs of some types (such as Java types, type types, package types) are created on the fly as their behaviours are extracted from an other entity (Java class, Type, and
147 Package, respectively). Some Typespecs must be retrieved from storage (called "Unit")
using type name and namespace (called "Package"). Some types has no named, so their
Typespecs are dynamically created at runtime or embedded in the code. Finally, some
types are generic variations of another type (e,g., the Typespec of List
List).
Pattern-type kind is a variation of Object-type kind. See the topic "Patterns" on page 164
for more detail.
Code Location
All executable units including the code expressions, may hold location information - the
code name and the line number. Regularly, this information is used for debugging and
error reporting. For the IGPM, code location tracking is also used for intention tracking
and conflict detection reporting.
The location information can be retrieved at runtime (from the Context). It can also be
retrieved in the form of ActionRecord, which also includes stackowner information (see
the topic "StackOwners" page 150).
DataBolderS
A DataHoider is the CurryVM implementation for variables. A DataHoider simply hold a
data value for later retrieval. Nine operations can be performed to the DataHoider: (1) get
value, (2) set value, (3) get type, (4) check if the DataHoider is readable, (5) check if the
DataHoider is writable, (6) check if the assignment to the DataHoider require type
checking, (7) configure the DataHoider, (8) get more information (additional data) and (9)
clone. All of these operations can be customizable by inheriting the DataHoider class.
148 DataHoiders are important mechanism for the IGPM. They are used to implement Ports (a
port hold an artifact/intention property). When an action is assigned to a port, it will be
given to the port via the configf...) operation. The action will be saved inside the port.
When the port is rendered or getvaiueo as it is called, all actions will be processed. See
"Ports" on page 154 for more detail about how actions are processed.
Executables
An Executable is an executable unit in an object form. It can be passed along like other
data object. There are four kinds of executables in CurryVM, each differ in the way the
parent stack is accessible by the executable code.
First, a Fragment, is an executable that has a direct access to the context that executes it
(its parent context). This means that all local variables of the parent stack is accessible. If
a new variable is created in a fragment, the variable will also exist in the parent stack
after its execution. A Fragment does not take parameters.
The second type of executables are Macros. A Macro is a parameterized-executable
object that can access all local variables of the parent stack. In contrast to those in a
Fragment, new variables created in a Macro will not be accessible to its parent stack.
In addition, subroutines are executables with parameters that have no access to their
parent stack. Subroutines are similar to functions in major programming languages.
Last, but not least, a Closure is an executable that carries the stack in which it is created.
This allows its code to access all local variables in that stack (in the similar way Macro
have access to the parent stack). However, Closure does not have access to any variables
149 in the context that executes it. In other words, a Closure looks like a Subroutine in the perspective of the caller, but it looks like a Macro to its creator.
Executables can be created at run-time (in fact, Closures can only be created at run time).
When this occurs, CurryVM has a mechanism for executables to carry some of the local variables' values with it in the form of constants called "Frozen variables." Frozen variables are used to customize the executable code since their values are frozen at the time the executable is created.
Executables and frozen variables are key mechanisms for the implementation of the
IGPM. In the IGPM, actions will be executed when ports/patterns are rendered. When an action is assigned to a port, an executable is created and all local variables that the action uses are frozen with it for later use.
StackOwaerS stackowner is an object that owns a stack (a level of executable context). In other words, the stack belongs to that object, stackowner of a stack is determined by the object that the executed code belongs to. For example, when obj .methodo is executed, the code of methodo will be executed with obj as Stackowner.
The stackowner will be used to determine access permission, stackowner enables
CurryVM to implement access control mechanisms similar to the one in Object-Oriented programming. CurryVM ensures that an access permission is properly enforced.
However, developers can customize permission access rules.
In the IGPM, the stackowner will be saved (together with the code location) into the
150 ActionRecord when an action is assigned to a port. The ActionRecord will be used for tracking the intention of the port's final value and for the error report.
6.2.2 Regular Parser (RegParser)
Pattern language is compiled by a pattern language compiler. The compiler is implemented using Regular Parser. Regular Parser is a parser and compiler engine inspired by Regular Expression language [150]. RegParser allows a parser to be defined as a token type and reused. This enables the creation of complex parsers. A parse result can also be compiled by the token type that parses it. The details of the pattern-language parser is outside of the scope of this thesis.
6.3 The Pattern Programming Language
The pattern programming language is a programming language developed for programming patterns in the IGPM. It supports the pattern programming paradigm described in 6.1 "Pattern Programming Paradigm" (page 140). The language's aim is not only to implement the pattern programming paradigm, but also to ensure that programs written with this language will be readable and easy to maintain (as it will improve the agility of the patterns). This section covers the pattern programming language.
6.3.1 Basic Elements
Since pattern language is built on top of the CurryVM, it derives a number of features and characteristics from it. The CurryVM is a virtual machine running on top Java, which is a very popular programming language, both CurryVM and the pattern language, in turn, borrow heavily from Java especially in the general imperative and object-oriented
151 programming aspects that are unrelated to the pattern programming paradigm.
The pattern language supports four major programming paradigms: 1) the imperative, 2)
the object-oriented programming, 3) the functional programming and 4) the pattern
programming. In this sub section, the imperative and object-oriented part of the language
will be presented.
Variables
In the pattern language, there are two kinds of variables: regular variables and ports. This
part will discuss only the regular variables. Ports will be discussed later.
Variables in the pattern language are very similar to those in Java. The main difference is
that pattern language can be accessed by its name at runtime. Also, variables with the same name can be declared in nested stacks. The hidden variable (of the parent stack) can
be accessed by using the stack name or 'parent' as prefix.
Operators
All Java operators exist in the pattern language. Few operators are added, partly to make
it suitable for the IGPM (mostly short-cut for string operations). For example, the operation '?$' convert the object to a string (similar to '.tostringf)' operation). Another example is the string format operation \f"Hello %s: i; -(-world-)' equivalent to
'string, format("Hello "World");'.
Since the pattern language is built on top of the Curry VM, direct access to the curry instruction can be made. For example, 'e:printin("Heyi 1<-)' will print the word 'Hey*' to the standard output.
152 Control-flow Statements
Similar to the operators, all Java control-flow statements are simulated by the pattern
language. They also have the same syntax. The following code will behave the same way
as it does in Java code.
llforfint i = 0; i < 10; i++) 2| System.out.println("i: " + i);
Types
All Java types and classes can be accessed by the pattern programs. Java primitive types
and their object counterparts are seen as the same type; hence, 'int i = io,-' is same as
'integer i = io;'. Other Java object types can also be seamlessly accessed.
Other than the Java types, a few kinds of types are added to make the language more
flexible so that pattern codes are easy to write and maintain. For example, variant types
can be used to allow multiple unrelated types to be assignable as a single type. For
example, a type Text used very often in the pattern programs can be assigned by char,
chart], charsequence and Hascharsequence (an interface for a type that can return a charsequence). Another example is Hascharsequnce, which is a duck type [151] or a relax
interface (interfaces that the implemented types do not need to declare for
implementation). To implement Hascharsequnce, a type simply has to implement a
method 'getcharsequence() .-charsequence' without the need to explicitly declare the
implementation of Hascharsequnce.
As an Object-oriented language, object classes can be written in. The classes as well as
patterns can implement Java interfaces for further compatibility with Java.
153 From the perspective of type checking, the pattern language is a strong and static typed
language. All expressions and operations are checked at the compiler time for type
compatibility. The compiler will warn or even stop the compilation when certain type
incompatibilities are discovered.
Access to Java Library
Since all Java Types can be accessed by the pattern program code, all of the Java libraries
both the built-in and the added-on is accessible by patterns. For example,
'system.out.printinf "Hello! iprints the word 'Hello! i /' to the standard output.
However, there is one exception, the Java generic does not work in the pattern programs.
To compensate, the pattern language comes with a set of basic collection classes (such as
list, set and map) so that it can be used with Java generic. For example, 'List
new simpieList
simple list of integer as the default value. Since List of the pattern language
implement java.utn.List interface, it is seen as a regular list object by Java code (in
case it is passed on as a parameter to a Java code).
6.3.2 Ports
A Port is a special reference or variable. In a regular programming paradigm (imperative
programming), a reference (variable) can be given a value to hold. As execution progress,
another value may be assigned to the reference, changing the state of the variable. A port
is similar to a variable in the sense that a value can be assigned to it. The difference of the
two is that only one port value will be used as a part of or will affect the final product (the
software code). That value is the port final value. This means that all values given to the
154 port but do not become the port's final value are never used. Therefore, allowing the value of a port to be changed over time does not have any benefit and may create confusion to developers as they may not know which assigned value will be used (See the section "6.1.1 Artifact Properties").
Since a port is only allowed to hold one value (the final value), there must be a mechanism to ensure the final value is really the desired value. This splits a port into two states: pre-rendering and post-rendering. Rendering is a process of determining the final value of the port; in other word, before and after the port's final value is determined.
Before the port is rendered, it has no value. Patterns (intentions) may propose the final value. This proposal is called "action." Actions can be assigned or given to the port during the pre-rendering state. The final value of the port is the result of the collective decision-making process of every assigned actions.
To understand this, consider the following example.
port int I - 10; port int J = I + 5; I := 5;
I-»render( ); J-»render();
In a regular imperative programming language, the value of i and J at the end of the execution should be 5 and is, respectively. In IGPM, on the other hand, the final value of i and J at the end of the execution will be 5 and 10 respectively.
In the above code, i and J are integer ports (as indicated by port int'). In the first line,
155 the default value of x is given to be JO (' = io' is an action to "set default value of x to io"2). This means that if there is no other assignment action given to the port x before the port is asked to render, this default value will be used. The second line specifies that, the default value of J will be five more than the final value of x. The action of '= i + 5' is assigned to J but since we do not know the final value of x yet (not yet rendered), the calculation is suspended for later execution ("6.1.2 Delayed Computation"). The third line specifies that port x must be set to 5 ('.•=' is the high-priority or forced assignment operation). The last two lines ask x and J to render and its final value is finally obtained.
In the code, the port x is given two actions 'x = 10' and 'x .•= 5*. Since the operation in the action 'x .*= 5' has a higher priority than the default value (given by the operation
'=' of the action 'x = 10'), the final value of the port x would be 5. Then, the value of the port J can be determined by the action given to it which is '= x + 5' so the result is 10.
Ports are usually employed as properties of Pattern objects, but, as seen above, they can also be declared as local ports.
Port Kinds
There are many kinds of ports distinguished by the type of actions that can be applied to them and the way intentional tracking is handled. For the moment, two kinds of ports are implemented: 1) single valued ports and 2) collection ports (list). Other possible kinds of ports are Array and Map.
A single-valued port is a port that the final value is a single value in the sense that the intention tracking of the value is a single entry. In other words, there is only one intention
2 '=' is the default value assignment operation.
156 or pattern responsible for the final value. Just for the sake of demonstration, if the port type is a list of integers, this list will be seen as a single value and the intention tracking is done on the entire list as one entity (see "6.3.6 Intentional Tracking").
A collection or list port is another kind of port already implemented. Values can be appended to the port and all of those values will become the elements of the collection.
The developers can track the intentions of each individual element. Collection ports are used generally for multiple-value data.
Array and Map ports will be useful to the pattern language, but at the moment, the single- valued and collection ports are sufficient.
The following code shows how a single-valued port i and collection port is, both of an integer, can be declared.
1 port int I =0; 2 port int* Is = new ();
Port Rendering and Port Actions
To obtain the value that a port holds, the port must first be rendered. The port can be asked to render by using the Datanoider operation "render" (see DataHolders on page
1 148). For example, 'i-»renderci) will render the port i. The port will also render if it is asked for its value. For example,'§:printin('i = - + i),-' will force the port i to render before its value is concatenated and printed to the standard output.
When a port is rendered, all actions given to it will be executed. As mentioned earlier, an action is an executable unit assignable to a port and have effect on the port final value in
157 some way. Actions can be grouped into two broad categories: 1) port and 2) pattern
actions. As the name implies, Port actions are the actions for Ports and Pattern actions are
to be used for Patterns. This part aims to describe only the port actions.
There are different kinds of port actions; they can be grouped into four categories based
on their priorities: 1) pre-rendering actions, 2) assignment actions, 3) configuration
actions and 4) post-rendering actions. When a port is rendered, all the actions assigned to
it will be processed.
Pre-Rendering Actions Pre-Rendering actions should be processed just before the port is rendered. These actions
may assign additional actions into the port. For example, the following code will return a
list Of '(0, 10, 20, 30, 40]'.
1 port int Length - 10; 2 port int* Inta » new() ; 3 before)Inst): { 4 for(int i = 0; i < Length; i++) 5 Ints <+= i*10; 6 > 7 Length := 5; 8 Ints-»render( );
The operation .-before?.... >' will create and assign a pre-rendered action into
port mts. When the ints is about to be rendered (on line 8), the action will be executed
and port mts will be appended with the value o, 10, 20, 30 and 40. These appended values
will later be processed in the rendering.
It is worth noting that a pre-rendering action assigned into a port after it is already
rendered will be executed right away and no error or warning will be raised (unless the
action itself produces an error).
158 Assignment Actions Assignment actions are actions that decide the object that will become the final value.
There are three kinds of assignment actions: 1) default assignment actions, 2) force assignment actions and 3) new-instance assignment actions.
Default Assignment Actions Default assignment actions are low-priority assignment actions. If the port has been given a forced assignment action, the default assignment actions will be ignored.
Force Assignment Actions Force assignment actions are high-priority assignment actions. If there are more than one of these actions and their suggested values are not the same value, a port value conflict error will be reported with all the suggested value and the ActionRecord for each value.
To assign a default assignment operation to a port, use the operation '=' while a force assignment operation can be given to a port using the operation The example in
"6.1.1 Artifact Properties" demonstrates the difference between these types of actions.
New-Instance-Assignment Actions The third kind of assignment actions is the new-instance assignment actions. This kind of actions is required because the nature of constructor and object initialization is incompatible with the concept of the artifact-property concept (See the section "6.1.1
Artifact Properties") - an essential concept for the pattern programming paradigm. To demonstrate this, consider that there are two pattern types: ArrayFieid and
ImmutableArrayField. ImmutableArrayField extends the pattern type ArrayFieid. Now, the programmer A wants a field to be an ArrayFieid, while programmer B wants the same field to be an immutabieArraFieid. The following is what they may code.
159 port Field F;
// Programmer A's Intention F := new ArrayFieid();
// Programmer B's Intention F := new ImmutabieArrayFieid();
F-»render(); //An conflict error is thrown here.
Since, by definition, immutabieArrayFieid is a kind of ArrayFieid, a new instance of immutabieArrayFieid should be considered a new instance of ArrayFieid. However, the code 'new ArrayFieido' produces an instance that is not the same with or even equal to the object produced by 'new immutabieArrayFieid()'; hence, the error message is thrown.
The immediate solution for this, as one might come up with, is to use default-assignment actions - which may solve this particular problem - but what if there is another pattern type, lets say, ClonedValuelmmutableArrayField extending from ImmutabieArrayFieid and programmer C want to assign it into F. This will simply complicate the problem.
Because the above scenario is likely to happen in a real-world project, there must be a proper solution. For pattern programming language, the solution is the new-instance assignment actions. With those actions, programmers can specify that they want the port to have a Pattern object of a particular type. If there are many types specified by many new-instance assignment actions, those types will be checked if an object of a type can be made that satisfies all the actions' requests. To demonstrate, consider the following code.
i port Field F; 2 3 If Programmer A's Intention 4 F new() as ArrayFieid; 5 6 // Programmer B's Intention 7 F • — new() as ImmutabieArrayFieid? 8 9 // Programmer C's Intention 10 F := new() as ClonedValuelmmutableArrayField; 11 12 F-»render();
160 Port F will hold the new value of type cionedvaiueimmutabieArrayFieid since the type is compatible with all other specified types. This should solve the problem in a satisfactory level, but it raises one important limitation - the parameters of the constructor.
The new value of ArrayFieid and immutabieArrayFieid will not be compatible if the construction of both requires parameters and the given parameters are not the same.
Therefore, only non-parameterized constructors should be allowed with the new-instance assignment actions.
Configuration Actions Configuration actions are actions that will change the setting of a port value. At this time, the port value is not yet considered to be the final value. An example of configuration actions are list-appending actions; the list object is there after all the assignment actions are executed. The list-appending actions just add a value to the list.
After all of the configuration actions are processed, the port will be marked as rendered.
All assignment and configuration actions given to the port after this point will result in a
"Port already rendered" error. Only pre-rendering and post-rendering actions may be assigned after this point.
Three built-in configuration actions are available and they are all for collection ports.
1. Append actions - actions that append values into the list.
2. Sort actions - actions that reorganize a list. There are two ways elements can be
sorted: by order number and by compared value.
3. Filter actions - actions that validate the element of the list. If the filtering fails, a
161 port assertion exception will be thrown.
The following code demonstrates the use of those actions.
i // New a regular array of number. 2 int[] Ints = new int[] { 5,3,8,2,7,9,6,1,0,4 }; 3 // New port 4 port int* Odds - new (); 5 port int* Evens • new (); 6 7 // Ensure that each element is even or odd accordingly. 8 -:filter(Evens):{:(Element):boolean; return (Element *2) »« 0; } 9 -:filter(Odds ):{:(Element):boolean; return (Element %2) == 1; } 10 11 forfint i = 0; i < 10; i++) { 12 int n - Ints[i]; 13 if((n % 2) «» 0) Evens <+« n; // Appending 14 else Odds <+= n; // Appending 15 } 16 17 // Sort by order number (process one element at a time) 18 sort(Evens):{:(Element):int; 19 // Lower order number comes first 20 return Element 21 > 22 // Sort by comparing (compare two elements) 23 ~:sort(Odds):{:(Elementl, Element2):int; 24 // Returns 0 for equals 25 // Returns negative for 'Elementl comes before Element2' 26 // Returns positive for 'Element2 comes before Elementl' 27 return Elementl — Element2; 28 29 30 Evens-»render() j 31 Odds -»render();
The above code will result in Evens is [0,2,4,6,8] and odds is [1,3,5,7,9],
Post-rendering Actions Post-rendering actions are actions processed directly following the port rendering. This type of action often makes use of the final value. If post-rendering actions are assigned to a port after the port is already rendered, the actions will be executed immediately.
The operation after(...is used to assign a post-rendering action to a port.
For example, if the value of the port i should be printed to the standard output when it is rendered, the following code may be written.
1|port int 1=0;
162 2 -:after(I): { 6:println(~I • " + I); > 3 ... 4 I-»render();
A common kind of post-rendering action is assertion action or action to validate the port
value. Since this is often used, there is a special syntax for assigning them. The following
code shows how we can ensure that the value of the port x will be under 10 (an error
should be raised if the final value is over 10).
1 port int 1=0; 2 ~:assert(I < 10); 3 ... 4 I-»render();
If the assertion fails, a port assertion exception will be thrown.
Final Value and Location Tracking
Once the rendering process is completed, the port will have its final value. The final
value will be of the same type with the port type. In case of collection ports, the value
will be of the List type.
The port will also hold another object after it is rendered; that object is the Finaivaiue
object (notice the capital litters). The Finaivaiue object holds extra information about the
final value especially about ActionRecord used for intention tracking (a Pattern that is
responsible for the final value). For the collection port, the Finaivaiue also holds the
ActionRecord of each element.
To obtain the Finaivaiue object of a port, call the DataHoider operation
'getFinalValueO' of the port; that is, for instance, 'ls~»getFinalValue()'. And the
ActionRecord can be obtain by using the method 'getActionRecord()' of a Finaivaiue
163 (i.e., 'is-»getFinaivaiue().getActionRecord()'). More information about intentional tracking can be found in the sub-section 6.3.6 "Intentional Tracking."
6.3.3 Patterns
For the IGPM, patterns represent intention. They encompass a set of software development actions corresponding to the intention. Patterns may also represent software artifacts. More about patterns in the perspective of the IGPM, see section "5.4 The
Intentional Generative Pattern Mechanism (IGPM)."
For the pattern language, patterns are instances of pattern types in the similar way
Objects are instances of Class types. Patterns also differ from regular objects because they have ports as their attributes rather than variables. Moreover, patterns are two-state entities (pre-rendered and post-rendered), which also means that they can be rendered. In this sub section, patterns are discussed as essential components to the pattern programming language.
Attributes
As an object, a pattern has a set of attributes (or fields). There are three kinds of pattern attributes: constants, in-port and out-port.
Constants Constants are type-scoped or static attributes. As the name implies, the value of constant is unchangeable. The role of constants in the pattern language is the same as static final fields in Java, one of which is to reuse a data. A pattern type's constant can be access by
operation, e.g., 'PatternTypel.Constantl'.
164 In-ports and Out-ports In-ports are ports that aim to be input properties of patterns, while out-ports are ports that
are intended to be output properties of the patterns. The most important difference of the
two is that in-ports will be forced to render at the beginning of the rendering process. In
contrast, out-ports will only be forced to render at the end of the rendering. This also
means that all ports of a pattern will be rendered by the time the pattern is rendered. A
pattern port can be accessed using operation, e.g., 'patterni .porti'. Since these are
ports, port operators can be used with them, for example,'-»render()' operation.
Operations
There are three kinds of Pattern operations: Procedures, Functions and Methods. Please
note that the name of these operations are given just for the sake of distinguishing them
and there is no implication for their characteristics to operations with similar names in
other languages.
Procedures Procedures are type-scoped operations. As type-scoped entities, procedures do not have
access to Patterns' ports. They are usually used as service operations with the Pattern
types as their namespace.
Functions and Methods Both Functions and Methods are instance-scoped operations. Functions are operations
that have no effect and are not effected by the state of Patterns. A Pattern's Function can
be called before, after, or during the rendering. In contrast, a Method can only be called
after the Pattern is rendered. If a Method is called before that, the Pattern will be force to
render. A Method cannot be called during the rendering as it will trig another rendering
process and cause a recursive rendering to be thrown. Because of this difference, a
165 Pattern's Method is likely to be an operation that require all Ports of the Pattern to be
rendered so that it can access to the value of those Ports. Besides this difference,
Functions and Methods are identical.
The Render Function The render function of a Pattern is an executable body that will be executed at the
transition of the Pattern from the pre-rendering to the post-rendering state. If the render
function is called while the Pattern has not yet been rendered, the calling will force the
Pattern to change its state. If the function is called when the Pattern is rendering, an
exception will be thrown. Nothing happens if the function is called after the Pattern has
rendered. In other words, the execution of the render function and the changing state of
the Pattern are mutually correlative events.
To explicitly force a Pattern to render, call its render function, e.g., 'patterni .rendero'.
It is also possible to check if the Pattern is yet to be rendered simply by calling
'isRenderedf)' function.
Pattern Actions
Pattern Actions are Actions given to Patterns. Since Patterns do not have to determine
their final value as Ports do, there are only two kinds of Actions for Patterns: pre-
rendering actions and post-rendering actions. Both kinds of actions behave and are
treated the same way as their Port Actions counterparts. To assign pattern actions, the
operation '-:after(...and '~:before<... should be used for pre-
rendering and post-rendering actions, respectively.
166 6.3.4 ActionRecord
An ActionRecord object holds information recorded when an action occurs.
AcctionRecords are designed and used for tracking intentions. ActionRecord is created when a port/pattern is given an Action or when a Pattern is created. An ActionRecord object holds two informations which are: actor (the stackowner at the time) and
Locationsnapshort.
The Actor is the stackowner at the time the record is taken. Therefore, it is the pattern or intention that is responsible for the action.
The Locationsnapshort holds information about the code location at the time the record is taken. These informations may include the signature of the method/function, the code- file name, the coordinate (column and line number) and the nearest documentation object.
6.3.5 Text
Texts in the pattern programming language are objects of the Java charsequence interface. It is designed in the way that multiple charsequence objects (which include
String) are to be composed, forming a structure of charsequence objects. It also allows additional information to be attached to any of the sub charsquence, joining the structure.
These two features are used to implement intentional tracking in Text. As a result, Text is a structural composition of all its sub-Texts. Each sub-Text is attached with the
ActionRecord for its value. As already mentioned, the ActionRecord can be used to find the Pattern responsible for the final value.
167 Text Literals
A short text literal is declared with the operation or $\..\ Meta character
backslash V can be used for coalition avoidance. Short text literals cannot contain a
newline (use '\n as an escape for that). On the other hand, long text literals can contain
new line characters. They can also directly contain any characters without escaping. To
declare a long text literal use <${...}$>, <$[...]$>, <$"{...}"$>, <$"[...]"$>,
<$•{...}• $>, or <$ • [...]' $>. There are many different open-close pairs because they can
be used to avoid escaping the closing marks. The following code shows how Texts can be
created.
i // Only in this code. 2 // Underlined characters are the begin and end of text literals. 3 II All bold characters are the part of texts. 4 // Crossed-out characters are those that are ignored. 5 Text T1 - $"A short-Y" text literal.1; 6 Text T2 - S~Another "short" text literal!; 7 Text T3 = <$fLong text literal goes across lines.
If an escape is necessary, use a Text template.
Text Templates
Text templates are used to create complex and dynamic-content texts. With text templates, the string or text value of an expression can be concatenated into a text. The
pattern language provides many flexible ways of creating text templates (again, to avoid
168 coalition). For this thesis, only one most-used way, however, is presented as it is enough for understanding examples presented later.
Text template's declaration is very similar to that in the long text literals. The inserted value is marked with '<(' and for expressions and '<{' and '>>' for statements. Within these marks, an echo statement can be used to append any text as part of the template's value. This seems like a complicated rule, but it is quite simple. The best way to understand this is to consider the following example.
i // Only in this code. 2 // Underlined characters are the begin and end of text literals. 3 II All bold characters are the part of texts. 4 II Crossed-out characters are those that are ignored. 5 // Italiced characters are expressions and statements to be executed. 6 Text T1 -
Again, it is worth emphasizing that ActionRecord is created to document each part of composed text template.
6.3.6 Intentional Tracking
The IGPM is designed to be a solution for Agile documentation with an aim to provide an efficient way to document all aspects of the software. It utilize automatic code generation to achieve that. There is one problem with this mechanism - the connection between the generated and the generator. These connections are required as vital parts of natural documentation because the developers can identify the pattern (the intention) that is responsible to a piece of code or a port value. Intention information is kept in an
ActionRecord object.
169 By accessing attributes of the ActionRecord object, the developer can obtain information about the executable context at the time the action occurs:
1. Actor (stacJcOwner)
2. Signature of the current executable.
3. Code file's name, line-number and column number.
4. The expression that performs the action.
5. The explicit documentation (if available).
This information should be more than enough to investigate and leam about how the final product code comes to be.
Since an ActionRecord may be created by variety of action and hold by variety of artifacts, there are different ways to obtain the ActionRecord objects. The following code shows some of those ways.
1 Text T; 2 Pattern P; 3 4 5 // ActionRecord when T is created. 6 T.getExtraIn£o(); 7 8 // ActionRecord when the third sub-text is created. 9 T.getStructure(2),getExtraInfo(); 10 11 // ActionRecord when P is created. 12 P.CreatorAction; 13 14 // ActionRecord when the value of the port Attr is assigned. 15 P .Attr-»getFinalvalue( ). getActionRecord(); 16 17 // ActionRecord when the forth element of the collection port Attrs is assigned. 18 P.Attrs-»getFinalValue().getFinalValue(3).getActionRecord();
170 It is possible to specially develop an application to graphically investigate intentions.
However, at the moment, the investigation can only be done programmatically.
6.4 Examples
This section provides examples demonstrating how the pattern programming language (as an implementation of the intentional generative pattern mechanism (IGPM)) can be used.
Each example is designed to show different aspects of the benefit of using the mechanism. The more complicated example that shows how the IGPM is used in a real- world-liked project will be provided in the next chapter.
6.4.1 Example #1: Patterns as Artifacts
Patterns can represent software artifacts. This is very straightforward; a pattern is used to represent an artifact and its ports represent the artifact's properties.
Problem
The product of the IGPM is software code. In most cases, the source code must be saved to a file before it can be further processed (i.g., compiled or execute); hence, saving code to a file is a very common task in the IGPM. This example shows how a pattern can represent a file. By representing a file with a pattern, the developer can utilize the
"artifact properties" and "delayed computation" concepts for source code file creation.
Patterns
As an artifact, a text file has properties which are: 1) Full name (1.1 name, 1.2 extension,
1.3 path and 1.4 base path) and 2) the content. The following is the pattern code for
Text File pattern.
171 TextFile.pattern 1 // JPattern: 2 package nawaman->text; 3 4 import net.nawaman.util.UText; 5 6 public pattern TextFile { 7 8 in port Name :Text; 9 in port EXT :Text; 10 in port Path :Text; 11 in port BaseDIR :Text; 12 13 in port Content :Text; 14 15 out port FullName:Text « <${$... 16 <(BaseDIR )><((?BaseDIR:"/"))>$. 17 <(Path )><((?Paths "/"))>$. 18 <(Name )>$. 19 <((?EXT:"."))><( EXT )>$. 20 }$>; 21 22 23 public new() { 24 this.ensureName_NotEmpty();
25 > 26 27 public function render():void < 28 this.saveFile(); 29 > 30 31 private function ensureName_NotEmpty():void { 32 assert(Name): (Name?$.length)) 1= 0); 33 } 34 35 protected function saveFile():void < 36 String theFileName - this.FullName?$; 37 String theContent « this.Content ?$; 38 UText.SaveTextToFile(theFileName, theContent); 39 } 40
The code is very intuitive and self-documented. There are five in-ports: Name (#8), EXT
(#9), path (#10), BaseDIR (#11) and content (#13). They all have no default value.
There is one out-port: FuiiName (#15). FuiiName is assigned with a default-assignment action (see Default Assignment Actions on page 159). The default value of FuiiName is a composition of four of its in-ports (#16-19); starting with BaseDiR and V if BaseDIR is not null; then Path and '/' if Path is not null; the name; ends with '.'if EXT is not null and the
EXT (see Text Templates on page 168)3.
3 The operation (?... .* ...) is the optional-text operation, which returns null if the expression after is null and returns the text afterwhen the expression is not null.
172 The pattern render function simply executes the saveFiieo action (#35). The action calls the utility method saveTextroFiie(...). The pattern also has one action - ensureName_NoteEmpty {). This action ensures that the name of the file is not empty. It is worthy to emphasize that the assertion will only be computed after the value of name is concluded (rendered) even though ensureName_NoteEmpty(), as a function, is executed by the constructor.
Usage
i TextFile TF1 = new(); 2 TF1.Name := "testtext.£ ile"; 3 TF1.Content := "Hello World 1!!"; 4 TF1.render(); C D 6 TextFile TF2 = new(); 7 TF2.Name := "heiio.php"; 8 TF2.Content := "Hello
The first part (#1-4) of the code above creates a file named 'testtext.fiie on the local directory with the word 'neiio world' in it. The later part creates a file named 'heiio.php at the folder'/var/www/exampleOl /' with a PHP code to print'Hello World! 11'.
In the case where many files share the same properties' values, a pattern type can also be created to handle that. In the following code, PHPFiie and PHPFIIE_Exampieoi pattern are created so that multiple PHP files for "exampleOl" project can easily be created.
1 pattern PHPFiie extends TextFile { 2 new() { 3 super(); 4 BaseDIR - "/var/www"; 5 EXT « "php"; 6 } 7 new(aName :Text) { 8 this( ); 9 Name := aName; 10 > 11 } 12 pattern PHPFile_Example01 extends PHPFiie { 13 new() { 14 super();
173 15 Path » "exampleOl";
16 > 17 newfaName :Text) { 18 this(); 19 Name aName; 20 } 21 } 22 23 TextFile PHP1 = new PHPFile("hello"); 24 PHPl.Content := "Hello World!II"; 25 PHP1.render(); 26 27 TextFile PHP2 - new PHPFile_Example01("customers"); 28 TF2.Content ; 29 TF2.render(); 30 31 TextFile PHP3 - new PHPFile_Example01("supplyers"); 32 TF2.Content := 33 TF2.render();
Notice that BaseDiR (#4) and EXT (#5) are given a default-assignment operation, while
Name (#9) is given a force-assignment operation. This is to reflect that PHP files are likely to be created in '/var/www'4 and its extension is likely to be ' .php but this can be overridden if needed. For example, some files may use '.inc as "PHP include files". On the other hand, if a file is given a name, the file name must be that given name.
The more realistic case of using the file pattern is to include it as part of a project pattern.
Then programmers of the project can manipulate the file. The following code shows a pattern type called ProjectExampleOl, ProgrammerA, ProgrammerB and the project
ProjectExmapleOl.
ProjectExampleOl.pattern l pattern ProjectExampleOl { 2 port CustomerPHP :PHPFile := new("customers"); 3 port SupplyerPHP iPHPFile :•= new("supplyers"); 4 5 function render()svoid { 6 CustomerPHP.render)); 7 SupplyerPHP.render(); 8 > 9 > 10 abstract pattern ProgrammerExampleOl < 11 port Project :ProjectExampleOl; 12 }
4 Where PHP files are employed or read by the web-server.
174 ProgrammerA.pattern pattern ProgrammerA extends ProgrammerExampleOl {
function render():void { Project.CustomerPHP.Content s- ... >
ProgrammerB.pattern pattern ProgrammerB extends ProgrammerExampleOl {
function render()ivoid { Project.SupplyerPHP.Path := ... Project.SupplyerPHP.Content s= ...
>
ProjectExampleOI .project ProjectExampleOI PExOl » new(); II Create both files ProgrammerExampleOl PgrA - new ProgrammerA(PEx01); II Manipulate the files ProgrammerExampleOl PgrB « new ProgrammerB(PExO1); // Manipulate the files
PgrA.render(); PgrB.render*); PexOl.render));
Programmer A and B can manipulate the files independently and have those manipulations affect the final generated code.
Discussion
A pattern can be used to represent an artifact (a tangible entity). This is done so that many intentions or patterns (written by many developers) can collectively specify the properties of the artifact. Consider the PHPFiie pattern; a project manager may be the one who determine what path the file should be in; a software architect may specify the name of the file and a programmer may give the file its content.
6.4.2 Example #2: Patterns as Documents
Patterns can be used to document software. They are intended to represent all aspects of the software. This example, however, only shows how patterns can be used to document the software code when no other technique can (except for the explicit documentation).
175 // ©Pattern: package ex82; Radmdinf •f-vam/wtimjW to 7uwVuMr/M«i.ptip-
inport nawa«an~>text«>*; lwrit<»ale /aanfll/n s"/-(['/]+)/?(•*) pattern Rewrite_UserPath { The beginning character public port RewriteRule :Text; / is a slash - follows by a tilde. public new() { -:before(RewriteRule): RewriteRule := this.prepareRewriteRule(); > ( Group ( $ 1J Everything public function prepareRewriteRuleO :Text { f return <${<(Word_RewriteRule())x(Spaces())x(Rule{))x( Spaces())x(Target())»>$>; but } / a slash 1 private function Word_RewriteRule():Text { return <${RewriteRule}$>; } + private function SpacesO :Text { return <${ }$>; J s: ) private function RuleO :T«xt { / Optional slash ©v return <$•{<{ •> The_beginning_character_is_a_slash_follows_by_a_tilde() )x( ( Group ($2) 50 In_group_one_contains_everything_but_slash_as_a_userna«e())x(
Many web-servers such as Apache (http://httpd.apache.org/) allow the required path
to be translated or converted into another path. The best example is the user folder,
'/-userl/page.html?param=value is converted to '/users/userl/page.html?param=value'.
This enables better flexibility and organization.
To add a rewrite rule, a rule and its target must be added to the file httpd.conf. The
problem is that the language for writing the rule is based on regular expression language,
which has no way to be naturally documented (only explicit documentation is allowed).
The code below is what a rule looks like.
Httpd.conf
RewriteRule "/-<[*/]+)/?(.*) /users/$l/$2
Patterns
To naturally document the rule, a pattern Rewrite_userPath is created. An excerpt of its
code is given below (the pattern's full code can be found in the Figure 6.1 and the
Appendix A).
RewriteUserPath.pattem // 6Pattern: package ex02;
import nawaman~>text=>*;
pattern Rewrite_UserPath {
port RewriteRule :Text;
public new() {...
function prepareRewriteRule():Text { return <${<(Word_RewriteRule())><(Spaces())><(Rule())><(Spaces())><(Target())>}$>;
>
function Word_RewriteRule():Text {... function Spaces() :Text {...
177 function Rule():Text { return <${<( The_beginning_character_is_a_slash_follows_by_a_tilde() )>< In_group_one_contains_everything_but_slash_as_a_username())>< Then_an_optional_slash() )>< Group_two_is_the_rest_of_the_path() >>}$>; ) function The_beginning_character_is_a_slash_follows_by_a_tilde ():Text {. function In_group_one_contains_everything_but_Blash_as_a_usemaine():Text {. function Then_an_optional_slash ():Text {. function Group two is the rest of the_path ():Text {.
function Target():Text { return <${<( Begin_with_the_user_folder() )>< ( Then_the_username_from_group_one() )><^ Next_is_a_slash() )><( End_with_the_rest_of_the_path_from_group_two() )>>$>; } function Begin_with_the_user_folder ():Text {... function Then_the_username_frora_group_one ():Text {... function Next_is_a_slash ():Text {... function End_with_the_rest_of_the_|>ath_from_group two():Text {...
function Start with (a :Text) :Text { function Group_of (a :Text) :Text { function The rest () :Text { function Optional of (a :Text) :Text { function Everything_ but(a :Text) :Text { function Slash () :Text {
Consider the action prepareRewriteRuieo. Its returned value is a text template for the
desired rewrite rule. The action calls other actions: Ruieo and Targeto, which take care
of the rule and the target part of the rewrite rule respectively. In addition, consider the
body of both Ruieo and Targeto; both actions call a set of sub-actions whose names
document the part of the re-write rule.
Usage CreateRewriteRule.pattem Rewrite_UserPath RUP • new(); 0:println("Result:"); §:println(RUP.RewriteRule); i:println();
8:println("Tracking:"); S:println(RUP.RewriteRule.toDetail()); i:printIn();
178 Discussion
The result of the createRewriteRuie.pattern is shown in the Figure 6.2. The result is not exactly human-friendly, but it gives an idea about what information is available. As mentioned in "6.3.6 Intentional Tracking," a more human-friendly way of investigating the intention tracking must be developed. For the time being, the result in the Figure 6.2 is reformatted and shown below.
Reformatted Result9 1 Result: 2 RewriteRule */-(["/]+)/?(.*) /users/Sl/S2 3 4 Tracking: 5 prepareRewriteRule() ( 9, 12) 6 Word_RewriteRule() ( 51, 15) |" Rewritel 7 Spaces)) ( 51, 16) "1 I" 8 Rule() ( 9, 19) 9 The_beginning_character_is_a_3lash_follows by a_tilde() ( 91, 26) 10 Start_with(Text) ( 57, 45) 11
Ultimately, it is easy to see that each part of the final text can be tracked back to the method that created it as well as the column and line number where it is created.
The pattern language allows intention tracking, which enables natural documentation.
5 The column and row displayed here ("(Col, Row)") is zero-started number while the line number in Figure 6.1 is one-started number.
179 0 8 88
f JI2H sssa's i * s 3 mi s***32 8 a Hisii • 3 3 3 2 if 1 2
I'is'j'a's'
r 133328 I 28 S c 3 2 2 2 8s • 8 2 <= 4t| !b !h !h 3 i J 4h 93iS I 3"S f I is I siislls! I*3A s iilihtl hit 1 r$8*?AS"s AS®8
Figure 6.2: Result and Tracking of the RewriteJJserPath
180 The developer can investigate from the code up to the high-level patterns.
In addition, this example only emphasize the usefulness of the IGPM for documentation.
The example, however, poorly demonstrates how the IGPM can be used to improve agility (the later example aims demonstrate that). This example can be improved to the point where it can be agile. For example, if a set of patterns are created to construct regular expressions and the target expressions. Then, the Rewritejuserpath pattern will become a higher-level pattern that using the set of patterns. Any modifications to high- level action in the Rewritejuserpath pattern (which may require no or little knowledge of regular expression) can be made and the rewrite rule will be automatically created once the pattern is rendered.
6.4.3 Example #3: Patterns as Low-level Intentions
Representing low-level intentions often involves code templates. This is very useful in the situation where multiple places in the code are controlled by a few configuration variables. With a few configurations, small modifications effect the final code in a significant way (small modifications cause big changes). A pattern also ensures that the changes to the final code are done in a cohesive manner and minimize mistakes.
In many cases, multiple computations can be encompassed into a bigger executable units such as functions, classes and so on. In a number of cases, this cannot be done. For example, in communication protocol, there must be code to send and receive data in a specific steps. That means that the code of sending and receiving must be synchronized.
Unfortunately, the synchronization cannot be done as the code of sending and receiving
181 are usually located in different executable units i.g., different functions.
Problem
The example patterns generate code that process parameters in Bash programs. A Bash
program is a simple script written in the Bash-shell language, usually used to extend (or,
in a sense, wrap) existing applications in a Unix-liked environment. Since Bash language
is not expressive in processing data, as it have very limited data-processing operations
and control-flow statements, doing such computation in Bash results in long program
code. The following code is an example:
Example code for option processing in Bash l SIZE="64" 2 FILE3*" " 3 ISREPLACE-"NOREPLACE" 4 5 while getopts ":rs:f:" opt 6 do 7 case "$opt" in e "a") SIZE="$OPTARG"; ;; 9 "£") FILE="$OPTARG"; ;; 10 "r") echo "'-r' requires no parameter"; exit; ;; n "?") echo "Unknown option $OPTARG"; exit; ;; 12 13 case "$OPTARG" in 14 "s") echo "Missing parameter for '-s' (SIZE)"; exit; ;; 15 "f") echo "Missing parameter for '-f' (FILE)"; exit; ;i 16 "r") ISREPLACE-""; ;; 17 "m") echo "-f FILE [-s SIZE] (—r]"; exit; ;; IB esac 19 exit;
20 t t 21 *) echo "Unknown options"? ?; 22 esac 23 done
In the above example, the program processes three options: '-s for size, '-f' for file name
and '-r' for a flag indicating if the output file should replace the input file.
'-s is an optional option ('[...]• in line#17). It requires a parameter (the size) ('echo
"Missing parameter ..exit;1 in line#14 and's:' in line#5) and its parameter value
will be assigned to the variable 'SIZE' ('SIZE=... • in line#8). The variable has 64as its
182 default value (SIZE="64" • in line#l). '-f' is a required option. It takes a parameter. The
value is assigned to the variable 'FILE', which have as the default value, '-r is an
optional option. It takes no parameter. The value is assigned to the variable 'ISREPLACE',
which has '"NOREPLACE"' as the default value.
Patterns
It is simple to see that modifications in properties of an option will result in changes in
multiple places in the code. To solve this, two pattern types are defined: option and
processoptions. The following is part of those patterns' code. The full source code of
both patterns can be found in the Appendix A.
option Pattern option is a pattern representing an option (as artifact)6. The pattern ports line #8-14, are
the pattern's properties. Figure 6.3 shows the effect of option's properties to the final
code. The constructors in line# 16,#24,#32 and #35 provide convenient ways to create an option with properties' values. In the default constructor (newo in the line #16), default
values are given to the port Name, OptionChar and IsNeedParameter. The pattern
encompassed two actions: ensurevarNamejuotmiiorEmptyo (line #21 and #41) and
ensure0ptionchar_NotNuii() (line #22 and #44). The pattern also has five service
functions that will be used to create different part of the code executed by the pattern
processoptions. Figure 6.3 also shows the line numbers of the service functions in option.pattern that produces each particular pieces of code.
Option.pattern // ^Pattern: package bashoption; import nawaman->textss>* ?
6 The code of Option pattern can be fount on page 246.
183 #52 SIZE="64" FILE' Default ISREPLACE»"NOH #55 while getopts VarName do case "$opt" IsParameterNeeded #58 "s") SIZE SOPTARG "f*rKEILE-w$OPTARG"j OptionChar "r") echo "'-r' reql is no /param< exit "?") echo "Unkng>nr^bption $pPTARG 6X1 U/
case TARG" in #66 nj5. echo "Missing parameter for (SIZE exit; "f") echo "Missing pa meter for '—f' FILE exit; "r") ISREPLACE-"" "m") echo "-f F^LE [-s SIZE] t-r]"; exit; esac #74 exit;
f t Name *1') echo "Unknown optjions"; donea° ValuelfNoParameter IsOptional
Figure 6.3: Options' properties
184 pattern Option {
port Name :Text; port VarName :Text; port Default :Text; port OptionChar :char; port IsOptional :boolean; port IsParameterNeeded :boolean; port ValuelfNoParameter :Text;
public new() { Name = VarName; OptionChar = this.getLowerCase_FirstChar_Of_VarName(); IsNeedParameter - false;
this.ensureVarName_NotNullOrEmpty(); this.ensureOptionCharNotNul1(); } private new(aVarName,aDefault,aOptionChar,alsOptional,alsParameterNeeded) {.. public new(aVarName,aDefault,aOptionChar,alsOptional) {... public new(aVarName,aDefault,aOptionChar,alsOptional,aValuelfNoParameter) {.
private function ensureVarName_NotNullOrEmpty():void {... private function ensureOptionChar_NotNull():void {... private function getLowerCase_FirstChar_Of_VarName():char (...
public function getVariableDeclaration():Text { ... public function getOptionChar_and_Colon():Text { ... public function getOptionWithParameterProcessing()iText < ... public function getOptionsWithoutParameterProcessing():Text { ... public function getManual():Text {... }
Here is an excerpt of option.pattern on the getManual o service function. In line #74, a
variable optionstr is created as a text of'-' follows by the OptionChar port. If the option
requires a parameter, optionstr will be append a space and the option's name (#76).
Finally, if the option is optional, the optionstr will be wrapped with a pair of'/-' and
(#77).
74 private function getManual():Text { 75 Text Optionstr = <${-<(OptionCiiar)>}$>; 76 if (IsParameterNeeded) Optionstr = <${<(Optio/iStr)>_<(Name)>}$>; 77 if(.IsOptional) Optionstr = <${J_<(OptionStr)>J.}$>; 78 return Optionstr; 79 }
processOptions Pattern processoptions is a pattern that composes pieces of code from each option into the final code. The full code for Processoptions pattern can be found on page 248.
185 ProcessOptions.pattem // 0Pattern: package bashoption; import nawaman->text=>*; pattern ProcessOptions {
in port Options :Option* = new(); out port Code :Text;
public new() { Code := this.prepareCode(); this.ensureOptions_NotEmpt;(); }
private function ensureOptions_NotEmpty():void {...
private function prepareCode():Text { if(Options?.size() == 0) return null;
return <${ }$> <(this.getVariableDeclarations())> while getopts ":<(this.getOptionsString())>" opt«l do J case "$opt" in J <(this.getOptionsHithParameterProcessing())>$... "?") echo "Unknown option $OPTARG"; exit; ;;J ":«) A case "SOPTARG" inJ <(thiB.getOptionsWithoutParameterProcessing())>$... "m") echo "<(thiB.getManual())>"/ exit; esacJ exit;J ;; J *; echo "Unknown options"; ;;4 esaci done J <${ }$>;
private function getVariableDeclarations():Iext { ... private function getOptionsString():Text { ... private function getOptionsHithParameterProcessing():Text { ... private function getOptionsWithoutParameterProcessing().-Text { ... private function getManual():Text { ...
The composition is done in the prepareCode() action. Five sub-actions on the line #48,
#54, #60, #66 and #72 document each portion of the result code. These sub-actions do no more than concatenate results from each option s service. One of these actions is the getManual() action on the line #72 (its code below). ProcessOptions .getManual() simply concatenates all results of option.getManual <).
186 72 private function getManual():Text { 73 return <${<{ 74 fromto(int i = 0 : Options?.size()) { 75 if(i 1= 0) echo 76 echo Options.get(i).getManual(); 77 } 78 >>>$>;
79 >
Usage The following code generates the code similar to the original source code at the beginning of the example.
CreateOptlonProcessor.project ProcessOptions POs • new(); POs.Options <+= new Option)$"FILE", 'f, false); POs.Options <+= new Option($"SIZE", S'"64"~, 's', true); POs.Options <+= new Option($"ISREPLACE", $""NOREPLACE"', 'r'( true, $'""");
Discussion
The code of both patterns are almost 30 functions about 160-lines long but it creates the final code of only 23-line. Is this worth it?
The answer is "Yes, this is worth it." The key to the answer is the client code in the file
Createoptionprocessor.project. From the code, it takes only four lines to construct the pattern network for generating the 23 lines of code. It takes one line to create an instance of ProcessOptions and only one additional line to each option. Since both patterns can be reused in other projects, the cost of creating those two patterns can quickly become neglectable.
Additionally, the patterns ensure that code fragments corresponding to an option is properly inserted to the final code and a modification made is properly performed. For example, if the developer want to allow output file name to be explicitly specified via option. He may first want to change the option '-f' for 'FILE' to '-i' for 'INPUT' and add an
187 option '-O for 'OUTPUT'. He can make the following changes and the patterns will ensure that the modification is done properly.
CreateOptionProcessor.project - new version 1 ProcessOptions POs = new(); 2 POs.Options <+= new Option($"INPUT", 'i', false); 3 POs.Options <+= new Option*$"OUTPUT", 'o', false); 4 POs.Options <+= new Option)$"SIZE", S~"64"~, 's', true); 5 POs.Options <+= new Option)$"ISREPLACE", $~"NOREPLACE"', 'r', true, $~""~);
To conclude, by having ProcessOptions pattern to represent the intention of creating
"option processor" allows the developers to effectively create (four lines of code) and maintain (one line for a new option) the product code. Additionally, any modifications that might happen will be more manageable and less mistake-prone.
6.5 Conclusion
This chapter focuses on the development of the pattern programming language as an implementation of the Intentional Generative Pattern Mechanism (IGPM). First, the
Pattern programming paradigm is developed as the existing programming paradigms are not suitable programming paradigm for the IGPM. After that, the development of the pattern programming language using Curry Virtual Machine and Regular Parser is discussed. The description about the pattern programming language is then presented.
Finally, some examples of how pattern programming language as in implementation of the IGPM, can realize Agile Documentation.
188 CHAPTER 7
CASE STUDY
The previous chapter presents the pattern programming paradigm and language as well as a few examples. Those examples are small and focus on each usage of the language for the IGPM. In this chapter, a larger, more realistic example will be presented as a case study on how the IGPM may be used in a project.
7.1 Introduction
The case study is a real-world-like assumed project. The project has similar qualities to many common small-sized, real-world projects. First, the necessity for the software is presented. Later, the system development is provided. A discussion, then, follows. After that, two changes are introduced and how the developers cope with those changes are presented.
It is important to emphasize that the aim of this case study is to demonstrate how patterns might be used in a project. Because only some fragments of these code are critical to the demonstration. Paragraphs immediately before or after any given code explain the code focusing on those fragments of code that are significant to the development. The rest of the code (for solving the project's problem) will not be discussed as it is not within the scope of this thesis.
189 7.2 Problem
Grocery2Home is a small local company providing a service of phone-in ordering for groceries. A customer calls a company's operator and provides a list of desired groceries, which will be delivered to the customer's home the next evening (just before supper time).
For years, the company's business has been generally good and reasonably sustainable.
However, in the past few months, the net profit has been shrinking in a rate that may affect the existence of the company. Looking into the problem, the general manager found the following problems.
1. Customers complained that the products arrived later than promised. Further
investigation found that the driver has to spend time going through order lists,
which contain the delivery information, then they must plan the delivery route.
This leads to time and energy spent on the planing, often every poorly-optimized
route and even missing some deliveries (so they have to return to the overlooked
house later on).
2. The operation costs have increased. These include the cost of gas, trucks and
system maintenance (ageing printing and billing facilities).
To cope with this problem, the manager consults a software company call ASC ("A
Software Company"). ASC's engineers analyze the problem and recommend that a software should be developed to facilitate the driver's tasks. This may, for example, allows for delivery routes to be determined in advance, not en route.
190 7.2.1 Requirements and General Planning
In a meeting, the development team brainstorms and collectively makes general high- level decisions about the system. The following are some of their decisions.
1. The information about the customer and the package to be delivered will be
entered by the operator when the customer calls.
2. Each route will be planned in advance (a few hours prior to the delivery time)
based on customers' addresses.
3. Only necessary information will be given to the driver and each session will be
transferred to the portable computer that the driver carries.
4. The project is broken down into a few sub-systems. The delivering sub-system
will be developed first as it is the information-using end of the system (it dictates
the necessary information).
5. Unless other uncompromisable constraints emerge, the system will be developed
using PHP as the development language and MySQL as the database back-end
since the team has a lot of experience with these particular programs.
After the general requirement gathering and planning are done, the team moves on and refines the requirements and plans for the first phrase and the first sub system to be developed (the delivering sub-system).
7.2.2 Infrastructure
Since the team has competed similar projects in the past, some patterns from those projects can be reused in this project. The code and brief descriptions of those patterns
191 can be found in Appendix B.
7.3 The Development of the G2H Delivery System
The team now focuses their attention on the first sub-system - the delivery system. Since this is a web-based application, in the sense that it is executed with PHP on a local web server, the system will be broken into pages. Each page provides some information and/or allows some control. In the meeting, the team has a general vision that this delivering system requires at least four pages.
The first page 'Route, shows information about a route (a list of streets). Once the driver arrives at the first street, he should be able to click on the street name to see the list of the customers on that street (the second page 'street'). This list of customers includes the address number in order. The driver can click a customer name and see the package number of the package to be delivered. Lastly, when the package is delivered, the driver clicks on the package name to tell the system that the package is delivered. Then, the driver can click on a link to go back to the previous page and deliver other packages.
From that initial vision, the team collectively constructs a few high-level patterns to represent requirements. All the source codes can be found in Appendix C on page 293.
The code below 'g2h/G2H.pattern contains six pattern types representing the project, four high-level requirements and an interface pattern for formatting the pages.
g2h/G2H.pattern 1 // ^Pattern: 2 package g2h; 3 4 import nawaman->text=>*; 5 import lib->phpdb=>*; 6 7 8 interface Req_Display_only_the_information_needed {}
192 9 interface Req_Provide_intuitive_link_between_each_page {} 10 interface Req_Distinguish_each_page_visually {} 11 interface Req_Display_pages_with_consistency {} 12 13 interface PageFormater { 14 public formatPage(pPage :Page) :Text; 15 public formatBody(pPage :Page, pBody :Text) :Text; 16 } 17 18 19 pattern G2H extends PHPDBProject_Simple { 20 21 constant Project sG2H = (G2H)(fhrunOncef) { new G2H(); }); 22 23 port configuration :G2HConfiguration = new (); 24 port design iG2HDesign = G2HDesign.Design; 25 26 port formater :PageFormater; 27 28 port Reql_DisplayInformations :Req_Display_only_the_information_needed; 29 port Req2_Link_between_pages :Req_Provide_intuitive_link_between_each_page; 30 port Req3_DistinguishDisplay :Req_Distinguish_each_page_visually; 31 port Req4_ConsistencyDisplay :Req_Display_pages_with_consistency; 32 33 private new () { 34 super(); 35 } 36 37 38 public addDisplayPage(pPageTitle :Text) :DisplayPage { 39 return (DisplayPage)this.addPage(pPageTitle, new DisplayPage()); 40 > 41 public addPage(pPageTitle rText) :Page { 42 return this.addPage(pPageTitle, new Page()); 43 } 44 private addPage(pPageTitle :Text, pPage :Page) :Page { 45 pPage.Name := pPageTitle?.toString()?.toLowerCase(); 46 pPage.title := pPageTitle; 47 return (Page)this.addPage(pPage); 48 ) 49 50 51 public render)): void { 52 configuration ?.render(); 53 Reql_DisplayInformations?.render)); 54 Req2_Link_between_pages ?-render) ); 55 Req3_DistinguishDisplay ?.render(); 56 Req4_ConsistencyDisplay ?.render(); 57 this.renderAHPages (); 58 ) 59 60 public renderAHPages)) :void < 61 foreach (Text aPageName : this.getPageNamesf)) { 62 TextFile aTextFile * (TextFile)this.getPage(aPageName?.toString()); 63 if(aTextFile != null) { 64 aTextFile.ToSave s- configuration.isToEmployFiles; 65 aTextFile.ToShow := true; 66 aTextFile.BaseDIR := configuration.pageBaseDIR; 67 68 if (formater != null) { 69 cast (Page aPage = aTextFile) 70 formater.formatPage(aPage); 71 } 72 73 aTextFile.render(); 74 ) 75 } 76 > 77 }
193 Pattern G2H represents the project (line #19). The project has four requirements (line #8-
#11). Each requirement is self-documented (by its name). The project's ports, defined on the line #28-31 show that the project takes the implementations of these requirements.
Each is rendered with the project rendering function (#53-56).
The project has a configuration object (#23), a design object (#24) and a pageFormatter
(#26) (used to ensure that all pages will look similar (#70)).
g2h/G2HConf!guratlon.pattern // 8Patterns package g2h;
import nawaman->text=>*; import lib->phpdb=>*;
pattern G2HConfiguration {
port dbconnection tDBConnection = new DBConnection($"$G2H DBConn", $"G2 H_DBConn.inc" ); port pageBaseDIR :Text = $"/var/www/g2h"; port isToEmployFiles :boolean = true; port Page_G2H_Logo :Text = $"g2h-logo.png"; port Page_ASC_Copyright :Text » Powered By ASC (c) 2009.";
The project configuration pattern holds data for setting the project or values to be used by
many parts of the project. Data like 'dbconnection' is used by many patterns that interact
with the project (G2HCon figuration^). Another example is the 'pageBaseDIR' which is the
target directory of all the result code files (G2HCon figuration# 10).
g2h/G2HDesign.pattern 1 // ^Pattern; 2 package g2h; 3 4 import nawaman->text=>*; 5 6 import lib~>db=>*; 7 import lib->phpdb=>*; 8 9 10 pattern G2HDesign { 11 12 constant Design :G2HDesign = (G2HDesign)®:runOnce() { new G2HDesign(); }; 13 14 // 'pg' = Page
194 15 port pgRoute :DisplayPage G2H.Project.addDisplayPage($"Route" ) 16 port pgStreet :DisplayPage G2H.Project.addDisplayPage($"Street" ) 17 port pgCustomer :DisplayPage G2H.Project.addDisplayPage($"Customer" ) ie port pgDelivered :Page G2H.Project.addPage ($"Delivered") 19 20 21 // 'db' • Database table 22 port dbRoute :lib->db=>Table := G2H.Project.addDatabaseTable($"Route"); 23 port dbStreets :lib~>db=>Table := G2H.Project.addDatabaseTable($"Streets"); 24 port dbCustomers :lib->db=>Table := G2H.Project.addDatabaseTable($"Customers")j 25 26 27 II 'fd' * Database field 28 port fdRoute_ID iField := dbRoute.addFieldf$"id"); 29 port fdRoute_StreetID :Field := dbRoute.addFieldf $"StreetID"); 30 port £dRoute_Order :Field := dbRoute.addFieldf$"Order"); 31 32 port £dStreets_ID :Field := dbStreets.addFieldf$"id"); 33 port fdStreets_Name :Field := dbStreets.addFieldf$"Street"); 34 35 port £dCustomers_ID :Field := dbCustomers.addField(S"id"); 36 port fdCustomers_Name :Field dbCustomers.addFieldf$"Name"); 37 port fdCustomers_Address :Field := dbCustomers.addFieldf$"Address"); 38 port fdCustomers_StreetID :Field :« dbCustomers.addField($"StreetID"); 39 port fdCustomers_PkgNumber :Field := dbCustomers.addFieldf$"PkgNumber"); 40 41 42 // 'lk' - Database link 43 port lkRoute_ToStreet :Link new LinkffdRoute_StreetID, £dStreets_ID); 44 port lkCustomer_ToStreet :Link new LinkffdCustomers_StreetID, £dStreets_ID); 45 46
G2HDesign's ports are common design elements anticipated to be used throughout the project witch includes elements such as pages (G2HDesign line #15-18), database tables
(#22-24), database fields (#28-40) and database links (#43-44). It is worthy to emphasize that these are ports and each element will be created only when the port is rendered (such as when the port is asked for its value). This means, for example, the database table
'Route' will only be added to the project (#22) when the port 'dbRoute' is used at least once in the project. Moreover, these elements are tentative, which means more can be added and some can be removed.
The team also agrees upon two kinds of pages to be used in the project: regular pages and display pages. Regular pages (page.pattern) are a PHP page (a PHPriie) with two additional properties: contentTitle (line Page# 11) and backgroundColor (Page# 12). Page patterns ensure that the body of each page is formatted by the project's PageFormater
195 IG2H1 ContwitTltw Content TttI* ContentTttla .. V" J"----J"-*'--J"J"J"J*-" .tafgnTabtoi; <1 baforaTaUf! 1 ( 1 • Body • Body 1 1 1 - - - - afJarTaWe! *K
P*gm DiaplayPaga Programmer A
o& V "is > *
uin •<••••.» ^'V* »*U»»*. . ' - 'tr • • »V**V f W—
ContentTltle
beforeTablet. "^STREET ^OMM
malnTabla afteiTable ASC CopyRight ©
(§) Link from Route to Street ©Link from Street to Customer ® Back to Route (D Back to Street (D Req_PrgmB_Distinguish_each_page_visually (#28-#33) O Task_Current_Selectedltem_for_reference (#29,#57) (3) DisplayPage (#25) © Req_PrgmA_Provide_intuitive_link_between_each_page (#17,#21) © Req_PrgmB_Display_with_consistency (#30) © Req_PrgmB_Display_with_corisistency (#47)
Figure 7.1: Page's components added by patterns
196 (page#25). The first picture on the top of the Figure 7.1 shows how a regular page may look. Another kind of page is Dispiaypage, which is a regular page with a DataTabie as a mainTable (DisplayPage#!\). Two ports of DisplayPage are assigned to the
DoBeforeTable and DoAfterTable of the table (DisplayPage#34-35). This allows additional information and/or execution before and after the table. The middle picture on the top of the Figure 7.1 shows how a DisplayPage might look. The following are code of both kind of page patterns.
g2H/Page.pattern l U 8Pattern: 2 package g2h; 3 4 import java.util.*; 5 6 import nawaman->text=>*; 7 import lib~>php=>* j 8 9 pattern Page extends PHPFile { 10 11 port contentTitle :Text; 12 port backgroudColor :Text; 13 14 15 protected prepareBody() :Text { 16 Text aBody = this.prepareBodyContent(); 17 aBody = this.applyG2HBodyStyle(aBody); 18 aBody = this.applyBodyStyle(aBody); 19 return aBody; 20 > 21 protected applyG2HBodyStyle(pBody :Text) :Text { 22 PageFormater aFormater = ((G2H)G2H.Project).formater; 23 24 if (aFormater != null) 25 pBody =
g2h/0isplayPage.pattern // iPattern: package g2h;
import java.util.*;
import nawaman->text=>*; import lib->phpdb=>*;
pattern DisplayPage extends Page implements PHPFile_withDataTable {
port mainTable :DataTabie;
port beforeTable :Text;
197 14 port afterTable :Text; 15 16 17 public new () { 18 super(); 19 this.assignMainTableAsBody(); 20 thi8.setMainTableProperties(); 21 > 22 23 24 private assignMainTableAsBodyf) :void { 25 body := <${<(mainTable.html)>}$>; 26 } 27 private setMainTableProperties() :void < 28 var aDBConn sDBConnection » ((G2HJG2H.Project).configuration.dbConnection; 29 30 require_once8 <+= aDBConn.includeFileName; 31 32 mainTable = new DataTable(); 33 mainTable.connection = aDBConn; 34 mainTable.DoBeforeTable = beforeTable; 35 mainTable.DoAfterTable - afterTable; 36 37 mainTable.file := this; 38 mainTable.file-»render (); 39 40 41
Additionally, a few service functions are added to the G2H pattern: addDisplayPagef...) (G2H# 38), addPage (...) (G2H# 41) and renderAHPages( ) (G2H# 60).
The first and the second functions enable pages to be added to the project so that they can
be rendered when the project is rendered (G2H#73). The function formats every page
using the formatter and renders them.
Finally, the team manager assigns the requirements to the team members that are. This is
recorded by the Manager pattern (Manager#13-16).
g2h/Manager.pattern II 8Pattern: package g2h;
import nawaman->text=>* ;
import g2h->Prgm_A-=>*; import g2h->Prgm_B=>*;
pattern Manager {
procedure assignTasks() tvoid { G2H aProject = G2H.Project; aProject.Reql_DisplayInformations := new Req_PrgmA_Display_only_the_information_needed();
198 14 aProject.Req2_Link_between_pages := new Req_PrgmA_Provide_intuitive_li.nk_between_each_page(); 15 aProject.Req3_DistinguishDisplay s» new ReqwPrgmB_Distinguish_each_page_visually(); 16 aProject.Req4_ConsistencyDisplay := new Req_PrgmB_Display_pages_with_consistency(); 17 } 18 19
From the above code, two requirements are assigned to Programmer A (Manager# 13-14) and another two requirements to Programmer B (Manager#15-16).
At this point, the team has four requirements assigned to its members ready to be implemented. Basic infrastructure for formatting each page is already established.
Database components foreseen by the team members are also defined (more can be added). The team can now begin the implementation. The following code should be executed to produce the final code.
G2H.project // ^Pattern:
import g2h=>*;
Manager.assignTasks(); G2H.Project.render();
7.4 The Implementation
As the project infrastructure is established, the team member can begin to implement the requirements.
7.4.1 Programmer A
Programmer A is responsible for two requirements, each of witch involves displayed information and links between each page.
The programmer creates a pattern 'prgmA' to hold some constants and a few utility
199 procedures. This is for centralizing purposes and perhaps for later reuse.
g2h/Prgm A/PrgmA.pattern 1 II GPattern: 2 package g2h->Prgm_A; 3 4 import nawaman->text=>*; 5 import g2h=>*; 6 import lib->phpdb=>*; 7 8 pattern PrgmA { 9 10 constant CSSFileName sText = $"prgm_a.css"; 11 constant ParamName_GoBack :Text - $"back"; 12 13 II 'ct' « ColumnTitle 14 constant ctStreetName :Text - $"Street"; 15 constant ctCustomerName :Text = $"Customer"; 16 constant ctCustomerAddress :Text = $"Address"; 17 constant ctCustomerPackage :Text = $"Package"; 18 19 20 procedure AddGoBack_fromPage_toPage( 21 pFromPage :DiaplayPage, 22 pToPage :DisplayPage) 23 :void { 24 pFromPage.addCSSInclude(PrgmA.CSSFileName); 25 pFromPage.afterTable := 26 <${ }$> 27
32 <${ >$>;33 > 34 35 procedure AddGoBackParameter_toLink( 36 pHyperLink :LinkTo_Page_o£_SelectedItem) 37 :void { 38 pHyperLink.paramExtra = 39 <${ }$> 40 <(PrgmA.ParamName_GoBack)>= 41 <${ >$>; 42 ) 43 >
Line #10 is the name of the Cascade Style Sheet (CSS) file containing the styles for
HTML elements created by the code from programmer A. The constants on the line #14-
17 are column titles. Two procedures on line #20 and 35, and the constant
'paramName GoBack' will be discussed later.
Requirement: Req_PrgmA_pisplay_only_the_in£ormation_needed
To implement this requirement, programmer A must decide what information is to be
200 displayed on each page. After some thinking, he decides that there are three criteria for selecting what is to be displayed on each page: 1) only necessary information for the delivery person to archive at each stage (each page), 2) only information that will lead to a delivery and 3) only information of selected items on each page (e.g., only customer names on a selected street).
Three tasks are, then, defined to implement the selection of each criteria as recorded by the pattern below.
g2h/Prgm A/Req PrgmA Display only the Information needed-pattern l // 6Pattern: 2 package g2h->Prgm_A; 3 4 import nawaman~>text=>*; 5 import g2h=>*; 6 7 8 pattern Req_PrgmA_Display_only_the_information_needed 9 implements Req_Display_only_the_information_needed { 10 11 port Taskl :Task_ShowOnly_NecessaryInformation_on_EachPage = new (); 12 port Task2 :Task_ShowOnly_TheUndelivered = new (); 13 port Task3 :Task_ShowOnly_SelectedRecords = new (); 14 15 16 public render)) :void { 17 Taskl?.render(); 18 Task2?.render(); 19 Task3?.render(); 20 > 21
Task: Task_ShowOnly_NecessaeyIn£ormation_on_EachPage This task specifies the information needed on each page. Programmer A creates the following pattern to accomplish the task.
g2h/Prgm A/Task ShowOnly Necessarytnformation^on_EachPage.pattern l II ^Pattern: 2 package g2h->Prgm_ft; 3 4 import nawaman~>text=>*; 5 import g2h=>*; 6 import lib~>db=>*; 7 import lib->phpdb=>*; e 9 10 pattern Task_ShowOnly_NecessaryInformation_on_EachPage { li
201 public render)) :void { this.showStreet_OrderedBy_RouteOrder(); this.showCustomer_OrderedBy_Address(); this.showPackage(); }
private showStreet_OrderedBy_RouteOrder() :void { G2HDesign aDesign » G2H.Project.design; DataTable aRouteTable = aDesign.pgRoute.mainTable, Field aStreetName - aDesign.fdStreets_Name;
ShowFieldColumn(aRouteTable, PrgmA.ctStreetName, aStreetName);
aRouteTable.query.orderby = new OrderBy(aDesign.fdRoute_Order); aRouteTable.query.links <+= aDesign.lkRoute_ToStreet;
> private showCustomer_OrderedBy_Address() :void { G2HDesign aDesign « G2H.Project.design; DataTable aStreetTable = aDesign.pgStreet.mainTable; Field aCustomerName = aDesign. £dCustomers_Naine ; Field aCustomerAddress = aDesign.fdCustomers_Address;
ShowFieldColumn(aStreetTable, PrgmA.ctCustomerAddress, aCustomerAddress); ShowFieldColumnfaStreetTable, PrgmA.ctCustomerName, aCustomerName);
aStreetTable.query.orderby - new OrderBy(aDesign.fdCustomers_Address); > private showPackage() tvoid { G2HDesign aDesign = G2H.Project.design; DataTable aCustomerTable » aDesign.pgCustomer.mainTable; Field aCustomerPackage = aDesign.fdCustomers_PkgNumber;
ShowFieldColumnfaCustomerTable, PrgmA.ctCustomerPackage, aCustomerPackage)j >
procedure ShowFieldColumnf pTable :DataTable, pTitle :Text, pField :Field) :void { ColumnField aShowField = new(); aShowField.table = pTable; aShowField.title = <${<(pTitle)>}$>; aShowField.field = pField; aShowField.render)); )
When the pattern above is rendered, three of its actions are executed (#13-15). The code of each action is self-documented, for example, the 'showcustomer_orderedBy_Address(/ action ensures Customer's information is shown ordered by the customer address. When looking further in the body of the action, Customer's address and name are shown as a column in the streetTabie (#35 and #36). The line #38 specifies that streetrabie should be ordered by the customer's address.
202 Task: Task_ShowOaly_TheUndelivered This task adds the code for filtering only the undelivered records. For example, in the first page, only streets that have customers with undelivered packages are shown. This is done by simply adding a condition to the Query code for all records that the package number is not null (#38-42). The code below is the pattern of this task.
g2h/Prgm A/Task ShowOnly TheUndelivered.pattern // ^Pattern: package g2h->Prgm_A;
import nawaman->text=>*; import g2h=>*; import lib->db«>*; import lib->phpdb«>*;
pattern Task_ShowOnly_TheUndelivered {
public render() :void { this.showOnly_Streets_withUndeliveredPackages(); this.showOnly_Customer_withUndeliveredPackages (); }
private showOnly_Streets_withUndeliveredPackages() :void { G2HDesign aDesign » G2H.Project.design; this.selectOnly_UndeliveredPackages( aDesign.pgRoute.raainTable, aDesign.XkCustomer_ToStreet ); } private showOnly_Customer_withUndeliveredPackages() :void { G2HDesign aDesign - G2H.Project.design; this.selectOnly_UndeliveredPackages( aDesign.pgStreet.mainTable, null )f >
private selectOnly_UndeliveredPackages( pTable :DataTable, pLink :Link) :void { G2HDesign aDesign - G2H.Project.design; Query aQuery = pTable.query; aQuery.conditions <+= new Condition( aDes ign.fdCustomers_PkgNumber, $»<>", $"'NULL'" ); if (pLink != null) aQuery.links <+= pLink;
The pattern above ensures that only streets and customers with undelivered packages are shown (actions in #13-14). The function 'selectOnlyjJndeliveredPackages (#32) is
203 reused in both actions. The function adds a condition of 'package number is not null' to the query. On the line #19-#22, for example, the selection is applied to the'mute' page to select only streets with undelivered packages.
Task: Task_ShowOnly_SelectedRecords This task adds code that filters only selected records. For example, it ensures that the street page shows only customers on that selected street. This involves a link to select a record and a code to filter the selected records.
Since this is a very common feature found in most web-application, the team already has a pattern to represent this intention. The pattern is called 'LinkTo_Page_of_seiecteditem can be found in the package 'iib~>phpdb' ("PHPDB Patterns" in the Appendix B on 282).
The pattern makes the column a link that passes on the value of the source field as a parameter. The parameter is then used for filtering in records of the target field to be shown in the target page (LinkTo_page_of_seiecteditem #30-47, page 288).
g2h/Prgm A/Task ShowOnly SelectedRecords.pattern 1 // 6Pattern: 2 package g2h~>Prgm_A; 3 4 import nawaman->text»>*; 5 import g2h*=>* ; 6 import lib->phpdb=>*; 7 8 pattern Task_ShowOnly_SelectedRecords { 9 10 port Taskl :Task_Current SelectedItem_for reference « new(); 11 12 13 public render() :void { 14 Taskl?.render(); 15 16 this.showOnly_Customer_on_SelectedStreet(); 17 this.showOnly Information_of SelectedCustomer(); 18 19 20 21 private showOnly_Customer_on_SelectedStreet() :void { 22 G2HDesign aDesign » G2H.Project.design; 23 DataTable aRouteTable = aDesign.pgRoute.mainTable; 24 25 LinkTo_Page_of_SelectedItem aHL = new (); 26 aHL.column = aRouteTable.addColumn(PrgmA.ctStreetName);
204 27 aHL.target Page <= aDesign.pgStreet; 28 aHL.sourceField - aDesign.£dStreets_ID; 29 aHL.targetField « aDesign.fdCustomers_StreetID; 30 31 PrgmA.AddGoBackParameter_toLink(aHL); 32 33 aHL.render(); 34 > 35 private showOnly_Information_of_SelectedCustomer() :void { 36 G2HDesign aDesign = G2H.Project.design; 37 DataTable aStreetPage « aDesign.pgStreet.mainTable; 38 39 LinkTo_Page_of_SelectedItem aHL = new (); 40 aHL.column « aStreetPage.addColumn(PrgmA.ctCustomerName); 41 aHL.targetPage = aDesign.pgCustomer; 42 aHL.sourceField - aDesign.£dCustomers_ID; 43 aHL.targetField - aDesign.fdCustomers_ID; 44 45 PrgmA.AddGoBackParameter_toLink(aHL); 46 47 aHL.render(); 48 49 50
On lines 25-29, necessary parameters are given to an instance of the pattern
fLinkTo_Page_ofseiecteditem to show only customers on a selected page. As mentioned
in the previous paragraph, the value of the street name column (of the Route table (#26))
will be made a hyper-link to the target page (the street table (#27)). The link will be added in the way to match Street's ID (#28) and Customer's streetiD field (#29). Finally, a necessary parameter is added as the hyper-link extra-parameter ('paramExtr in
PrgmA#38) via the procedure 'PrgmA.AddGoBackParameter_toLink(. . . on line #31.
At this point, programmer A has come to realize that he needs another feature - the way
to tell the user what the current selected item is (e.g., What the current street name is).
The programmer, then, create a task 'Task_Current_SelectedItem_for_reference to add
this new feature. Since the new feature depends on the feature added by the task
Task_showoniy_seiectedRecord, he adds this new task as a sub-task (#10), which will be rendered when 'Task_showoniy_seiectedRecords' is rendered (#14).
205 i/Prgm A/Task Current Selectedltem for reference.pattern 1 // ((Pattern: 2 package g2h~>Prgm_A; 3 4 import nawaman~>text=>*; 5 import g2h=>*; 6 import lib->db->*; 7 import lib->phpdb=>*; 8 9 10 pattern Task_Current_SelectedItem_for_reference { 11 12 public render)) :void { 13 this.showCurrent_Street(); 14 this.showCurrent_Customer(); 15 } 16 17 18 private showCurrent_Street() :void { 19 G2HDesign aDesign ~ G2H.Project.design; 20 DisplayPage aStreetPage - aDesign.pgStreet; 21 DataTable aStreetTable = aDesign.pgStreet.mainTable; 22 23 Text aCurrent_StreetName = GetFieldValue( 24 aStreetTable, 25 aDesign.fdStreets_Name, 26 aDes ign.lkCustomer_ToStreet 27 ); 28 29 aStreetPage.beforeTable 30 <$< }$> 31
206 71 if (pLink 1= null) pTable.query.links <+= pLink; 72 return <${['<(pField.alias)>'); ?>>$>; 73 74
From the pattern code above, the current item's information is shown as 'beforeTabie property (#29 and #57). For the street page, the current street name is shown (#32); for the customer page, the current customer's name, address and street name are shown as current item's information (#60).
Requirement: Req_PrgmA_Provide_intuitive_link_between_each_page
To implement this requirement, code to enable smooth navigation between pages must be added. Fortunately, links from Route page to street page and to customer page already exist (added by task_showoniy_seiectedRecords above). Therefore, what need to be added are more links for navigating back from customer to street and to Route page. As such, programmer A creates the following pattern.
g2h/Prgm A/Req PrgmA Provide intuitive Jink between each page.pattem 1 // ((Pattern: 2 package g2h->Prgm_A; 3 4 import nawaman->text=>*; 5 import g2h=>*; 6 7 pattern Req_PrgmA_Provide_intuitive_link_between_each_page 8 implements Req_Provide_intuitive_link_between_each_page { 9 10 public render() :void { 11 this.addGoBack_from_Street_to_Route_Page(); 12 this.addGoBack_£rom_Custon>er_to_Street_Page(); 13 } 14 15 private addGoBack_from_Street_to_Route_Page() :void { 16 var aDesign :G2HDesign » G2B.Project.design; 17 Pr gmA.AddGoBack_fromPage_toPage(aDesign.pgStreet, aDesign.pgRoute); 18 19 private addGoBack_£rom_Customer_to_Street_Page() :void { 20 var aDesign :G2HDesign » G2H.Project.design; 21 PrgmA.AddGoBack_fromPage_toPage(aDesign.pgCustomer, aDesign.pgStreet); 22 } 23
This pattern has two actions. One action adds a link from street to Route page (#17), while another adds a link from customer to street page (#21).
207 Programmer A may later refactor the code to ensure that the forward links are created
when this requirement is rendered. However, he considers that the forward links are
essential to the first requirement, which, in turn, is a very fundamental requirement for
the project. Therefore, it is very likely that the forward links are always been created. As
the result, he keeps the current code and begins testing.
7.4.2 Programmer B
At the same time programmer A is coding, programmer B is also implementing her
requirements. She is responsible to two requirements involving the display of the pages.
She starts by creating a pattern that contains (as anticipates) often-used constants and
procedure.
g2h/Prgm B/PrgmB.pattern 1 // SPattern: 2 package g2h->Prgm_B; 3 4 import nawaman~>text=>*; 5 import lib~>html=>*; 6 7 pattern PrgmB { 8 9 constant CSSFileName :Text = $"prgm_b.css"; 10 constant MainTableStyler :TableStyler = PrgmB.NewMainTableStyler(); 11 12 II 'pg' - Page 13 constant pgColor_Route :Text = $"#99CCFF" 14 constant pgColor_Street :Text = $"#FFFFCC" 15 constant pgColor_Customer :Text = $"#D7F4D7" 16 constant pgColor_Delivered :Text = $"#EB613D" 17 18 II 'pg' « Page 19 constant pgContentTitle_Route :Text = $"Route"; 20 constant pgContentTitle_Street :Text = $"Street"; 21 constant pgContentTitle_Customer :Text = $"Customer" 22 23 24 procedure NewMainTableStyler() :TableStyler { 25 TableStyler aTS = new (); 26 aTS.table = new Styler_Tag($~table~, <${class="mainTable"}$> ); 27 aTS.title_row = new Styler_Tag($~tr~, <$
208 From the code, the pattern has nine constants and one procedure. The constant
'cssFiieName holds the value of a cascade styler sheet file (use for styling web-page elements) (prgmB#9). MainTabiestyier is a styler for the table that, when used, assigns style classes to table, title row and value row of the styled table (#25-29). The procedure helps to create the default value for the constant MainTabiestyier. The rest of the constants hold pages' background colours and content titles (#13-21).
Requirement: Req_PrgmB_Distinguish_each_page_visually
Programmer B starts with the ,Req_PrgmB_Dispiay_pages_wtih_consistency pattern. The pattern should ensure that all the pages look similar unless there is different information to be displayed (which is governed by another requirement). To do that, the programmer implements PageFormater and assigns it as the formatter of the project (See line #8 and
#11). The function 'formatpagef...)' (#15) assigns properties of the given page. In this case, the 'id' and the page background colour are given as attributes of the page's HTML body (#55 and #58 respectively). If the given page is a Dispiaypage, the page's mainTabie will be given the Tabiestyier (prepared in prgmB) (#61-62). All pages are given the CSS include (line 65).
g2h/Prgm B/Req PrgmB Display pages with consistency.pattern 1 // ?Pattern: 2 package g2h~>Prgm_B; 3 4 import nawaman->text=>*; 5 import g2h=>*; 6 7 pattern Req_PrgmB_Display_pages_with_consistency 8 implements Req_Display_pages_with_consistency, PageFormater { 9 10 public render)) svoid { 11 G2H.Project.formater := this; 12 ) 13 14 15 public formatPage( 16 pPage :Page) 17 :Text { 18 this.setBodylD (pPage);
209 19 this.setBodyBackgroundColor(pPage); 20 this.setMainTableStyler (pPage); 21 this.addCSSFile (pPage); 22 > 23 24 public formatBody( 25 pPage :Page, 26 pBody :Text) 27 :Text { 26 return 29 <$< >$> 30
34 35
40 <(8:addTabs(pBody,1))> 41
42
43
44
47
On the other hand, the page body is formatted by the function 'formatBody(/ (#24). This function adds G2H's logo at the top-left corner of the page (#31), while showing the copy-right text on the bottom-right corner (#48). Finally, the function aligns the contentTitle and the actual content body (pBody) of the page at the centre (#35-44). The right picture on the top of the Figure 7.1 shows how pages formatted by this requirements may look.
210 Requirement: Req_PrgmB_Distinguish_each_page_visually
Next, programmer B implements ,Req_prgmB_Distinguish_each_page_visuaiiy' which (as the name imply) is the requirement that differs each page visually so that the driver can easily notice the transition from one page to another. Upon confirming with others that the project still has four pages, programmer B writes the following pattern.
g2h/Prgm B/Req PrgmB Distinguish each page yisually.pattern l // ^Pattern: 2 package g2h->Prgm_B; 3 4 import nawaman->text->*; 5 import g2h=>*; 6 7 pattern Req_PrgmB_Distinguish_each_page_visually 8 implements Req_Distinguish_each_page_visually { 9 10 public render!) :void { 11 this.setDifferentBackgroundColors(); 12 this.showInformationTitle_of_EachPage(); 13 } 14 15 private setDifferentBackgroundColors() :void { 16 G2HDesign aDesign = G2H.Project.design; 17 ~:after(aDesign.pgRoute): 18 aDesign.pgRoute .backgroudColor := PrgmB.pgColor_Route; 19 -:after(aDesign.pgStreet): 20 aDesign.pgStreet .backgroudColor := PrgmB.pgColor_Street; 21 after(aDesign.pgCustomer): 22 aDesign.pgCustomer .backgroudColor := PrgmB.pgColor_Customer; 23 -:af ter(aDes ign.pgDe1ivered): 24 aDesign.pgDelivered.backgroudColor := PrgmB.pgColor_Delivered; 25 } 26 private showInformationTitle_of_EachPage() :void < 27 G2HDesign aDesign = G2H.Project.design; 28 after(aDesign.pgRoute)t 29 aDesign.pgRoute .contentTitle := PrgmB.pgContentTitle_Route; 30 -:after(aDesign.pgStreet): 31 aDesign.pgStreet .contentTitle := PrgmB.pgContentTitle_Street; 32 after(aDesign.pgCustomer)s 33 aDes ign.pgCustomer.contentTitle := PrgmB.pgContentTitle_Customer; 34 ) 35 36
The pattern has two actions. The action 'setDifferentBackgroundcoiorsf)' sets each page to a different colour and the action 'showinformationTitie_of_EachPage()' sets each page with a different contentTitle.
7.5 The Final Product
Now that all four fundamental requirements are implemented, the team can begin
211 * V *<* % / *«wr«i*jrA*CW2**. f *3 * V * V % 1 S <• A # % i^v i r #' "'A * ' »*<£ **" ** -•••"'
Dona ' % fi 0:3 f* 0
H mi yyi»ffl^| H £http-V/toca*>ost/Q2h/ttrotphp?5t vwm~ — ii ID STREET OimR •MKT9IWT8 Powarad ly ASC (c) 2009. Don* % fi 0:3 4-' OS nn iD CUSTOMER Mr. On# -111 On* Rd. Packag* 56465 ucKTOtmnrr Pomrad By ASC (c) 2000 Done % 3 0:3 +• e Figure 7.2: G2H's Delivering System 212 generating the final product software. As mentioned, this is done by executing g2h/G2H.pattern (on page 192). The execution generates three files: route.php, street .php and customer.php. The code of all three files can be found in "Appendix C.IV Result Programs" on page 311. Figure 7.2 shows the resulting program in during the execution. To use the program, the driver must browse to http://iocaihost/g2h/route.php. The page displays a list of streets in order. The driver can click any street (although he should click the top one). The second page shows a list of customers on the selected street. Then, the driver can select a customer name, which will open the third page that shows the package to be delivered to that customer. Further in the development, the package number should be click-able; when clicked, a package-delivered confirmation page is shown. 7.6 Discussion The ultimate goal of the 1GPM is to improve software process agility. To achieve this, patterns are created to naturally document and generate the software code. In the case- study, patterns are used (in a useful way) for both documentation and code generation. 7.6.1 Patterns for Documentation Every pattern documents something. In the case study, patterns are used to naturally document many aspects of the software. They are used to document requirements, task decomposition and assignment, design, code manipulation and so on. 213 Requirements Four requirements are presented in the case study. For example, the pattern Req_Provide_intuitive_link_between_each_page documents that intuitive links between each page must exist. Task Decomposition and Assignment The declaration of requirements is, in a way, a decomposition of task since it can be assigned to team members to implement. The Manager pattern (see code in page 198 and the Figure 7.3) assign instances of the patterns that implement (or realize) each requirement to the project (Manager# 13-17). The assignment also implies the assignment of the actual tasks as the patterns must be implemented. For example, the port Req4 of the project is assigned by Req_PrgmB_Display_pages_with_consistency (page 209), witch means that ,Req_prgmB_Dispiay_pages_with_consistency must be created to implement the requirement 'Req_Display_pages_with_consistency . Additionally, three patterns are explicitly declared to represent three tasks programmer A will do to satisfy the requirement Req_Display_only_the_information_needed. Like a real task, the programmer can implement each task the way he sees fit or even passes some of them to others to implement. Design Many patterns document design. The best example is the pattern G2HDesign (page 194) which specifies four pages, three database tables, ten database fields and two database links. Another example is the three task patterns of the pattern Req_PrgmA_Display_only_the_information_rieeded (page 201) specifying that there are 214 Reql :Req_Display_only_theJnformation_needed A . -Req_PrgmA_Display_only_the_information_needed •Task_ShowOnly_Necessarylnformation_on_EachPage kTask_ShowOnly_SelectedRecords ~~~~~""*-Task_Current_Selectedltem_for_refernce kTask_ShowOnly_TheUndelivered Req2 :Req_Provide_irituition_link_between_each_page A . -Req_PrgmA_Provide_intuitiveJink_between_each_page Req3 :Req_Distinguish_each_page_visually A j :--4--Req_PrgmB_Distinguish_each_page_visually Req4 :Req_Display_pages_with_consistency A | :--4--Req_PrgmB_Display_pages_with_consistency Manager Figure 7.3: G2Hproject's requirements and tasks 215 three criteria of selecting what data must be displayed. Some patterns document design in a less obvious way. For example, the DispiayPage pattern, which documents that there is a main table on a display page to display some information. Code Manipulation Some patterns manipulate the final code directly so the patterns' name (or its action) documents those manipulations. Req_prgmB_pispiay_pages_with_consistency pattern, for example, adds a piece of code to show G2H location (page 209). Another example is the prgmA .AddGoBackParameter_toLink that creates a PHP code that obtains the full request URL and uses it as a parameter (page 200). It is also possible to track upward to see what pattern is responsible for a piece of code. For example, one of the resulting file, street.php, is composed of 363 pieces of text. (To see that, the command 'g:println(G2H .Project .design.pgStreet. Content .toDetail()) should be executed after the project is rendered). The intentional-tracking result can be found in Appendix C.V. This allows a developer who is not familiar with pattern language to study this tracking and subsequently try to understand it. 7.6.2 Code Generation Changes can be quickly applied because the IGPM allows code to be broken down into many independent parts; hence, a change can be concentrated in a very few locations. The effect of modifying a pattern to the end software can be applied by simply executing the pattern network. The pattern network acts as an integration framework by putting effects of each patterns (the decomposed tasks) together to form the final software. 216 In addition, changes can be quickly applied because the IGPM enables a great degree of reuse; hence, the modification can make use of many already existing solutions without rewriting it. Task Decomposition "Dependency is one of the most critical problems in software development" [152], Patterns allow tasks to be decoupled to a higher degree because it does not depend on the target programming language. This enables the developers to work separately and perhaps concurrently on different parts of the software with little or no dependency. For example, only one pattern is responsible for specifying what data to be display on each page. Only one pattern deals with the selection of current items (current street or customer). Only one pattern is responsible for formatting the page. Only one pattern is responsible for creating a go-back link. It is also noteworthy that none of the patterns created within this project deal with the HTML table directly. The patterns that handle this are iib~>htmi=>Tabie (page 276) and Hb->htmi=>Tabiesytier (page 209) which are part of the infrastructure patterns. Task composition enabled by patterns occurs in such great degree that other parts of the project does not need to worry about HTML table at all even though HTML table appears to be very-tightly integrated with other parts of code in every display page (see the code of each page in 311). Being less-coupling, the modifications can be quickly identified (when a change occur) and can be applied in a smaller number of locations within the existing code. Additionally, the effect of the modification can be easily determined [153], 217 Reusability Reuse can be found everywhere in this project. For example, consider the pattern Task_ShowOnly_NecessaryInformatiori_on_EachPage (page 201); it takes a line to add a column to show a field value (e.g., line #36 to show Customer name in the street table). This can be done because the rest of the operation is performed by other patterns or actions. First, the action 'showFieidcoiumn (#49-59) makes it possible to show the field column in one line. Second, coiumnFieid pattern (page 282) ensures a column is added (if not already) and sets the value of the column to be a PHP code that accesses to the field value of this table (coiumnFieid #23 and #24). The iib~>php=>Datarabie (page 285) pattern inserts the value code in the table and ensures that they are appropriately iterated. The Query patterns (page 261) ensures that the field will be a part of the SQL command (so that the field's value can be obtained). A similar trend can be seen throughout the project. For example, the pattern 1LinkTo_Page_of_SelectedItem (page 288) is reused in Task_ShowOnly_SelectedRecords to show a selected street and a selected customer. Programmer A (who programs the Task_showoniy_seiectedRecords pattern) only has to specify a few properties of the LinkToj?age_of_seiecteditem and calls its render function. To conclude, a pattern naturally documents parts of the software it handles, which allows a developer to understand the code (without much effort on synchronize code and documentation). A pattern can be created to handle one independent task, which minimizes the modification when changes occur. A pattern (if properly designed) can be 218 reused, which increases development efficiency. All of the above, increase project agility. 7.7 Change 1: Add Customer's Phone Number This section aims to demonstrate how the team handle with a small change. 7.7.1 Problem The delivery system is implemented and is given to a driver for testing. After a few days, the driver informs the team that the system should show the customer's phone number. The phone number will be used to contact the customer when no one answers the door. As it was done in the past, the driver could phone the customer and ask if the package can be left at the door or with a neighbour. 7.7.2 Response After considering the driver's feedback, the project manager passes the feedback on to programmer A as he is responsible for displaying information. Programmer A examines the request and finds it to be an easy one to implement. Firstly, he requests to add a field named 'phonemmber' to the Customer table. The manager agrees and add a line that define the field into G2HDesign .patttern. g2h/G2HDesign.pattern (modified) 34 35 port fdCuatomers_StreetID :Field := dbCustomers.addField($"StreetID"); 36 port fdCustomers_PkgNumber :Field := dbCustomers.addField($"PkgNumber"); 37 port fdCustomers_PhoneNumber :Field ; = dbCustomers. addField($ " PhoneNumber "); 38 39 40 // 'lk' = Database link 41 port lkRoute ToStreet :Link := new Link(fdRoute StreetID, fdStreets ID); 42 Then, he modifies the action 'showcurrent_customer()' to the task 219 'Task_CurrentJSelectedItem_for_reference' to show the phone number after the customer name. The bold parts of the following code are the modifications. g2h/Prgm A/Task Current Selectedltem for reference (modified) } private showCurrent_Custoraer() :void { G2HDesign oDesign = G2H.Project.design; DisplayPage aCustomerPage = aDesign.pgCustomer; DataTable aCustomerTable « aDesign.pgCustomer.mainTable; Text crCustomerName = GetFieldValue( aCustomerTable, aDesign.fdCustomers_Name, null ) f Text crCustomerAddress - GetFieldValue( aCustomerTable, aDesign.fdCustomers_Address, null >; Text crCustomerStreetName = GetFieldValue! aCustomerTable, aDesign.fdStreets_Name, aDesign.lkCustomer_ToStreet ) ? Text crCustomerPhoneMuaber = GetFieldValue( aCustomerTable, aDesign.fdCustomers_PhoneNunber, null )} aCustomerPage.beforeTable := <${ j$>
<(crCustomerAddress)> <(crCustomerStreetName)>
<(crCustomerPhoneNumber)>
Now, the resulting program can be generated and tested. The customer page now shows
three lines of customer information: 1) customer's name, 2) address number and street
and 3) the phone number.
7.7.3 Discussion
The problem with small changes is how the developer knows where to modify. In this
example, the programmer should quickly locate the position to apply the modification;
220 since the phone number that should be shown is the phone number of the selected
customer, the programmer knows he should start looking in the pattern
'Task_current_seiecteditem_for_reference. In that pattern, the programmer can easily
spot the function (or action) named 'showcurrentj:ustomer((#36); In side the action, he
can easily spot a group of code that assign the field values and the field value is used in
the code template just below.
With patterns, it is possible to easily connect the necessary dots leading to where the
modification should be applied. It is also possible to limit the modification to a small
area.
7.8 Change 2: Revert Address Numbers
This section aims to demonstrate how the developer deals with a medium-sized change.
7.8.1 Problem
Soon after the customer phone's number is added, the test driver identifies another
possible improvement. Sometimes, the delivery truck arrives at a different end of a street,
but the address numbers shown in the Street page (the second page) are always ordered
from lower number to higher number. When that happens, the driver needs to click from
the bottom upward to the top of the address table. It would be better if there is a way to
revert the ordering.
This task is, again, passed on to programmer A. He analyzes the problem and finds that
there are two things must be done to realize this; he needs a way to change the a query
order based on a page's parameter; He also needs a way for the driver to specified if the
221 query order should be reverted.
7.8.2 Response
The programmer starts by creating a new task to encompass this requirement. The task is called 'Task_Make_customerAddress_Reversibie. The task is, then, added as part of a requirement 1Req_PrgmA_Display_only_the_information_needed
The following code shows the modified requirement.
g2h/Prgm A/Req PrgmA Display only the information needed.pattern (modified) l // 8Pattern: 2 package g2h->Prgm_A; 3 4 import nawaman->text=>*; 5 import g2h=>*; 6 7 8 pattern Req_PrgmA_Display_only_the_information_needed 9 implements Req_Display_only_the_information_needed { 10 11 port Taskl iTask_ShowOnly_NecessaryInformation_on_EachPage = new (); 12 port Task2 :Task_ShowOnly_TheUndelivered = new (); 13 port Task3 :Task_ShowOnly__SelectedRecords = new (); 14 port Task4 :Task_Make_CustomerAddress_Reversible - new (); 15 16 17 public render() :void 18 Taskl?.render(); 19 Task2?.render(); 20 Task3?.render(); 21 Task4 ?.render(); 22 > 23
Programmer A, then, creates a pattern to change the order based on a condition. The following is the pattern's code.
g2h/Prgm A/OrderBy witHCondition.pattern 1 // 8Pattern: 2 package g2h~>Prgm_A; 3 4 import nawaman->text=>*; 5 import lib->db=>*; 6 7 pattern OrderBy_withCondition extends OrderBy { 8 9 port condition :Text; 10 11 public new () {
222 12 super(); 13 } 14 public new (pOrderBy :Text) { 15 super(pOrderBy); 16 } 17 public new (pOrderBy :Textr pCondition :Text) { 18 this(pOrderBy); 19 condition - pCondition; 20 > 21 22 protected prepareSQL() :Text { 23 if (condition " null) 24 return super.prepareSQL(); 25 26 return 27
The inner workings of this pattern will not be discussed as it is not within the scope of this thesis, but the code should be easy to understand to those who are familiar with SQL language.
Next, the programmer creates a pattern that enables the driver to change the ordering.
This pattern is called 'orderReversibie_coiumn and the following is its code.
g2h/Prgm A/OrderReversible Column.pattern // $ Pattern: package g2h->Prgm_A;
import nawaman->text=>*; import g2h=>*; import lib->db=>*; import lib~>html=>*; import lib->phpdb=>*;
pattern OrderReversible_Column { port page :DisplayPage; port column :Column; port revertParamName jText = "revert"; port revertText :Text = "REVERT";
public new() { this.addQueryString_UtilityIncludeFile(); this.addRevertLink_toTheColumn_Title(); this.setOrderBy_withCondition(); }
private addQueryString_UtilityIncludeFile() :void { ~:after(page): page.require_onces <+= $"QueryStr.inc"; )
223 27 private addRevertLink_toTheColumn_Title() :void { 28 after(column): 29 column.titleStyler » this.createStyler_ThatAddRevertLink(); 30 } 31 32 private setOrderBy_withCondition():void { 33 after(page)s { 34 Text aConditionByParam = this.getConditionByParam(); 35 page.mainTable.query.orderby new() of OrderBy_withCondition; 36 -:after(page.mainTable.query.orderby):{ 37 OrderBy_withCondition aOrderBy = (OrderBy_withCondition)page.mainTable.query.orderby; 38 aOrderBy.condition = <${<(aConditionByParam)>}$>; 39 ) 40 ) 41 } 42 43 private createStyler_ThatAddRevertLink() :Styler { 44 Text aRevertValue «
Again, how this pattern works is not in the interest of this project. Roughly, the pattern
creates a styler that adds a hyper-link used to swap 'revert' parameter value from 'true' to
'false and back (#43-52). The styler is then assigned to the title of a given column (#29).
Finally, the parameter value is used to change the order of given table's query (#32-#41).
Now that both necessary patterns are implemented, the programmer implements the task
'Task Make CustomerAddress Reversible'.
g2hPrgm A/Task Make CustomerAddress Reversible 1 // ^Pattern: 2 package g2h~>Prgm_A; 3 4 import nawaman->text=>*; 5 import g2h=>*; 6 import lib~>db=>*; 7 import lib->phpdb=>*; 8 9 pattern Task_Make_CustomerAddress_Reversible { 10
224 11 public render() :void { 12 G2HDesign aDesign « G2H.Project.design; 13 DisplayPage aStreetPage « aDesign.pgStreet; 14 DataTable aStreetTable « aStreetPage.mainTable; 15 16 OrderReversible_Column aORColumn = new(); 17 aORColumn.page := aStreetPage; 18 aORColumn.column :•= aStreetTable.addColumn (PrgmA. ctCustomerAddress); 19 aORColumn.render)); 20 21 22
This task pattern is very simple. An orderReversibie_coiumn pattern is constructed (#16).
The page where the column appears is given (#17) and the column to be made reversible is assigned (#18). Then, the reversible column is rendered (#19).
Last but not least, the programmer modifies a small piece of existing code that dictates the ordering of the address name and makes it compatible with the added task. The following shows the modification.
g2h/Prgm _A/Task .ShowOnly_Necessarylnformatlon„on^EachPage (modified)7 27 23 } 24 private showCustomer_OrderedBy_Address() :void { 25 G2HDesign aDesign - G2H.Project.design; 26 DataTable aStreetTable - aDesign.pgStreet.mainTable; 27 Field aCustomerName = aDesign.fdCustomers_Name; 28 Field aCustomerAddress « aDesign.£dCustomers_Address; 29 30 ShowFieldColumnfaStreetTable, PrgmA.ctCustomerAddress, aCustomerAddress); 31 ShowFieldColumnfaStreetTable, PrgmA.ctCustomerName, aCustomerName); 32 33 aStreetTable.query.orderby := new() of OrderBy; 34 -s after(aStreetTable.query.orderby): 35 aStreetTable.query.orderby.orderby = aDesign.fdCustoners_Address; 36 } 37 private showPackage() :void { 38
Previously, line #33 dictated that the 'orderby property of the streetrabie's query must be a new orderBy pattern. This must be modified because orderReversibie_coiumn needs the property 'orderby to be an orderBy_withcondition pattern (see line #35 of orderReversibie_coiumn). As in the line #33-35 above, the property is given a new-
7 See the code before the modification on page 201
225 instance-assignment action to ensure that the value is an orderBy pattern. Finally, the
Customer's address field is given only after the property is rendered (#35).
7.8.3 Discussion
The larger, more complex changes require larger and more complex code modifications.
The patterns keep that process make sense and more manageable.
For this example, the modifications to the existing code are minimal. Most of the work is
done to create additional patterns. This makes the modification more manageable.
The modification is also easily reversible. First, the additional line in
Req_PrgmA_Display_only_the_information_needed can be deleted without any
dependency. It can also be temporarily deactivated by commenting the 'newo' part of the
added code (so the task as no implementation). The modification in the
Task_ShowOnly_Necessarylnformation_on_EachPage does not require change to be
reversible (it can be left as it without any effect).
For the newly created pattern, the program creates a new task
'Task_Make_customerAddress_Reversibie because what he wants to do is to make the
customer address column to be reversible. Then, he creates two patterns:
OrderBy_vithConection and OrderReversible_Column, OrderBy_withConection allows the
driver to revert the ordering of a query based a run-time parameter.
orderReversibie_coiumn creates necessary links and necessary computations to allow
column ordering to be reversible with a click.
226 The task Task_Make_customerAddress_Reversibie has a very specific use (to make the customer address column reversible). While the creation of these two patterns performs very general development activities, which can be reused later on. This means that once created these two patterns can be reused to allow any column to be reversible. Imagine, if someone already created these two patterns, the modifications that programmer A must perform to handle to the change will be very small and manageable.
7.9 Conclusion
This chapter shows that the intentional generative pattern mechanism (the IGPM) can be used in a real-world project. It documents software in many different aspects other techniques cannot do without additional effort. It helps a development team separate the task and decouple the components of the system. It also enables a higher degree of reusability. The mechanism also allows the team to cope with changes better with less effort and in a much more manageable way.
227 CHAPTER 8
CONCLUSIONS AND FUTURE WORK
This chapter concludes the thesis with a summary of the contributions of the research and proposes several topics that should be considered by future work.
8.1 Conclusion
This research investigates the software process agility problem and proposes a method called the intentional generative pattern mechanism (IGPM) to improve the responsiveness to change (process agility).
Background Software Process Agility is one of the most challenging topics in software engineering today. With today's fast-paced and ever-changing business demands, plus the growing complexity of software, changes during development are inevitable.
Normally, agile approaches are used to improve process agility. These approaches prefer minimalism. Upfront effort and documentation should be performed at a minimum.
When changes occur, work already finished may become useless. The reduction of up front effort should reduce the wasting of effort by producing only what is immediately necessary. Using less documentation should enable less-effort modification to software since there is no need for the synchronization between the code and the documents.
228 However, many researchers argue that these approaches do not work because it results in other development and management problems, such as quality control and progress tracking. Also, agile processes can only be used in small projects. In other words, agile processes are not adoptable. The above arguments and other similar issues indicate that agile approaches may not be the ideal solutions to achieve true agility.
Further Investigation In the search for solutions, this research revisits the problem by investigating the source of the problem - the sources of changes. As a result, eight sources of changes are identified which are: 1) business-situation changes, 2) business-need changes,
3) requirement changes, 4) technological changes, 5) management changes, 6) high-level solutions and architectural changes, 7) implementation changes and (8) test-result changes.
Possible Solutions and Existing Realizations Six common solutions of "Problem Solving" techniques (i.e., prevention, elimination, reduction, treat, toleration and redirection) are, then, applied to the context of the process agility problem. The results are a set of characteristics that a true agile process should have. These characteristics are: 1) iterative frameworks that allows feedback, 2) the utilization of risk management, 3) the appropriate level of stability and up-front effort,
4) the management of changes, 5) effective means of communication and 6) document the software easily and thoroughly.
Agile Documentation A number of techniques, from many existing processes (both agile and traditional), are evaluated. Except for one, there are suitable realizations of all of those characteristics.
229 The exception is "documenting software easily and thoroughly" or "Agile
Documentation."
Agile documentation requires that documents be created to represent software in every aspect and level of abstraction using little effort. The documents will be useful when changes occur as the developers can use them to better understand the code. The developers can then proper evaluate the consequences and formulate modifications.
Moreover, the persistency of the created documents also helps the development team counter with personel changes.
A more thorough review is conducted focusing on techniques collectively called
"Generative Programming" or GP. The results support the initial conclusion that there is no existing documentation technique suitable for agile documentation.
The Proposed Solution Since the ultimate target of this research is to solve the process agility problem, an agile documentation method must be realized. A software-documentation method, the
Intentional Generative Pattern Mechanism (1GPM), is proposed for agile documentation.
The IGPM utilizes a building block called "pattern." A pattern is used to capture a developer's intention. It encapsulates software development actions and a label describing the intention. The label acts as a natural documentation of those actions. One pattern should encompass one concise intention. More complex intentions can be represented by a group of connected patterns called "a pattern network." An entire piece of software can also be naturally-documented in the same way.
230 Patterns are also executable. When a pattern is executed, it performs its encapsulated actions. An execution of a pattern network ultimately results in the development of a software program (by generating the software code). Code generation allows easy synchronization between the documents (the pattern network) and the code.
Since patterns can naturally document an entire software and can be used to automatically generate the software code, they enable software documentation to be done easily and thoroughly, which makes the IGPM a perfect choice for agile documentation.
Implementation and Experimentation In order to prove that such mechanism can be developed, the pattern programming language is developed as a realization of the IGPM. It is then used in a case study to demonstrate how the mechanism can be used and to show that it is possible to document software code thoroughly with less effort. Modifications to the existing code can be applied to the existing code quickly with less effort and in manageable fashion. This should prove that the IGPM can be used as a promising technique for Agile
Documentation.
8.2 Future Work
Although the case study shows that the pattern language as the realization for the IGPM can be used in a project and helps improve agility, there are seveSomewhere In Time ral open issues and possible problems that require further study.
Missing Features From the case study, two missing features seem to be very useful for patterns programming: 1) map-port kind and 2) human-friendly intention tracking.
231 Map-Port Kind The currently implement port kinds (single and collection) seem to be adequate for the proof of concept implementation currently implemented. However, the missing array and map port kinds makes it harder to implement flexible and concise patterns. For example, without the Map-port kind, PHPProject (page 280) must handle pages by itself which greatly complicates the pattern code (#19-56).
Human-friendly Intention tracking For the moment, intention tracking or investigation of the chain of intentions can only be done programmatically. This is inconvenient or hardly usable for large projects. A human-friendly intention tracking mechanism should be developed so that the full potential of the IGPM for agile documentation can be reached.
Pattern Network Complexity A pattern network is supposed to naturally document the final software. If the network (in itself) is complex, there will be further need to document it. This is undesirable. Further studies are required to find ways of simplifying the complexity that may occur in large pattern networks. This may include: 1) new features for the pattern language (such as the map-port kind above), 2) design patterns for patterns, or 3) design patterns for target software that may be easy for the IGPM.
232 LIST OF REFERENCES
[1] F. P. Brooks, The Mythical Man-Month: Essays on Software Engineering, 20th Anniversary Edition. Addison-Wesley Professional, 1995. ISBN: 0201835959.
[2] D. Leffingwell and D. Widrig, Managing Software Requirements: A Use Case Approach (2ndEdition). Addison-Wesley Professional, 2003. ISBN: 0-321-12247-X.
[3] F. Trui, Managing Software Projects. Sudbury, MA, USA: Jones and Bartlett Publishers, 2004. ISBN: 0-7637-2546-3.
[4] P. G. Armour, The Laws of Software Process: A New Model for the Production and Management of Software. Auerbach Publications, 2004. ISBN: 0-8493-1489-5.
[51 N. C. Shu, Visual Programming. Van Nostrand Reinhold Company Inc., 1988. ISBN: 0-442-28014-9.
[6] S. S. Alhir, Learning UML. O'Reilly Media, Inc., 2003. ISBN: 0596003447.
[7] M. M. Burnett, A. Goldberg and T. G. Lewis, Visual Object-Oriented Programming. Manning Publications, 1995. ISBN: 1-884777-01-5.
[8] K. E. Wiegers, Software Requirements. Redmond, Washington: Microsoft Press, 2003. ISBN: 0-7356-1879-8.
[9] IEEE, "IEEE recommended practice for software requirements specifications," The Institute of Electrical and Electronics Engineers (IEEE), Tech. Rep. IEEE Std 830-1998, October 1998.
[10] W. S. Bennett, Visualizing Software: A Graphical Notation for Analysis, design and Discussion. New York: Marcel Deffer, Inc., 1992. ISBN: 0-8247-8714-5.
[11] Martin Fowler, UML Distilled: A Brief Guide to the Standard Object Modeling Language, Third Edition. Addison-Wesley Professional, 2003. ISBN: 0321193687.
[12] D. Leffingwell and D. Widrig, Managing Software Requirements: A Use Case Approach (1st Edition). Addison-Wesley Professional, 2000. ISBN: 0201615932.
[13] J. Highsmith, Agile Software Development Ecosystems. Addison-Wesley Professional, 2002. ISBN: 0-201-76043-6.
233 [14] R. S. Pressman, Software Engineering: A Practition's approach, 6th Edition. McGraw Hill, 2005. ISBN: 0-07-285318-2.
[15] B. Boehm and R. Turner, Balancing Agility and Discipline: A Guide for the Perplexed. Boston, MA, USA: Addison-Wesley Professional, 2003. ISBN: 0-321-18612-5.
[16] K. Beck, "Embracing Change with Extreme Programming," Computer, vol. 32, no. 10, pp. 70-77,1999.
[17] W. Humphrey, "Comments on eXtreme Programming: eXtreme Programming: Pros and Cons - What Questions Remain?," No longer available online document, 2000.
[18] A. Cockburn, Agile Software Development. Addison-Wesley Professional, 2001. ISBN: 0201699699.
[19] P. Kruchten, "Agility with the RUP," Cutter IT Journal, vol. 14, no. 12, pp. 27-33,2001.
[20] D. J. Reifer, F. Maurer, and H. Erdogmus, "Scaling Agile Methods," IEEE Software, vol.20, no.4, pp.12-14, 2003.
[21] M. Stephens and D. Rosenberg, Extreme Programming Refactored: The Case Against XP. Apress, 2003. ISBN: 1590590961.
[22] Marriam-Webster Online Dictionary, "Marriam-Webster Online Dictionary," [Online], [cite 2005 July 24], Available: WWW: http://www.m-w.com/.
[23] IEEE, "IEEE Standards Collection: Software Engineering," IEEE, Tech. Rep. IEEE Standard 610.12-1990, 1993.
[24] M. Hammer, J. Champy, Reengineering the Corporation: A Manifesto for Business Revolution (Reprint edition). HarperBusiness, 1994. ISBN: 088730687X.
[25] T. Olson, W. Humphrey, and D. Kitson., "Conducting SEI-assisted Software Process Assessments," Software Engineering Institute, Tech. Rep. Technical Report CMU|SEI-90-TR-7, February 1989.
[26] M. C. Paulk et al., "Capability Maturity Model® for Software (CMM-SW), Version 1.1," Software Engineering Institute, Tech. Rep. CMU/SEI-93-TR-24, February 1993.
[27] B. Boehm, "A Spiral Model for Software Development and Enhancement," Computer, vol. 21, no. 5, pp. 61-72,1988.
[28] R. S. Pressman, Software Engineering: A Practition's approach, 4th Edition. McGraw Hill, 1997. ISBN: 0-07-052182-4.
234 [29] W. Humphrey, A Discipline for Software Engineering. Addison-Wesley Professional, 1995. ISBN: 0201546108.
[30] D. Hamlet and J. Maybee, The Engineering of Software. Addison-Wesley Longman, Inc, 2001. ISBN: 0-201-70103-0.
[31] L. Osterweil, "Software processes are software too," in Proceedings of the 9th international conference on Software Engineering. (Monterey, CS, USA, March 30 - April 2, 1987, pp. 2-13).
[32] W. W. Royce, Managing the Development of Large Software Systems: Concepts and Techniques. Los Angeles: WESCON, 1970.
[33] V. R. Basili and A. J. Turner, "Iterative Enhancement: A Practical Technique for Software," IEEE Transactions on Software Engineering, vol. 1, no. 4, pp. 390-396, 1975.
[34] J. Martin, Rapid Application Development. Prentice-Hall, 1991. ISBN: 0023767758.
[35] T. Gilb, Principles of Software Engineering Management. Addison-Wesley Professional, 1988. ISBN: 0201192462.
[36] CMMI Product Team, "Capability Maturity Model® Integration (CMMISM), Version 1.1: Continuous Representation," Software Engineering Institue, Tech. Rep. CMU/SEI-2002-TR-011, March 2002.
[37] L. Probasco, "The Ten Essentials of RUP: The Essence of an Effective Development Process," Rational Software, Tech. Rep. TP-177 9/00, September 2000.
[38] CMMI Product Team, "Capability Maturity Model® Integration (CMMISM), Version 1.1: Staged Representation," Software Engineering Institue, Tech. Rep. CMU/SEI-2002-TR-012, March 2002.
[39] R. Jeffries, A. Anderson and C. Hendrickson, Extreme Programming Installed. Addison-Wesley Professional, 2000. ISBN: 0201708426.
[40] W. Royce, Software Project Management: A Unified Framework. Addison-Wesley Professional, 1998. ISBN: 0-201-30958-0.
[41] C. Ghezzi, M. Jazayeri and D. Mandrioli, Fundamentals of Software Engineering. Prentice-Hall, 1991. ISBN: 0-13-820432-2.
[42] B. Oestereich, Developing Software with UML: Object-Oriented Analysis and Design in Practice (2nd Edition). Addison-Wesley Professional, 2001. ISBN: 0-201-75603-X.
235 [43] J. McLean, "A Formal Method for the Abstract Specification of Software," Journal of the ACM, vol. 31, no. 3, pp. 600 - 627, 1984.
[44] S. W. Ambler and B. McGibbon, The Object Primer: Introduction to Techniques for Agile Modeling. Cambridge University Press; (2nd Edition), 2001. ISBN: 0521785197.
[45] ISO/IEC TR 15504, "ISO/IEC TR 15504," [Online], [cite 2005 July 27], Available: WWW: http://isospice.com/standard/trl5504.htm.
[46] M. C. Paulk et al., "Key Practices of the Capability Maturity ModelSM, Version 1.1," Software Engineering Institue, Tech. Rep. CMU/SEI-93-TR-025, February 1993.
[47] J. Link and P. Frohlich, Unit Testing in Java: How Tests Drive the Code. Morgan Kaufmann Publishers, 2003. ISBN: 1558608680.
[48] W. Humphrey, Managing Technical People. Addison-Wesley Professional, 1997. ISBN: 0201545977.
[49] T. Demarco and T. Lister, Peopleware : Productive Projects and Teams (2nd Edition). Dorset House Publishing Company, Incorporated., 1999. ISBN: 0932633439.
[50] E. MacGregor, Y. Hsieh and P. Kruchten, "The Impact of Intercultural Factors on Global Software Development," in Proceedings of the 18th Canadian Conference on Electrical and Computer Engineering (CCECE'05). (Saskatoon, SK, Canada, May 1-4, 2005, pp. 890-896).
[51] P. Abrahamsson el at., Agile Software Development Methods: Review and Analysis. VTT Publication, 2002.
[52] W. C. Wake, Express Programming Explored. Addison-Wesley Professional, 2002. ISBN: 0-201-73397-8.
[53] Answer.com, "Definition and Much More from Answer.com," [Online], [cite 2005 July 24], Available: WWW: http://www.answers.com/.
[54] A. Cockburn, Agile Software Development: The Cooperative Game. 2000.
[55] P. Kroll and P. Kruchten, The Rational Unified Process Made Easy: A Practitioner's Guide to the RUP. Addison-Wesley Professional, 2003. ISBN: 0-321-16609-4.
[56] R. C. Linger, "Cleanroom Software Engineering for Zero-Defect Software," in The 15th International Conference on Software Engineering. (Baltimore, MD, USA, May 17-21,1993, pp. 2-13).
236 [57] W. S. Humphrey, "Why Big Software Projects Fail: The 12 Key Questions," CROSSTALK The Journal of Defense Software Engineering, 2005.
[58] M. B. Rosson and J. M. Carroll, Usability Engineering: Scenario-based Development of Human-Computer Interaction. Academic Press, 2002. ISBN: 1-55860-712-9.
[59] E. M. Hall, Managing Risk: Methods for Software Systems Development. Addison-Wesley Professional, 1997. ISBN: 0-201-25592-8.
[60] S. Dart, "Concepts in Configuration Management Systems," in Proceedings of the 3rd international workshop on Software configuration management. (Trondheim, Norway, 1991, pp. 1 -18).
[61] C. Larman, Applying UML and Patterns: An Introduction to Object-Oriented Analysis and Design and the Unified Process (2nd Edition). Prentice-Hall, 2001. ISBN: 0130925691.
[62] R. Jeffries, "Essential XP: Documentation," [Online], 2001 Dec 11, [cite 2005 May 24], Available: WWW: http://www.xprogramming.com/xpmag/expDocumentationInXP.htm.
[63] A. Shalloway and J. Trott, Design Patterns Explained: A New Perspective on Object-Oriented Design. Addison-Wesley Professional, 2001. ISBN: 0201715945.
[64] E. Gamma, R. Helm et al., Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley Professional, 1995. ISBN: 0-201-63361-2.
[65] J. Frechot, "Domain-Specific Languages," [Online], 2003 Sep 25, [cite 05 Aug 4], Available: WWW: http://compose.labri.fr/documentation/dsl/.
[66] K. Beck, Extreme Programming Explained: Embrace Change. Addison-Wesley Professional, 1999. ISBN: 0201616416.
[67] M. Stephens and D. Rosenberg, "Will Pair Programming Really Improve Your Project?," Methods & Tools, vol. 11, no. 4, pp. 23-26, 2003.
[68] G. Derbier, "Agile Development in the Old Economy," in Proceedings of the Agile Development Conference (ADC\J)3). (2003).
[69] Bluelnk, "Rapid Application Development," [Online], [cite 2005 May 26], Available: WWW: http://www.blueink.biz/RapidApplicationDevelopment.aspx.
[70] W. S. Avis, Gage Canadian Dictionary. Toronto: Gage Publishing Limited, 1973. ISBN: 0-7715-9122-5.
[71] Agile alliance, "Agile Manifesto," [Online], 2001, [cite 2005 July 28], Available: WWW: http://agilemanifesto.org/.
237 A. Silberschatz, H. F. Korth and S. Sudarshan, Database System Concepts. McGraw Hill, 1998. ISBN: 0-07-031086-6.
G. Booch, J. Rumbaugh and I. Jacobson, The Unified Modeling Language User Guide. Addison-Wesley Professional, 1998. ISBN: 0201571684.
H. D. Mills, M. Dyer, and R.C. Linger., "Cleanroom Software Engineering," IEEE Software, pp. 19-24,1987.
J.-P. Corriveau, "Traceability Process for Large OO Projects," Computer, vol. 29, no. 9, pp. 63 - 68,1996.
J. A. McDermid, Software Engineer's Reference Book. Butterworth-Heinemann, 1990. ISBN: 0750610409.
G. Scheider and J. P. Winters, Applying Use Cases: A Practical Guide (2nd Edition). Addison-Wesley Professional, 2001. ISBN: 0-201-70853-1.
J. M. Spivey, The ZNotation: A Reference Manual (2nd Edition). Oriel College, Oxford, English: Prentice-Hall, 19980139785299.
Y. Ledru and P.-Y. Schobbens, "Applying VDM to large developments," in Proceeding of the ACM SIGSOFT International Workshop on Formal Methods in Software Development. (Napa, California, US, May 9-11, 1990, pp. 55-58).
S. J. Mellor and M. J. Balcer, Executable UML: A Foundation for Model-Driven Architecture. Addison-Wesley Professional, 2002. ISBN: 0-201-74804-5.
K. E. Wiegers, "Misconceptions of the Capability Maturity Model," [Online], November 1996, [cite 2005 Nov 1], Available: WWW: http://www.processimpact.com/articles/miscon.html.
J. Moore, "ISO 12207 and Related Software Life-Cycle Standards," [Online], [cite 2005 Oct 25], Available: WWW: http://www.acm.org/tsc/lifecycle.html.
R. C. Linger, "Cleanroom Development Model," IEEE Software, vol. 11, no. 2, pp. 50-58, 1994.
Wikipedia, "Rapid Application Development," [Online], 2005 November 9, [cite 2005 November 10], Available: WWW: http://en.wikipedia.org/wiki/Rapid_application_development.
J. Butler, "Rapid Application Development in Action," Managing System Development, Applied Computer Research, vol. 14, no. 5, pp. 6-8, 1994.
Wikipedia, "Rational Unified Process," [Online], 2005 Oct 24, [cite 2005 Oct 28], Available: WWW: http://en.wikipedia.org/wiki/Rational_Unified_Process.
238 [87] Rational Software, "Rational Unified Process: Best Practices for Software Development Teams," Rational Software, Tech. Rep. TP026B, November 2001.
[88] D. R. Windle and L. R. Abreo, Software Requirements Using the Unified Process: A Practical Approach. Peason Education, 2002. ISBN: 0-13-096972-9.
[89] I. Jacobson, Object-Oriented Software Engineering: A Use Case Driven Approach. Addison-Wesley Professional, 1992. ISBN: 0-201-54435-0.
[90] W. S. Humphrey, "Personal Software Process," Software Engineering Institue (SEI), Tech. Rep. CMU/SEI-2000-TR-022, November 2000.
[91] W. S. Humphrey, "Team Software Process," Software Engineering Institue (SEI), Tech. Rep. CMU/SEI-2000-TR-023, November 2000.
[92] A. Kaniss and L. L. Crosby, "CMM, TSP, PSP: a winning combination for NAVAIR systems, software," [Online], 2003 April 10, [cite 2005 Oct 28], Available: WWW: http://www.dcmilitary.com/navy/tester/8_14/commentary/22557-l.html.
[93] A. S. Koch, "TSP Can Be the Building Blocks for CMMI," The Journal of Defense Software Development (CrossTalk), [Online] March. 2005. Available: WWW: URL: http://www.stsc.hill.af.mil/crosstalk/2005/03/0503Koch.html.
[94] J. Amador-Monteverde, "The Hierarchical Object Oriented Design method," [Online], 1996 Nov 14, [cite 2005 Oct 28], Available: WWW: http://www.estec.esa.nl/wmwww/WME/oot/hood/.
[95] I. White, Using the Booch Method: A Rational Approach. The Benjamin/Cummings Publishing Company Inc., 1994. ISBN: 0-8053-0614-5.
[96] G. Kiczales, J. Lamping et al., "Aspect-Oriented Programming," in Proceeding of the European Conference on Object-Oriented Programming (ECOOP). (Finland, June, 1997).
[97] R. C. Martin, Agile Software Development: Principles, Patterns and Practices. Upper Saddle River, NJ, USA: Prentice-Hall, 2003. ISBN: 0-13-597444-5.
[98] A. Cockburn, "Learning From Agile Software Development," CROSSTALK The Journal of Defense Software Engineering, pp. 10-14, 2002 (October).
[99] K. Schwaber, Agile Project Management with Scrum. Microsoft Professional, 2004. ISBN: 073561993X.
[100] M. Poppendieck and T. Poppendieck, "Lean Programming," [Online], 2004 Nov 28, [cite 2005 Oct 26], Available: WWW: http://www.poppendieck.com/lean.htm.
[101] M. Poppendieck and T. Poppendieck, "Lean Design," [Online], 2003 Jun 09, [cite 2005 Oct 26], Available: WWW: http://www.poppendieck.com/design.htm.
239 [102] P. Abrahamsson, J. Warsta el at., "New Directions on Agile Methods: A Comparative Analysis," in Proceedings of the 25 International Conference on Software Engineering (ICSE'03). (Portland Oregon, May 3-10, 2003, pp. 244-254).
[103] J. Sutherland, "Scrum Godfathers: Takeuchi andNonaka," [Online], 2004 March 24, [cite 2005 Aug 4], Available: WWW: http://jeffsutherland.com/scrum/2005/03/scrum-godfathers- takeuchi-and-nonaka.html.
[104] L. Rising and N. S. Janoff, "The Scrum Software Development Process for Small Teams," IEEE Software, vol. 17, no. 4, pp. 26-32, 2000.
[105] DSDM Consortium, "DSDM Consortium - Helling you deliver on time," [Online], [cite 2005 Oct 25], Available: WWW: www.dsdm.org.
[106] D. Millington, "Developing A RAD Standard," IEEE Software, vol. 12, no. 5, pp. 54-55,1995.
[107] M. Clifton and J. Dunlap, "What Is DSDM?," [Online],2003 Sep, [cite 2005 Nov], Available: WWW: http://www.codeproject.com/gen/design/dsdm.asp.
[108] Nebulon Pty Ltd., "Feature Driven Development (FDD)," [Online], [cite 2005 Oct 28], Available: WWW: http://www.featuredrivendevelopment.com.
[109] S. R. Palmer and J. M. Felsing, A Practical Guide to Feature-Driven Development. NewYork, USA: Practice Hall PTR, 2002. ISBN: 0130676152.
[110] Joseph S. Martinich, Production and Operations Management: Applied Modern Approach. McGraw Hill, 1997. ISBN: 0-471-54632-1.
[111] K.. Beck, K. Auer et al., "What is eXtreme Programming?," [Online], [cite 2005 Aug 4], Available: WWW: http://www.xprogramming.com/what_is_xp.htm.
[112] M. A. Cusumano and R. W. Selby, "How Microsoft Builds Software," Communication of the ACM, vol. 40, no. 6, pp. 53-61, 1997.
[113] The Pragmatic Programmers, LLC, "The Pragmatic Programmers, LLC," [Online], [cite 2005 Nov 1], Available: WWW: http://www.pragmaticprogrammer.com/.
[114] S. W. Ambler, "Agile Modelling (AM) Home Page: Effective Practices for Modeling and Documentation," [Online], 2005 Oct 30, [cite 2005 Nov 1], Available: WWW: http://www.agilemodeling.com/.
[115] T. T. Barker, Writing Software Documentation: A Task-Oriented Approach. A ViaCom Company, 1998. ISBN: 0-205-19576-8.
[116] T. DeMarco and B. Boehm, "The Agile Methods Fray," Computer, vol. 35, no. 6, pp. 90-92, 2002.
240 [117] A. Endres and D. Rombach, A Handbook of Software and System engineering: Empirical Observations, Laws and Theories. Peason - Addison Wesley, 2003. ISBN: 0-321-15420-7.
[118] C3, "Chrysler Comprehensive Compensation," [Online], 2004 Dec 21, [cite 2005 July 24], Available: WWW: http://c2.com/cgi/wiki? ChryslerComprehensiveCompensation.
[119] L Bass, P. Clements and R. Kazman, Software Architecture in Practice (2nd Edition). Addison-Wesley Professional, 2003. ISBN: 0-321-15495-9.
[120] P. Hodgetts, "Refactoring the Development Process: Experiences with the Incremental Adoption of Agile Practices," in Proceedings of the Agile Development Conference (ADCD04). (2004).
[121] X Bin, Yang Xiaohu et al., "Extreme programming in reducing the rework of requirement change," in Canadian Conference on Electrical and Computer Engineering, 2004.. (May 2-5, 2004, pp. 1567 - 1570).
[122] C3, "C3 Project Terminated," [Online], 2005 Mar 23, [cite 2005 July 27], Available: WWW: http://c2.com/cgi/wiki7CthreeProjectTerminated.
[123] B. Fazackerley, "Is DSDM Agile?,'* [Online], [cite 2005 Nov 1], Available: WWW: http://www.dsdm.org/timebox/issuel6/agile.asp.
[124] www.agileapproach.co.uk, "Why Modular Prototyping?," [Online], [cite 2005 Nov], Available: WWW: http://www.agileapproach.co.uk/html/why_modular_prototyping_.html.
[125] J. Highsmith, "What is Agile Software Development," The Journal of Defense Software Development, pp. 4-9, 2002.
[126] J. P. Lewis, "Large Limits to Software Estimation," ACM Software Engineering Notes, vol. 26, no. 4, pp. 54-59, 2001.
[127] R. Lafore, Object-Oriented Programming in C++ (3nd). SAMS Publishing, 1999. ISBN: 1-57169-160-X.
[128] R. Harris, "Introduction to Problem Solving," [Online], 1998 Jul 2, [cite 2005 May 27], Available: WWW: http://www.virtualsalt.com/crebook3.htm.
[129] Scott Morrison, "Google Searches for Staffing Answers ," [Online], [cite 2009 May 19], Available: WWW: http://online.wsj .com/article/SB12426903 8041932531 .html+.
[130] M. Mason, Pragmatic Version Control - Using Subversion, 2nd Edition. Raleigh, NC, USA: The Pragmatic Bookshelf, 2006. ISBN: 0-9776166-5-7.
241 131] M. McKeever, How to Write a Business Plan. Consolidated Printers, Inc., 1999. ISBN: 0873375440.
132] Free Software Foundation, Inc., "GNU Make," [Online], 2004 Oct, [cite 2005 Nov], Available: WWW: http://www.gnu.org/software/make/.
133] The Apache Software Foundation., "Apache Ant," [Online], 2005 Jun 2, [cite 2005 Nov 10], Available: WWW: http://ant.apache.org/.
134] IBM Rational software, "Case studies: By customer," [Online], [cite 2005 Nov 10], Available: WWW: http://www- 306.ibm.com/software/success/cssdb.nsf/customerVW? Open V ie w&Count=1 O&RestrictT oCategory=rational.
135] R. C. Martin, Clean Code - A Handbook of Agile Software Craftmanship, Boston, MA, USA: Prentice Hall, 2008. ISBN: 0-13-235088-2.
136] K. Czarnecki and U. W. Eisenecker, Genertive Programming: Methods, Tools, and Applications. Addison-Wesley Professional, 2000. ISBN: 0-201-30977-7.
137] The MathWorks, Inc., "The MathWorks - MATLAB and Simulink for Technical Computing," [Online], [cite 2005 Nov 10], Available: WWW: http://www.mathworks.com/.
138] C. Simonyi, "The Death Of Computer Languages, The Birth of Intentional Programming," Microsoft Research, Tech. Rep. MSR-TR-95-52, September 1995.
139] S. Dmitriev, "Language Oriented Programming: The Next Programming Paradigm," JetBrains Inc., Tech. Rep. http://www.onboard.jetbrains.eom/isl/articles/04/l O/lop/mps.pdf, November 2004.
140] S. MacDonald, D. Szafron et al., "Generative Design Pattern," in Proceedings of the I7th IEEE International Conference on Automated Software Engineering (ASE'02). (Sept 23-27,2002, pp. 23 - 34).
141] Sun Microsystems, Inc., "JavaDoc Tool Home Page," [Online], [cite 2005 Nov 10], Available: WWW: http://java.sun.com/j2se/javadoc/.
142] D. Heesch, "Doxygen," [Online], 2005 Nov 8, [cite 2005 Nov 10], Available: WWW: http://www.stack.nl/~dimitri/doxygen/.
143] W3C®, "W3C HTML Home," [Online], 2005 Oct 13, [cite 2005 Nov 10], Available: WWW: http://www.w3.org/MarkUp/.
144] National Instruments Corporation., "LabVIEW - The Software That Powers Virtual Instrumental," [Online], [cite 2005 Nov 10], Available: WWW: http://www.ni.com/labview/.
242 [145] Electronics Workbench Corporation, "Electronics Workbench — A National Instrument Company," [Online], [cite 2005 Nov 1], Available: WWW: http://www.electronicsworkbench.com/.
[146] M. Fowler, Refactoring: Improving the Design of Existing Code. :Addison-Wesley Professional, 1999. ISBN: 0-201-48567-2.
[147] Randal L. Scheartz, Learning Perl. Sebastopol, California, USA: O'Reilly, 2005. ISBN: 978-0-596-10105-3.
[148] Kevin McArthur, Pro PHP: Patterns, Frameworks, Testing and More. Berkeley, CA, USA: Apress, 2008. ISBN: 1-59059-819-9.
[149] C. Newham and B Rosenblatt, Learning the bash Shell, 3rd Edition. Sebastopol, California, USA: O'Reilly Media, Inc., 2005. ISBN: 978-0-596-00965-6.
[150] J. E. F. Friedl, Mastering Regular Expression. Sebastopol, California, USA: O'Reilly, 2002. ISBN: 0-596-00289-0.
[151] Jacob Harris, Rubyisms in Rails. Boston, MA, USA: Addison-Wesley Professional, 2006. ISBN: 0-321-47407-4.
[152] M. C. Feathers, Working Effectively with Legacy Code. Upper Saddle River, NJ, USA: Prentice Hall, 2005. ISBN: 0-13-117705-2.
[153] A Hunt and D. Thomas, The Pragmatic Programmer: From Journeyman To Master. Raleigh, NC, USA: Addison-Wesley Professional, 1999. ISBN: 0-201- 61622-X.
243 APPENDIX A
THE CODE EXAMPLES
I Example#!
TextFile.pattern - Exmaple #1 of 6.4.1 // ^Pattern! package nowaman->text;
import net.nawaman.util.UText;
public pattern TextFile {
in port Name :Text; in port EXT sText; in port Path :Text; in port BaseDIR :Text;
in port Content :Text;
out port FullName:Text = <${$... <(BaseDIR )><((?BaseDIR:"/"))>$. <(Path )><((?Path: "/"))>$. <(Name )>$. <((?EXT:"."))><( EXT )>$. >$>;
public new() { this.ensureName NotEmptyf); )
public function render():void { this.saveFile(); )
private function ensureName_NotEmpty():void < assert(Name): (Name?$.length() != 0); }
protected function saveFile():void { String theFileName « this.FullName?$; String theContent = this.Content ?$; UText.SaveTextToFile(theFileName, theContent); }
244 II Example #2
EZ write UserPath.pattern - Exmaple #2 of 6.4.2 1 // SPattern: 2 package ex02; 3 4 import nawaman->text=>*; 5 6 pattern Rewrite_UserPath { 7 8 public port RewriteRule :Text; 9 10, public new() { before(RewriteRule): RewriteRule := this.prepareRewriteRule(); > 11 12 public function prepareRewriteRule():Text { 13 return <${<(Word_RewriteRule())><(Spaces())><(Rule())><(Spaces())><(Target())>}$>; 14 } 15 16 private function Word_RewriteRule():Text { return <${RewriteRule}$>; ) 17 private function Spaces)) :Text { return <${ }$>; > 18 19 private function Rule() :Text { 20 return <${<( 21 The_beginning_character_is_a_slash_follows_by_a_tilde() )><( 22 Xn_group_one_contains_everything_but_slash_as_a_username())><( 23 Then_an_optional_slash() )><( 24 Group_two_is_the_rest_of_the_path() 25 >>>$>; 26 } 27 private function The_beginning_character_is_a_slash_follows_by_a_tilde():Text { return <$<<(Start_with(<${/-}$>))>>$>;
> 28 private function In_group_one_contains_everything_but_slash_as_a_username():Text { return <${<(Group_of(<${<(Everything_but(Slash()))>>$>))>}$>;
> 29 private function Then_an_optional_slash():Text { return <${<(Optional_of(Slash()))>}$>; } 30 private function Group_two_is_the_rest_of_the_path():Text { return <${<(Group_of(The rest()))>}$>;
31 32 33 private function Target():Text { 34 return <${<( 35 Begin_with_the_user_folder() )><( 36 Then_the_username_from_group_one() )><( 37 Next_is_a_slash() )><( 38 End_with_the_rest_of_the_path_from group two() 39 )>)$>; 40 41 private function Begin_with_the_user_folder():Text { return <${/users/>$>; > 42 private function Then_the_username_from_group_one():Text { return <${$1>$>; } 43 private function Next_is_a_slash():Text { return Slash(); } 44 private function End_with_the_rest_of_the_path_from_group two():Text { return
45 46 private function Start_with (a :Text) :Text { return <${-<(a)>}$>;
47 private function Group_of (a :Text) :Text < return <${(<(a)>)}$>; 48 private function The_rest () :Text { return <${.*}$>; 49 private function Optional_of (a :Text) :Text { return <${<(a)>?}$>; 50 private function Everything but(a :Text) :Text { return
245 Example #3
ition.pattern - Exmaple #3 of 6.4.3 1 // ^Pattern: 2 package bashoption; 3 4 import nawaman->text=>*; 5 6 pattern Option { 7 8 port Name :Text; 9 port VarName :Text; 10 port Default :Text; 11 port OptionChar :char; 12 port IsOptional :boolean; 13 port IsParameterNeeded :boolean; 14 port ValuelfNoParameter :Text; 15 16 public new() { 17 Name = VarName; 18 OptionChar = this.getLowerCase_Fir8tChar_0f_VarName{) 19 IsNeedParameter = false; 20 21 this.ensureVarName_NotNullOrEmpty(); 22 this.ensureOptionCbar NotNull(); 23 } 24 private new(aVarName :Text, aDefault :Text, aOptionChar :char, alsOptional :boolean, E IsParameterNeeded :boolean) { 25 this(); 26 VarName := aVarName; 27 Default aDefault; 28 OptionChar = aOptionChar; 29 IsOptional alsOptional; 30 IsParameterNeeded alsParameterNeeded ; 31 } 32 public new(aVarName :Text, aDefault :Text, aOptionChar :char, alsOptional :boolean) { 33 this(aVarName, aDefault, aOptionChar, alsOptional, true); 34 > 35 public new(aVarName :Text, aDefault sText, aOptionChar :char, alsOptional :boolean, aValuelfNoParameter:Text) { 36 this(aVarName, aDefault, aOptionChar, alsOptional, false); 37 ValuelfNoParameter := aValuelfNoParameter; 38 } 39 40 41 private function ensureVarName_NotNullOrEmpty()svoid { 42 -!assert(VarName): (VarName?.toString()?.length() != 0); 43 } 44 private function ensureOptionChar_NotNull():void { 45 -iaasert(OptionChar): (OptionChar 1= null); 46 } 47 private function getLowerCase_FirstChar_Of_VarName():char < 48 return VarName.toString().toLowerCase().charAt(0); 49 ) 50 51 52 public function getVariableDeclaration():Text { 53 return <${<(VarName)>=<(Default)x($"\n" )>}$>; 54 > 55 public function getOptionChar_and_Colon():Text { 56 return <${<(OptionChar)><{ if{IsParameterNeeded) echo >>}$>; 57 } 58 public function getOptionWithParameterProcessing():Text { 59 Text Processing; 60 i f(IsParameterNeeded) 61 Processing = <${<(VarName)>="50PTARG";}$>; 62 else Processing = <${echo " '-<(OptionC/iar)>' requires no parameter"; exit;}$>; 63 64 return <${ "<(OptionChar)>" ) <(,Processing)> ;;<( $"\n" )>>$>; 65
246 public function getOptionsWithoutParaaeterProcessing(>:Text { Text Processing; if(IsNeedParameter) Processing = <$
return <${ "<(OptionChar)>") <(Processing)> ;;<($"\n")>}$>; > public function getManual():Text { Text OptionStr • <${-<(OptionCJjar)>>$>; if(IsParameterNeeded) OptionStr = <${<(OptionStr)> "<(Name)>"}$>; if(IsOptional) OptionStr = <${[<(OptionStr)>]}$>; return OptionStr; > }
247 i.pattern - Exmaple #3 of 6.4.3 1 // GPattern: 2 package bashoption; 3 4 import nawaman->text=>*; 5 6 pattern ProcessOptions { 7 8 in port Options :Option* « new(); 9 out port Code :Text; 10 11 12 public new() { 13 Code :» this.prepareCode(); 14 this.enBureOptions_NotEmpty(); 15 ) 16 17 18 private function enBureOptionB_NotEmpty():void { 19 -:as8ert(Options): (Options?.size() !» 0); 20 > 21 22 private function prepareCode():Text < 23 if(Options?.size() == 0) 24 return null; 25 26 return 27 <${ }$> 28 <(this.getVariableDeclarations())> 29 while getopts ":<(this.getOptionsString( ))>" opt J 30 do J 31 case "$opt" in J 32 <(this.getOptionsHithParaneterProcessing())>$... 33 "?") echo "Unknown option $OPTARC"; exit; ;;J 34 J 35 case "SOPTARG" inJ 36 <(this.getOptionsWithoutParameterProcessing())>$... 37 "m") echo "<(this.getHanual())>"; exit; ;; J 38 esac J 39 exit; 4 40 ;; J 41 *) echo "Unknown options"; ;; J 42 esac J 43 done J 44 <$< >$>; 45 46 47 48 private function getVariableDeclarations() :Text { 49 return <${<{ 50 fromto(int i - 0 : Options?.size()) 51 echo Options.get(i).getVariableDeclaration(); 52 }>>$>; 53 } 54 private function getOptionsString():Text { 55 return <${<{ 56 fromtofint i « 0 : Options?.size()) 57 echo Options.get(i).getOptionChar_and_Colon(); 58 >>)$>; 59 } 60 private function getOptionsNithParameterProcessing():Text < 61 return <${<{ 62 fromtofint i = 0 : Options?.size()) 63 echo Options.get(i).getOptionWithParameterProceBaing(); 64 )>)$>? 65 > 66 private function getOptionsHithoutParameterProcessing():Text { 67 return <${<{ 68 fromto(int i » 0 : Options?.size()) 69 echo Options.get(i).getOptionsWithoutParameterProcessing();
248 70 >>}$>; 71 > 72 private function getManualO :Text { 73 return <${<{ 74 fromto(int i = 0 : Options?.size()) { 75 if(i 0) echo 76 echo Options.get(i).getManualf); 77 } 78 }>)$>; 79 > 80
249 APPENDIX B
INFRASTRUCTURE PATTERNS FOR THE CASE STUDY
I Introduction
This set of patterns was created and used in the ASC's previous projects and reused in the case study.
II Brief Description
II.l Database Patterns
This set of patterns handles database artifacts and intentions. The code of these patterns can be seen later in this Appendix. The following are short descriptions of each pattern
(in alphabetical order).
• condition - A query condition for selecting only desired records .
• Field - An artifact pattern for representing a database field .
• BasFieids - An interface of pattern for those with database fields.
• hasLinks - An interface of pattern for those with database links.
• Join - A relationship between two tables that do not require records from both tables to exist.
• Limit - A limit to the number of records to be queried.
• Link - A relationship between two tables that require records from both tables to exist.
250 • orderBy - A query clause that specifies how the queried records are ordered.
• Query - A pattern that represents a database query and produces corresponding SQL commands for that query.
• Table - An artifact pattern for representing a database table.
II.II HTML Patterns
These patterns represent artifacts and intentions for HTML-code manipulation.
• column - A column of an HTML table.
• BTMLFiie - An HTML file.
• styier - A styler for any text. These patterns are used to wrap the given text with other text, which makes it possible to put a styler to the given text.
• styier Nested - A styler that nests another styler.
• styier simpie - A simple styler (wrapping the given text with a prefix and suffix).
• styierjrag - A styler by an HTML tag (wrapping the given text with an HTML Tag).
• Table - An HTML table.
• Tabiestyier - A styler for an HTML table. It contains stylers for tables, rows and cells.
II.I1I PHP Patterns
This group of patterns represents artifacts and intentions for PHP-code manipulation.
• PHPFile - A PHP file.
• PHPProject - An interface for a PHP project.
251 • PHPProj ect_sinpie - A simple implementation of PHPProject.
II.IV PHP Database Patterns
This group of patterns contains artifact and intentional patterns for manipulating PHP code that interacts with the database.
• coiumnFieid - An intention pattern that shows a filed value as a DataTabie column.
• coiumnLink - An intention pattern that makes a column a hyper-link.
• condition_MatchParanieter - A condition for matching a database field to a Web- page parameter.
• DataTabie - An HTML table that shows data from a database query.
• Disconnection - An artifact pattern containing information about a database connection.
• LinkTo_page_of_seiecteditem - An intention pattern that links one page with another by having a hyper-linked column of the source page to open a target page by showing only the selected record that matches a value from the source column.
• PHPDBProject - An interface of a project for PHP with databases.
• PHPDBProject_simpie - A simple implementation for PHPDBProject.
• PBPFile withDataTable - PHPFile with a Main DataTabie.
252 Ill Code
III.I Database Patterns
lib/db/Condition.pattern // 6Pattern: package lib->db;
import nawaman->text->*;
pattern Condition implements AsText {
port first :Text; port second :Text; port comparator :Text;
port sql :Text := this.prepareSQL();
public new () { this.ensureAll_NotEmpty(); } public new ( pFirst :Text, pSecond :Text) < this(pFirst, pSecond); } public new ( pFirst :Text, pComparator :Text, pSecond :Text) { this(); first pFirst; second pSecond; comparator pComparator;
public asText() :CharSequence { return sql; }
private ensureAll_NotEmpty() :void { assert(first): first ?.length() != 0 assert(second): second ?.length)) 1- 0 assert(comparator): comparator?.length() != 0 }
protected prepareSQL() :Text { return <${(<(first)> <(comparator)> <(second)>)}$>; }
253 lib/db/Field.pattern // ^Pattern: package lib->db;
import nawaman~>text=>*;
pattern Field implements AsText { in port name :Text; in port table :Text; in port alias :Text = name;
out port sql :Text := this.prepareSQLf); out port definition :Text := this.prepareDefinition();
public new () { this.ensureName_NotEmpty(); > public new ( pTable :Text, pName :Text) { this(); name := pName; table pTable;
> public new ( pTable :Text, pName :Text, pAlias :Text) { this(pTable, pName); alias :- pAlias; >
public asTextf) :CharSequence { return sql;
>
private ensureName_NotEmpty() :void { assert(name)s name?.length() != 0; }
protected prepareSQLf) :Text { return <${<(table)><((?table:S"."))><(name)>}$>; > protected prepareDefinition() :Text { return <${<(sql)><{ if ((alias != null) && 1 alias.toString().equals(name?.toStringl))) echo <${ AS <(alias)>}$>; }>}5>; )
254 lib/db/HasFields.pattern // 9Pattern: package lib->db;
import nawaman->text=>*;
duck HasFields {
port fields :Field*;
static public AddAllFieldsf pSources :HasFields, pTargets :HasFields) :void {
if ((pTargets null) || (pSources == null)) return;
fromto (int i = 0 : pSources.fields?.size()) { Field aField - pSources.fields.get(i);
if (aField -- null) continue;
pTargets.fields <+- aField;
255 lib/db/HasLinks.pattem II ^Pattern: package lib->db;
import nawaman~>text=>*;
duck HasLinks {
port links sLink*;
static AddAllLinks( pSources :HasLinks, pTargets :HasLinks) svoid {
if ((pTargets == null) (pSources == null); return;
fromto (int i = 0 : pSources.links?.size()) { Link aLink - pSources.links.get(i);
if (aLink •« null) continue;
pTargets.links <+= aLink;
256 lib/db/Join.pattern // SPattern: package lib->db;
import nawaman~>text=>*;
pattern Join extends Link {
port condition :Text := super.prepareCondiion();
public new () < super(); isExclusive := false;
> public new ( pLeft :Field, pRight :Field) { this(); left := pLeft; right := pRight; >
257 lib/db/Limit.pattern // ^Pattern: package lib~>db;
import nawaman~>text=>*;
pattern Limit implements AsText {
port start :int; port count :int;
port sql :Text := this.prepareSQL();
public new () { this.ensureValue_NonNegative(); } public new (pCount :int) { this(); count :» pCount; } public new ( pStart :int, pCount :int) { this(pCount); start :» pStart;
>
public asText() :CharSequence { return sql;
>
private ensureValue_NonNegative() :void < -sassert(start): (start?? >= 0); -tassert)count): (count?? >= 0); }
protected prepareSQLf) :Text { if ((start == null) && (count == null)) return null;
Text Comma = ((start != null) && (count 1= null)) ? : null; return <$(LIMIT <(start)><(Comma)><(count)>>$>;
258 lib/db/Link.pattern // SPattern: package lib->db;
import nawaman~>text=>*;
pattern Link {
in port left :Field; in port right sField; in port isExclusive :boolean = true;
out port sql :Text := this.prepareDefinition();
public new () { this.ensureLeftRight_NotNull(); } public new( pLeft :Field, pRight :Field) { this(); left i= pLeft; right := pRight; }
private ensureLeftRight_NotNull( ) :void { -:a8sert(left): left 1= null; assert(right): right 1= null; assert(isExclusive): isExclusive i= null; }
protected prepareCondiion() :Text { return <${(<(left)> * <(right)>)}$>;
>
protected prepareDefinition() :Text { Text aCondition = this.prepareCondiion();
if (isExclusive) return aCondition;
return <${LEFT JOIN <(right.table)> ON <(aCondition)>}$>;
259 lib/db/OrderBy.pattern II ((Pattern: package lib->db;
import nawaman~>text=>*;
pattern OrderBy implements AsText {
port orderby :Text; port isDescending :boolean « false;
port sql :Text := this.prepareSQL();
public new () { this.ensureOrderBy_NonNull(); ) public new (pOrderBy :Text) { this( ); orderby :•= pOrderBy; } public new ( pOrderBy :Text, pIsDescending :boolean) {
this(pOrderBy); isDescending := pIsDescending;
public asTextf) :CharSequence { return sql; }
private ensureOrderBy_NonNull() :void { assert(orderby): (orderby?.length)) >= 0);
>
protected prepareSQL() :Text { return <$(ORDER BY <(orderby)><{ echo (isDescending ? $" DESC" null); }>>$>; }
260 '.pattern 1 // SPattern: 2 package lib->db; 3 4 import nawaman->data=>*; 5 import nawaman~>text=>*; 6 7 pattern Query implements AsText { 8 9 port isDistinct :boolean = true; 10 port fields :Field* - new(); 11 port tables :Text* = new(); 12 port links :Link* = new(); 13 port conditions :Text* = new(); 14 port limit :Limit; 15 port orderby :OrderBy; 16 port extra :Text; 17 18 port sql :Text this.prepareSQL(); 19 20 21 public isEmpty() iboolean { 22 return ((fields?.size() + links?.size()) «- 0); 23 } 24 25 public asText() :CharSequence { 26 return sql; 27 } 28 29 30 protected prepareSQL() :Text { 31 return 32 <${SELECT <( 33 IsDistinct)))><( 34 Fields() )><( 35 Tables() >><< 36 Links() )><( 37 Where( ) )><( 38 OrderBy() )><( 39 Limit() )><( 40 extra 41 >>>$>; 42 } 43 44 private IsDistinct() :Text { 45 return (isDistinct ? $"DISTINCT " : null); 46 > 47 private Fields)) :Text { 48 if (fields?.size() == null) 49 return null; 50 51 Text Separator = ", "; 52 return <${<{ 53 List
261 71 72 private getlgnoredTables() :List
262 141 } 142 } 143 } 144 > 145 146 // All the tables in the conditional link (exclusive) 147 fromto (int i » 0 : links?.size()) { 148 Link aLink = links.get(i); 149 if (this.isToUseLinkAsCondition(aLink)) { 150 Text aRightTable - aLink.right?.table; 151 if ((aRightTable 1= null) && IIgnoredTables.contains(aRightTable)) { 152 if( !IsFirst) 153 echo $","; 154 else IsFirst « false; 155 156 echo <${<(aRightTable)>}$>; 157 IgnoredTables.add(aRightTable); 158 } 159 160 Text aLeftTable « aLink.left?.table; 161 if ((aLeftTable 1= null) && 1IgnoredTables.contains(aLeftTable)) { 162 if( HsFirst) 163 echo $","; 164 else IsFirst = false; 165 166 echo <${<(aLeftTable)>}$>; 167 IgnoredTables.add(aLeftTable); 168 } 169 170 continue; 171 } 172 } 173 >>}$>; 174 175 if (Tables?.length() «» 0) 176 return null; 177 178 return <${ FROM <(Tables)>}$>; 179 } 180 private Links() :Text { 181 if (links?.size( ) -- 0) 182 return null; 183 184 Text Prefix = " "; 185 Text Separator - " "; 186 return <${<{ 187 boolean isFirst « true; 188 fromto (int i = 0 : links.size()) { 189 Link aLink = links.get(i); 190 if (this.isToUseLinkAsCondition(aLink)) 191 continue; 192 193 if (isFirst) { 194 echo Prefix; 195 isFirst - false; 196 > else { 197 echo Separator;
198 > 199 echo <${<(aLink.sql)>}$>; 200 ) 201 >>>$>; 202 } 203 204 private isToUseLinkAsCondition(pLink :Link) :boolean ( 205 return (pLink != null) && pLink.isExclusive; 206 } 207 private getNumberOfConditionFromLinks() :int { 208 int Count_condition_from_link = 0; 209 fromto (int i = 0 : links.size()) { 210 if (this.isToUseLinkAsCondition(links.get(i))) 211 Count_condition_from_link++;
263 212 > 213 return Count_condit ion_front_l ink;
214 > 215 private getConditions_from_conditions() :Text { 216 return <${<{ 217 boolean isFirst « true; 218 fromto (int i = 0 : conditions?.size()) { 219 Text aCondition = conditions.get(i); 220 if (aCondition -- null) 221 continue; 222 223 if (!isFirst) 224 echo $" and "; 225 226 isFirst - false; 227 echo <${<(aCondition)>}$>; 228 } 229 >>}$>; 230 > 231 private getConditions_from_links() ;Text { 232 return <${<{ 233 boolean isFirst = true; 234 fromto (int i = 0 : links?.Bize()) { 235 Link aLink = links.get(i); 236 if (1 this.isToUseLinkAsCondition(aLink)) 237 continue; 238 239 Text aCondition « <${<(aLink.sql)>)$>; 240 if (aCondition == null) 241 continue; 242 243 if (1isFirst) 244 echo $" and "; 245 246 isFirst = false; 247 echo <${<(aCondition)>}$>;
248 > 249 >>>$>; 250 } 251 private Where() :Text { 252 if ((conditions?.size() »- 0) && (this.getNumberOfConditionFromLinks() == 0)) 253 return null; 254 255 Text Conditions_from_conditions « this.getConditions_from_conditions(); 256 Text Conditions_from_links = this.getConditions_from_links(); 257 258 boolean isNoConditions_from_conditions = (Conditions_from_conditions?.length() == 0); 259 boolean isNoConditions_from_links « (Conditions_from_links ?.length() == 0); 260 261 if (isNoConditions_from_conditions && isNoConditions_from_links) 262 return null; 263 264 boolean is_AND_needed = !(isNoConditions_from_conditions || isNoConditions_frora_links); 265 266 return 267 <${ WHERE <( 268 Condit ions_f rom_condit ions )>< { 269 if(is_AND_needed) echo $" and "; }><( 270 Conditions_from links 271 )>>$>; 272 } 273 274 private OrderBy() :Text { 275 if ((orderby == null) || orderby.sql?.length)) == 0) 276 return null; 277 278 return <${ <(orderby)>}$>; 279 } 280 private Limit() :Text { 281 if ((limit == null) || limit.sql?.length() == 0)
264 return null; return <${ <(limit)>>$> .pattern 1 // ^Pattern: 2 package lib->db; 3 4 import java.util.Hashtable; 5 import nawaman~>text=>*; 6 7 pattern Table implements AsText { 8 9 in port name :Text; 10 in port fields sField* = new(); 11 in port links :Link* - new(); 12 13 14 private port theFields :Hashtable = new Hashtable(); 15 16 17 public new ( ) { 18 this.ensureName_NotEmpty();
19 > 20 public new (aName :Text) { 21 this(); 22 name := aName; 23 } 24 25 26 public asText() :CharSequence { 27 return name; 28 } 29 public newQuery() :Query { 30 Query aQuery = new Query(); 31 this.addToQuery(aQuery); 32 return aQuery; 33 } 34 public addToQuery(pQuery :Query) :void { 35 HasFields.AddAllFields(this, pQuery); 36 HasLinks .AddALlLinks (this, pQuery); 37 > 38 39 40 public addField(pFieldName :Text) :Field { 41 Field aField - this.getField(pFieldName); 42 43 if (aField !« null) 44 return aField; 45 46 aField = new Field(this, pFieldName); 47 48 fields <+= aField; 49 theFields.put(pFieldName?$, aField); 50 51 return aField; 52 > 53 public addField( 54 pFieldName :Text, 55 pAlias :Text) 56 :Field { 57 58 Field aField = this.addField(pFieldName); 59 aField.alias := pAlias; 60 61 return aField; 62 > 63 public getField(pFieldName :Text) :Field < 64 if (pFieldName == null) 65 return null; 66 67 return (Field)theFields.get(pFieldName??); 68 } 69 70
266 71 protected function freezeFieldsAndLinks() :void { 72 this.prepareFields!); 73 this.prepareLinks(); 74 75 f ields-»render(); 76 links -»render(); 77 } 78 protected function prepareFields() :void {> 79 protected function prepareLinks() :void {} 80 81 82 private ensureName_NotEmpty() :void { 83 assert(name): name?.length!) 1= 0; 84 ) 85 private ensureAllFields_BelongToThisTable() :void { 86 Table ThisTable • this; 87 fields *: filtered with ((SS.table «• ThisTable) || ($S.table «= null)); 88 > 89
267 III.II HTML Patterns
llb/html/Column.pattern // SPattern: package lib->html;
import nawaman->text=>*;
pattern Column {
port table :Table; port name :Text; port title :Text; port value :Text; port styler :Styler;
public new (pName :Text) { name := pName; this.assignTitle_DefaultValue(); }
private assignTitle_DefaultValue() :void { before!title): title - name; }
268 /html/HTMUFile.pattern 1 // §Pattern: 2 package lib~>html; 3 4 import nawaman->data=>*; 5 import nawaman~>text=>*; 6 7 pattern HTMLFile extends TextFile { e 9 port title :Text; 10 port body :Text; n port bodyStyler :Styler; 12 13 port urlPath :Text = <${<(Path)><((?Path:"/"))><(Name)><((?EXT:"."))><
22 > 23 24 25 public addCSSInclude(pCSSFile sText) :void { 26 headElements <+= <${}$>; 27 } 28 29 30 public prepareHead() :Text { 31 boolean hasContent = false; 32 Text aHead = 33 <${ }$> 34
<{ 35 if (title != null) { 36 hasContent - true; 37 38 echo $"\n"; 39 echo <${269 > protected applyBodyStyle(pBody :Text) :Text { if (bodyStyler 1= null) pBody - <${<(bodyStyler.style(pBody))>}$>; return pBody; )
protected prepareContent() :Text { Text aHead = this.prepareHead(); Text aBody » this.prepareBodyf); return <$< }$> <{ if (aHead 1= null) { echo $"\n"; echo <${<(aHead)>>$>; } >> echo $" "; echo <${<(aBodyAttribute)>}$>; > >»<{ if (aBody !•= null) { echo $"\n"; echo <${<(aBody)>}$>; > }>