White Paper for Workshop on New Visions for Software Design and Productivity Title: Value Based Software Engineering Barry Boehm and Dan Port, University of Southern California Kevin Sullivan, University of Virginia Abstract The primary thesis of Value-Based Software Engineering (VBSE) is that the integration of a software system’s stakeholder value propositions into the system’s definition, design, development, deployment, and evolution is critical to the system’s success. This white paper: · Analyzes the sources of software project failure in the Standish Report, and shows that many of the failed projects were caught in the vise of value- insensitive software engineering. · Discusses promising research ideas for improving our capability to perform VBSE. · Presents a roadmap for making progress toward VBSE and its resulting benefits. The white paper concludes with a summary of the relations between VBSE and other software research and applications areas. It is unavoidably involved with software and information system product and process technology, and their interaction with human values. It is strongly empirical, but includes new concepts in need of stronger theory. It uses risk considerations to balance software discipline and flexibility, and to answer other key “how much is enough?” questions. And it helps illuminate information technology policy decisions by identifying the quantitative and qualitative sources of cost and value associated with candidate decisions. Failed Software Projects: Sources and Remedies The 1995 CHAOS Report [Standish, 1995] surveyed several hundred software projects and found that only 16% of them were completed within their planned budget and schedule. The report analyzed the major sources of failure for the other projects, and found that eight problem sources accounted for 80% of the failures. Each of these sources is discussed below in terms of their relation to value-based approaches to software engineering. 1. Incomplete Requirements (13.1% of project failures) A major source of project overruns due to incomplete requirements is the neglect of critical off-nominal requirements. Empirical studies have shown that 80% of the software rework comes from 20% of the problems, and that many of these critical problems involve neglect of off- nominal requirements [Boehm-Basili, 2001]. An effective value-based technique for addressing these problems is risk management, particularly the assessment of the relative risk exposure (probability of loss times size of loss) of the various off-nominal operational scenarios [Boehm, 1989]. Other effective techniques involve goal-oriented approaches to software requirements [Dardenne et al., 1991; Young, 2001], in which goal elaboration provides a value oriented approach to identifying critical off-nominal or missing requirements. Further research in each of these areas would strengthen projects’ ability to avoid such overruns. 2. Lack of User Involvement (12.4%) We would generalize this to “lack of critical-stakeholder involvement,” as many system failures also result from lack of higher management, customer, maintainer, and interoperator, as well as user involvement. Here again, several approaches involving stakeholder value-based considerations have been developed and successfully applied, such as Participatory Design and Joint Application Development [Carmel et al., 1993], Quality Function Deployment [Eureka- Ryan, 1998], and stakeholder win-win [Boehm-Ross, 1989]. Further research has considerable potential in areas such as client-developer mutual requirements learning [Majchrzak-Beath, 2001] and multi-attribute tradeoff analysis [Keeney-Raiffa, 1993]. 3. Lack of Resources (10.6%) Software cost and schedule estimation models have saved many projects from overruns. But the pace of software engineering reinvention (COTS, open source, rapid development, agile development, iterative software process models) requires the software estimation field to be continually reinventing itself as well [Boehm et al., 2000, Chapter 6]. Other promising areas involve innovative critical-resource driven approaches such as the theory of constraints [Friedman-Leondes, 1969; Goldratt, 1990], timeboxing [McConnell, 1996] and the schedule/cost/quality as an independent variable process [Boehm-Brown, 2001]. 4. Unrealistic Expectations (9.9%) This can be considered the as the opposite side of the coin from “Lack of Resources”. Some additional value-based approaches for creating realistic expectations are business case analysis [Reifer, 2002], the DMR Group’s Benefits Realization Approach and its Results Chain method linking initiatives to contributions, assumptions, and outcomes [Thorp, 1998], and techniques for expectations management [Karten, 1994]. 5. Lack of Executive Support (9.3%) As discussed in point 2 above, lack of executive stakeholder support is often highly correlated with lack of executive involvement in formulating an initiative. Techniques such as Results Chain and business case analysis can also help here. 6. Changing Requirements (8.7%) A highly promising value-based approach here is Sullivan’s Strategic Design initiative, which uses techniques from real options theory [Amram-Kalutilaka, 1998] and the economic theory of modular design [Baldwin-Clark, 1999] to analyze the economic value of techniques for modular software design such as information hiding [Parnas, 1972] in facilitating adaptation to changing requirements [Sullivan et al., 2001]. 7. Lack of Planning (7.5%) This has a straightforward good-practices component, but value based planning approaches such as DMR’s Benefits Realization Approach avoid additional failures by identifying complementary initiatives (e.g., data collection, conversion, training, logistics) that must also be pursued to make the software application a success. 8. Absence of Need (7.5%) Here again, business case analysis, improved developer-client mutual learning, and stakeholder requirements prioritization capabilities can help avoid software “gold plating” phenomena. Software Economics Roadmap Value-based software engineering is a major component of the overall field of software economics. Our roadmap [Boehm-Sullivan, 2000] for the next major phase of research in software economics begins with the goal of developing fundamental knowledge that will enable the end objective of significant, measurable increase in the value created over time by software and information technology projects, products, portfolios and the industry. Working backwards from the end objective, we identify a network of important intermediate outcomes. The roadmap in Figure 1 illustrates these intermediate outcomes, dependence relationships among them, and important feedback paths by which models and analysis methods will be improved over time. The lower left part of the diagram captures tactical concerns, such as improving cost estimation for software projects, while the upper part captures strategic concerns, such as reasoning about real options and synergies between project and program elements of larger portfolios. Better macroeconomic data and models Better national- level strategic IT decision-making Market structures Better models of (R&D, policy) more favorable to sources of value Increased SW/IT in SW/IT including productivity options, synergies Better Software & competition Engineering Education Significantly and Better models of Better tactical and measurably greater links from SW/IT strategic SW/IT value created by product, process product, process SW/IT projects, & portfolio design portfolio design programs, portfolios to benefits created decision-making and industry Better SW/IT Better monitoring, Better models for project benefits dynamic analysis of estimating SW/IT realization mgmt. SW/IT products, benefits tracking processes, portfolios and Better data for environments estimating SW/ Better SW/IT IT benefits Better SW/IT project/portfolio system/portfolio status, valuation, business-case, & risk assessment Better data for payoff modeling estimating SW/ decision aids IT costs & schedule Better SW/IT Better models for project cost & estimating SW/IT schedule mgmt. costs & schedule tracking Figure 1: Roadmap for research in software engineering economics. Making Decisions that are Better for Value Creation The goal of our roadmap is supported by a key intermediate outcome: designers at all levels must make design decisions that are better for value added than those they make today. Design decisions are of the essence in product and process design, the structure and dynamic management of larger programs, the distribution of programs in a portfolio of strategic initiatives, and to national software policy. Better decision-making is the key enabler of greater value added. Design decision-making depends in turn on a set of other advances. First, the design space within which designers operate needs to be sufficiently rich. To some extent, the design space is determined by the technology market structure: what firms exist and what they produce. That structure is influenced, in turn, by a number of factors, including but not limited to national-level strategic decision-making, e.g., on long-term materials that are produced that designers can then employ, and their properties. Second, as a field we need to understand better the links between technical design mechanism (e.g. architecture), context, and value creation, to enable both better education and decision-making in any given situation. An improved understanding of these links depends on developing